WO2018003090A1 - Image processing device, image processing method, and program - Google Patents

Image processing device, image processing method, and program Download PDF

Info

Publication number
WO2018003090A1
WO2018003090A1 PCT/JP2016/069528 JP2016069528W WO2018003090A1 WO 2018003090 A1 WO2018003090 A1 WO 2018003090A1 JP 2016069528 W JP2016069528 W JP 2016069528W WO 2018003090 A1 WO2018003090 A1 WO 2018003090A1
Authority
WO
WIPO (PCT)
Prior art keywords
shine
area
image data
necessary information
information area
Prior art date
Application number
PCT/JP2016/069528
Other languages
French (fr)
Japanese (ja)
Inventor
満 西川
清人 小坂
Original Assignee
株式会社Pfu
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 株式会社Pfu filed Critical 株式会社Pfu
Priority to PCT/JP2016/069528 priority Critical patent/WO2018003090A1/en
Publication of WO2018003090A1 publication Critical patent/WO2018003090A1/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/67—Focus control based on electronic image sensor signals

Definitions

  • the present invention relates to an image processing apparatus, an image processing method, and a program.
  • Patent Document 1 a technique for correcting luminance unevenness in an image using an average value in a window of color information such as luminance in the image as a background is disclosed (see Patent Document 1).
  • a whiteout area in the photographed image is detected, and the whiteout area is used as a correction target area, and an area in another image corresponding to the correction target area is extracted and combined to generate an image without a whiteout area.
  • a technique is disclosed (see Patent Document 2).
  • Patent Document 1 Even when a plurality of images are combined, an area not including characters can be selected as a joint for combining images, that is, a boundary between images. However, there is a case where a joint that crosses the character portion is selected, and an image with an image quality inappropriate for OCR (Optical Character Recognition) due to blurring of the character may be acquired. Had.
  • OCR Optical Character Recognition
  • the present invention has been made in view of the above-described problems.
  • An image processing apparatus and an image processing apparatus that can acquire image data suitable for OCR or the like even under a situation where a subject is damaged by illumination such as a fluorescent lamp. It is an object to provide a method and a program.
  • an image processing apparatus includes a frame acquisition unit that acquires a captured frame, a subject region acquisition unit that acquires subject region image data of a subject region from the frame, Necessary information area acquisition means for detecting a necessary information area in the subject area and acquiring necessary information area image data of the necessary information area; Detecting means for detecting shine on the necessary information area image data; and And non-shiny area obtaining means for obtaining non-shiny area image data that is the necessary information area image data in which the shine has not been detected by the detecting means.
  • the image processing method includes a frame acquisition step of acquiring a captured frame, a subject region acquisition step of acquiring subject region image data of a subject region from the frame, and a necessary information region in the subject region.
  • the necessary information area acquisition step for detecting and acquiring necessary information area image data of the necessary information area
  • a shine detection step for detecting shine on the necessary information area image data
  • the shine detection in the shine detection step is a non-shiny area obtaining step of obtaining non-shiny area image data that is the necessary information area image data that has not been detected.
  • the program according to the present invention detects a frame acquisition step for acquiring a captured frame, a subject region acquisition step for acquiring subject region image data of a subject region from the frame, and a necessary information region in the subject region.
  • the necessary information area acquisition step for acquiring the necessary information area image data of the necessary information area the shine detection step for detecting shine on the necessary information area image data, and the shine detection in the shine detection step.
  • a non-shine area acquisition step of acquiring non-shine area image data that is the necessary information area image data that has not been included is executed by a computer.
  • a boundary that does not include a necessary area such as a character portion can be selected in advance as a joint for image composition, there is no factor that leads to image quality degradation of the character portion due to composition, and high OCR accuracy is realized. be able to.
  • OCR can be carried out even in an environment that could not be used conventionally, and the usage applications are expanded.
  • FIG. 1 is a block diagram illustrating an example of the configuration of the image processing apparatus according to the present embodiment.
  • FIG. 2 is a flowchart illustrating an example of processing in the image processing apparatus according to the present embodiment.
  • FIG. 3 is a diagram illustrating an example of a necessary information area in the present embodiment.
  • FIG. 4 is a diagram showing an example of necessary information area division in the present embodiment.
  • FIG. 5 is a diagram showing an example of necessary information area division in the present embodiment.
  • FIG. 6 is a diagram illustrating an example of necessary information area division in the present embodiment.
  • FIG. 7 is a diagram showing an example of necessary information area division in the present embodiment.
  • FIG. 8 is a diagram showing an example of necessary information area division in the present embodiment.
  • FIG. 9 is a diagram illustrating an example of image composition in the present embodiment.
  • FIG. 10 is a diagram illustrating an example of an OCR processing result in the present embodiment.
  • FIG. 1 is a block diagram illustrating an example of the configuration of the image processing apparatus 100 according to the present embodiment.
  • the embodiment described below exemplifies the image processing apparatus 100 for embodying the technical idea of the present invention, and is not intended to specify the present invention to the image processing apparatus 100.
  • the present invention is equally applicable to the image processing apparatus 100 of other embodiments included in the scope of claims.
  • the form of function distribution in the image processing apparatus 100 exemplified in the present embodiment is not limited to the following, and may be configured to be functionally or physically distributed / integrated in arbitrary units within a range where similar effects and functions can be achieved. can do.
  • the image processing apparatus 100 is, for example, portable information such as a tablet terminal, a mobile phone, a smartphone, a PHS, a PDA, a notebook personal computer, or a wearable computer such as a glasses type or a watch type. It may be a processing device (mobile terminal).
  • the image processing apparatus 100 is generally configured to include a control unit 102, a storage unit 106, a photographing unit 110, an input / output unit 112, a sensor unit 114, and a communication unit 116. .
  • the image processing apparatus 100 is illustrated as a mobile terminal including a photographing unit 110 in a housing. However, the image processing apparatus 100 does not include the photographing unit 110 in a housing, and captures captured image data from an external photographing device. It may be configured to receive (for example, a desktop personal computer).
  • an input / output interface unit (not shown) for connecting the input / output unit 112 and the control unit 102 may be further provided.
  • Each unit of the image processing apparatus 100 is connected to be communicable via an arbitrary communication path.
  • the communication unit 116 is a network interface (NIC (Network Interface Controller), etc.), Bluetooth (registered trademark), infrared communication, etc. for transmitting and receiving IP data by wired communication and / or wireless communication (WiFi, etc.) May be an interface for performing wireless communication.
  • NIC Network Interface Controller
  • Bluetooth registered trademark
  • WiFi wireless communication
  • the image processing apparatus 100 may be communicably connected to an external apparatus via a network using the communication unit 116.
  • the sensor unit 114 detects a physical quantity and converts it into a signal (digital signal) of another medium.
  • the sensor unit 114 includes a proximity sensor, a direction sensor, a magnetic field sensor, a linear acceleration sensor, a luminance sensor, a gyro sensor, a pressure sensor, a gravity sensor, an acceleration sensor, an atmospheric pressure sensor, and / or a temperature sensor. Also good.
  • the input / output unit 112 performs data input / output (I / O).
  • the input / output unit 112 may be, for example, a key input unit, a touch panel, a control pad (for example, a touch pad and a game pad), a mouse, a keyboard, and / or a microphone.
  • the input / output unit 112 may be a display unit that displays a display screen of an application or the like (for example, a display, a monitor, a touch panel, or the like configured by liquid crystal or organic EL).
  • a display screen of an application or the like for example, a display, a monitor, a touch panel, or the like configured by liquid crystal or organic EL.
  • the input / output unit 112 may be an audio output unit (for example, a speaker or the like) that outputs audio information as audio.
  • the input / output unit (touch panel) 112 may include a sensor unit 114 that detects physical contact and converts it into a signal (digital signal).
  • the image capturing unit 110 acquires still image data by capturing a still image of a subject (for example, a form or the like).
  • the imaging unit 110 may acquire captured image data.
  • the photographing unit 110 may acquire continuous (moving image) image data (frames) by continuously capturing images (moving image capturing) of the subject.
  • the imaging unit 110 may acquire video data.
  • the imaging unit 110 may acquire ancillary data.
  • the frame may be non-compressed image data.
  • the frame may be high-resolution image data.
  • the high resolution may be full high vision, 4K resolution, super high vision (8K resolution), or the like.
  • the photographing unit 110 may shoot moving images at 24 fps or 30 fps.
  • the image capturing unit 110 may be a camera including an image sensor such as a CCD (Charge Coupled Device) and / or a CMOS (Complementary Metal Oxide Semiconductor).
  • the storage unit 106 is storage means, for example, a memory such as RAM / ROM, a fixed disk device such as a hard disk, an SSD (Solid State Drive), and / or a tangible storage device such as an optical disk, or a storage circuit. Can be used.
  • a memory such as RAM / ROM
  • a fixed disk device such as a hard disk, an SSD (Solid State Drive)
  • a tangible storage device such as an optical disk, or a storage circuit. Can be used.
  • the storage unit 106 stores various databases, tables, buffers, and / or files (necessary information area file 106a, image data file 106b, etc.).
  • the storage unit 106 may store a computer program or the like for giving a command to a CPU (Central Processing Unit) and performing various processes.
  • CPU Central Processing Unit
  • the necessary information area file 106a stores boundary data of the necessary information area in the subject area.
  • the necessary information area may be an image corresponding to the whiteboard (entire).
  • the necessary information area may be an area where necessary information can be visually recognized in the subject area.
  • the area where necessary information can be visually recognized may be an area including letters, numbers, symbols, figures, photographs, and / or seals.
  • the subject area may be a document image of a document (form) included in a read image based on a frame.
  • the form may be a prescribed form such as various licenses including a driver's license, various identification cards, or a health insurance card.
  • the necessary information area file 106a stores boundary data of boundaries that are joints of necessary information areas in advance for a known document that is a subject.
  • the image data file 106b stores image data (such as a frame).
  • the image data file 106b includes subject area image data, necessary information area image data, non-shiny area image data, composite image data, divided area image data, non-shiny divided area image data, captured image data, and / or Document image data may be stored.
  • the control unit 102 is a CPU, GPU (Graphics Processing Unit), DSP (Digital Signal Processor), LSI (Large Scale Integration / Integration Specified and ASIC), which controls the image processing apparatus 100 in an integrated manner.
  • it may be composed of a tangible controller including a FPGA (Field-Programming Gate Array) or a control circuit.
  • control unit 102 has an internal memory for storing a control program, a program defining various processing procedures, and necessary data, and performs information processing for executing various processes based on these programs. Do.
  • control unit 102 functionally conceptually includes a frame acquisition unit 102a, a subject region acquisition unit 102b, a necessary information region acquisition unit 102c, a shine detection unit 102d, a non-shine region acquisition unit 102e, a divided region acquisition unit 102f, and an image.
  • a synthesis unit 102g, a division synthesis unit 102h, and an OCR unit 102i are provided.
  • the frame acquisition unit 102a acquires a frame (captured image data of a captured image).
  • the frame acquisition unit 102a may acquire a frame imaged by the imaging unit 110 or an external imaging device.
  • the frame acquisition unit 102a may acquire captured image data of a captured image including a subject area.
  • the frame acquisition unit 102a may acquire still image data by controlling still image shooting by the shooting unit 110.
  • the frame acquisition unit 102a may acquire a frame corresponding to one frame by controlling continuous image shooting or moving image shooting by the shooting unit 110.
  • the subject area acquisition unit 102b acquires subject area image data of the subject area from the frame.
  • the subject area acquisition unit 102b may acquire document image data of a document image from a frame.
  • the necessary information area acquisition unit 102c detects a necessary information area in the subject area and acquires necessary information area image data of the necessary information area.
  • the necessary information area acquisition unit 102c may detect the necessary information area based on the boundary data of the necessary information area stored in the necessary information area file 106a and acquire the necessary information area image data.
  • the shine detection unit 102d detects shine on the necessary information area image data.
  • the shine detection unit 102d may detect shine on each divided region image data.
  • the shine detection unit 102d may detect the shine based on a comparison between the luminance of the necessary information area image data and a predetermined threshold value.
  • the non-shine area acquisition unit 102e acquires non-shine area image data that is necessary information area image data in which no shine is detected by the shine detection section 102d.
  • the non-shine area acquisition unit 102e may acquire non-shine division area image data that is division area image data in which no shine has been detected by the shine detection section 102d.
  • the divided area acquisition unit 102f detects a non-character area by character detection processing on the necessary information area image data in which the shine is detected by the shine detection unit 102d, and divides the necessary information area into the non-character areas. Get the data.
  • the divided region acquisition unit 102f detects the non-character region by the edge detection method for the necessary information region image data in which the shine is detected by the shine detection unit 102d, and acquires the divided region image data of the divided region. Also good.
  • the image composition unit 102g acquires composite image data obtained by combining a plurality of non-shine area image data acquired by the non-shine area acquisition unit 102e.
  • the image composition unit 102g may acquire composite image data obtained by combining the non-shiny region image data and the region external image data of the subject region excluding the necessary information region.
  • the image composition unit 102g may acquire composite image data obtained by combining a plurality of non-shine area image data.
  • the division synthesis unit 102h acquires non-shine area image data by synthesizing the non-shine division area image data acquired by the non-shine area acquisition unit 102e.
  • the OCR unit 102i performs OCR processing on the image data and acquires character data.
  • the OCR unit 102i may perform the OCR process on the non-shine area image data to acquire character data.
  • the OCR unit 102i may perform character image data by performing OCR processing on the composite image data.
  • FIG. 2 is a flowchart illustrating an example of processing in the image processing apparatus 100 according to the present embodiment.
  • the frame acquisition unit 102a initializes the setting of the imaging unit 110 according to the subject, controls the start of moving image shooting by the imaging unit 110, and acquires a frame (step SA-1). .
  • moving image shooting of the subject by the camera device is started.
  • the subject area acquisition unit 102b detects a subject area in the captured image based on the frame, acquires subject area image data of the subject area, and stores (records) it in the image data file 106b (step SA-2).
  • the subject area in the captured image is detected during moving image shooting.
  • the detection of the subject region in the captured image may be performed using processing such as edge detection and / or feature point detection.
  • the necessary information area acquisition unit 102c detects the necessary information area in the subject area and acquires the necessary information area image data of the necessary information area (step SA-3).
  • FIG. 3 is a diagram illustrating an example of a necessary information area in the present embodiment.
  • area 1 name
  • area 2 date of birth
  • area 3 address
  • area 4 delivery date
  • area 5 expiration date
  • the area 6 license number
  • the area 7 face photograph
  • the dotted line portion may be a joint (boundary).
  • a region that is not covered by the necessary information region in the subject image may be outside the region.
  • the outside of the area may be specified from the subject image acquired last before the image composition.
  • the shine detection unit 102d detects the shine for each necessary information area image data based on the comparison between the luminance of the necessary information area image data and a predetermined threshold (step SA-4).
  • the necessary information area without the shine in the subject is specified by detecting the shine in the subject area.
  • the detection of the shine in the subject area it may be determined that the shine has been detected when the average value in the window> 200 is established as the threshold value determination of the background estimation value.
  • the non-shine area acquisition unit 102e acquires non-shine area image data, which is necessary information area image data in which no shine has been detected by the shine detection section 102d, and records it in the image data file 106b (step SA-5). .
  • image data of a necessary information area without shine is recorded.
  • FIG. 4 and FIG. 5 are diagrams showing an example of necessary information area division in the present embodiment.
  • the necessary information area when only a part of the necessary information area is detected when the shine is detected, the necessary information area may be divided to acquire the non-shine area image data.
  • the necessary information area A that is the date of driver license is divided into a divided area B and a divided area C as shown in FIG. 5. .
  • the boundary between the divided region B and the divided region C becomes a new joint, but character detection is performed so that the joint does not include a character, and it is determined that the character is not a character.
  • a non-character area may be set as a joint for division.
  • an edge detection method or the like may be used as a character detection method.
  • the non-shiny divided region image data is obtained for each of the divided region B and the divided region C, and the non-shiny region image data of the necessary information region A is obtained by recombining. May be.
  • FIGS. 6 to 8 are diagrams showing an example of necessary information area division in the present embodiment.
  • an image captured using a whiteboard as a subject may be detected as the necessary information area D.
  • the division method of the necessary information area D is used.
  • the boundary between the divided area E and the divided area F may be set as a joint under the character string “> Agenda” where no edge is detected.
  • the gloss of the whiteboard reflects the office lighting well, and there is a problem that it is greatly affected by the shine, but by utilizing this embodiment, the whiteboard that avoids the influence of the shine is a problem.
  • the acquisition of non-shine area image data by scanning is realized.
  • the whiteboard on which the contents of the proceedings are recorded is taken with a mobile camera and OCR is performed. Since it can be converted into text data, efficiency in office work such as meeting minutes is promoted.
  • the non-shine area acquisition unit 102e determines whether non-shine area image data of all necessary information areas included in the subject area has been acquired (step SA-6).
  • step SA-2 to step SA-5 is repeated to determine whether image data of the necessary information area without the shine of the entire subject has been acquired.
  • step SA-6 determines that non-shine area image data of all necessary information areas included in the subject area has not been acquired. If the non-shine area acquisition unit 102e determines that non-shine area image data of all necessary information areas included in the subject area has not been acquired (step SA-6: No), the process proceeds to step SA-2. To migrate.
  • step SA-6 determines that non-shine area image data of all necessary information areas included in the subject area has been acquired.
  • the image composition unit 102g obtains composite image data obtained by combining the non-shiny region image data of all necessary information regions included in the subject region and the region external image data of the subject region excluding the necessary information region ( Step SA-7).
  • the images are combined to create a subject image without shine.
  • FIG. 9 is a diagram illustrating an example of image composition in the present embodiment.
  • the necessary information area image data in which the shine of both image data is not detected may be recorded as the non-shine area image data.
  • the OCR unit 102i performs OCR processing on the composite image data to acquire character data, and records the character data in the image data file 106b in association with the subject area image data (step SA-8). ), The process is terminated.
  • a highly accurate OCR success rate can be achieved by performing the OCR process on the acquired composite image data.
  • FIG. 10 is a diagram illustrating an example of an OCR processing result in the present embodiment.
  • the OCR process is performed on the synthesized image data synthesized in FIG. 9, so that the OCR process is performed on the shining image data itself.
  • high OCR accuracy is realized.
  • the personal information (name, date of birth, address, delivery) described in the driver's license is obtained by scanning the driver's license with a mobile camera and performing OCR processing. Date and license number, etc.).
  • processing such as personal authentication by comparing character data acquired by OCR processing with a personal information database can be easily performed.
  • an information area for example, a text field or a face photograph
  • a known document type such as various licenses or invoices.
  • the boundary of the necessary information area is used as a joint for a document type known in advance.
  • noise components may be mixed into the text portion, which may cause deterioration in OCR accuracy.
  • the text portion is used as a synthesis joint. By not doing so, high OCR accuracy is achieved.
  • the OCR process can be performed on the synthetic image data without the shine using the shine detection, the OCR process avoiding the adverse effect due to the shine is realized, and the character recognition has a high accuracy. Can be achieved.
  • image data without shine can be quickly synthesized by combining shine detection and moving image processing technology.
  • the shine detection is executed for each necessary information area in the subject area in the photographed image, it is not necessary to repeat photographing until the entire subject area is not shining.
  • the number of retries during moving image shooting can be reduced, and quick OCR processing can be performed.
  • document media such as various licenses or health insurance cards can be easily taken and applied to OCR, it can be used as a means of personal authentication, so that contract procedures at commercial counters or stores can be simplified. It is becoming.
  • a technique for dealing with shine a technique for dealing with shine using the characteristics of a mobile camera that allows a mobile camera to shoot a movie while moving the shooting position is disclosed.
  • a partial image that does not have shine is extracted and synthesized by including a necessary content, so that an image of a subject image without shine is obtained.
  • Data generation is realized.
  • OCR processing is performed on the composite image data to achieve high accuracy of OCR.
  • retry processing is practically performed by detecting shine before OCR processing, so that, in the case of an image that has been lost, OCR execution is rejected, and until an image without shine is obtained, photographing is requested again. It was.
  • the OCR when performing OCR quickly, even if shine occurs in a part of the image, the OCR is performed on a portion that is not shining, so that photographing is performed as much as possible.
  • the number of retries can be reduced.
  • the image processing apparatus 100 may perform processing in a stand-alone form, performs processing in response to a request from a client terminal (which is a separate housing from the image processing apparatus 100), and the processing result is You may make it return to a client terminal.
  • a client terminal which is a separate housing from the image processing apparatus 100
  • all or a part of the processes described as being automatically performed can be manually performed, or all of the processes described as being manually performed can be performed.
  • a part can be automatically performed by a known method.
  • processing procedure, control procedure, specific name, information including parameters such as registration data or search conditions for each processing, screen examples, or database configuration shown in the description and drawings are specially noted. It can be changed arbitrarily except for.
  • the illustrated components are functionally conceptual, and need not be physically configured as illustrated.
  • each device of the image processing apparatus 100 particularly the processing functions performed by the control unit 102, are executed by the CPU and a program interpreted and executed by the CPU. You may implement
  • the program is recorded on a non-transitory computer-readable recording medium including a programmed instruction for causing a computer to execute the method according to the present invention, which will be described later, and an image processing apparatus as necessary.
  • 100 mechanically read. That is, in the storage unit 106 such as a ROM or an HDD, computer programs for performing various processes by giving instructions to the CPU in cooperation with an OS (Operating System) are recorded. This computer program is executed by being loaded into the RAM, and constitutes a control unit in cooperation with the CPU.
  • OS Operating System
  • the computer program may be stored in an application program server connected to the image processing apparatus 100 via an arbitrary network, and may be downloaded in whole or in part as necessary. is there.
  • the program according to the present invention may be stored in a computer-readable recording medium, or may be configured as a program product.
  • the “recording medium” includes a memory card, USB memory, SD card, flexible disk, magneto-optical disk, ROM, EPROM, EEPROM, CD-ROM, MO, DVD, and Blu-ray (registered trademark). It includes any “portable physical medium” such as Disc.
  • program is a data processing method described in an arbitrary language or description method, and may be in any form such as source code or binary code. Note that the “program” is not necessarily limited to a single configuration, and functions are achieved in cooperation with a separate configuration such as a plurality of modules and libraries or a separate program represented by the OS. Including things. In addition, a well-known structure and procedure can be used about the specific structure for reading a recording medium in each apparatus shown in embodiment, a reading procedure, or the installation procedure after reading.
  • Various databases and the like stored in the storage unit 106 are storage means such as a memory device such as a RAM or a ROM, a fixed disk device such as a hard disk, a flexible disk, and / or an optical disk.
  • Various programs, tables, databases, and / or web page files used may be stored.
  • the image processing apparatus 100 may be configured as an information processing apparatus such as a known personal computer, or may be configured by connecting an arbitrary peripheral device to the information processing apparatus.
  • the image processing apparatus 100 may be realized by installing software (including programs, data, and the like) that causes the information processing apparatus to realize the method of the present invention.
  • the specific form of distribution / integration of the devices is not limited to that shown in the figure, and all or a part of them may be functional or physical in arbitrary units according to various additions or according to functional loads. Can be distributed and integrated. That is, the above-described embodiments may be arbitrarily combined and may be selectively implemented.
  • the image processing apparatus, the image processing method, and the program can be implemented in many industrial fields, particularly in the image processing field that handles images read by a camera, and are extremely useful.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Studio Devices (AREA)

Abstract

In the present invention a photographed frame is acquired, photographic subject region image data for a photographic subject region is acquired from the frame, a required information region in the photographic subject region is detected, required information region image data for the required information region is acquired, glare with respect to the required information region image data is detected, and non-glare region image data, which is required information region image data for which glare has not been detected, is acquired.

Description

画像処理装置、画像処理方法、および、プログラムImage processing apparatus, image processing method, and program
　本発明は、画像処理装置、画像処理方法、および、プログラムに関する。 The present invention relates to an image processing apparatus, an image processing method, and a program.
　従来から、画像データのテカリを補正する技術が開示されている。 Conventionally, techniques for correcting the shine of image data have been disclosed.
　ここで、画像内の輝度等の色情報の窓内平均値を背景とし、画像内の輝度ムラを補正する技術が開示されている（特許文献１を参照）。 Here, a technique for correcting luminance unevenness in an image using an average value in a window of color information such as luminance in the image as a background is disclosed (see Patent Document 1).
　また、撮影画像内の白飛び領域を検出し、白飛び領域を修正対象領域として、修正対象領域に相当する他の画像における領域を抽出し、合成することで白飛び領域の無い画像を生成する技術が開示されている（特許文献２を参照）。 Also, a whiteout area in the photographed image is detected, and the whiteout area is used as a correction target area, and an area in another image corresponding to the correction target area is extracted and combined to generate an image without a whiteout area. A technique is disclosed (see Patent Document 2).
特表２０１５－５０３８１３号公報JP-T-2015-503813 特開２０１３－２２９６９８号公報JP 2013-229698 A
　しかしながら、従来の画像処理装置（特許文献１等）においては、複数の画像を画像合成する場合であっても、文字を含まない領域を画像合成の繋ぎ目、すなわち、画像間の境界として選択できるものではないため、文字部分を横切るような繋ぎ目が選択されることがあり、文字のボケ等によるＯＣＲ（Ｏｐｔｉｃａｌ　Ｃｈａｒａｃｔｅｒ　Ｒｅｃｏｇｎｉｔｉｏｎ）に不適切な画質の画像が取得されてしまうことがあるという問題点を有していた。 However, in the conventional image processing apparatus (Patent Document 1, etc.), even when a plurality of images are combined, an area not including characters can be selected as a joint for combining images, that is, a boundary between images. However, there is a case where a joint that crosses the character portion is selected, and an image with an image quality inappropriate for OCR (Optical Character Recognition) due to blurring of the character may be acquired. Had.
　本発明は、上記問題点に鑑みてなされたもので、蛍光灯などの照明により被写体がテカってしまう状況下でも、ＯＣＲ等に適した画像データを取得することができる画像処理装置、画像処理方法、および、プログラムを提供することを目的とする。 The present invention has been made in view of the above-described problems. An image processing apparatus and an image processing apparatus that can acquire image data suitable for OCR or the like even under a situation where a subject is damaged by illumination such as a fluorescent lamp. It is an object to provide a method and a program.
　このような目的を達成するため、本発明に係る画像処理装置は、撮影されたフレームを取得するフレーム取得手段と、前記フレームから被写体領域の被写体領域画像データを取得する被写体領域取得手段と、前記被写体領域における必要情報領域を検出し、前記必要情報領域の必要情報領域画像データを取得する必要情報領域取得手段と、前記必要情報領域画像データに対してテカリを検出するテカリ検出手段と、前記テカリ検出手段により前記テカリが検出されなかった前記必要情報領域画像データである非テカリ領域画像データを取得する非テカリ領域取得手段と、を備えたことを特徴とする。 In order to achieve such an object, an image processing apparatus according to the present invention includes a frame acquisition unit that acquires a captured frame, a subject region acquisition unit that acquires subject region image data of a subject region from the frame, Necessary information area acquisition means for detecting a necessary information area in the subject area and acquiring necessary information area image data of the necessary information area; Detecting means for detecting shine on the necessary information area image data; and And non-shiny area obtaining means for obtaining non-shiny area image data that is the necessary information area image data in which the shine has not been detected by the detecting means.
　また、本発明に係る画像処理方法は、撮影されたフレームを取得するフレーム取得ステップと、前記フレームから被写体領域の被写体領域画像データを取得する被写体領域取得ステップと、前記被写体領域における必要情報領域を検出し、前記必要情報領域の必要情報領域画像データを取得する必要情報領域取得ステップと、前記必要情報領域画像データに対してテカリを検出するテカリ検出ステップと、前記テカリ検出ステップにて前記テカリが検出されなかった前記必要情報領域画像データである非テカリ領域画像データを取得する非テカリ領域取得ステップと、を含むことを特徴とする。 The image processing method according to the present invention includes a frame acquisition step of acquiring a captured frame, a subject region acquisition step of acquiring subject region image data of a subject region from the frame, and a necessary information region in the subject region. The necessary information area acquisition step for detecting and acquiring necessary information area image data of the necessary information area, a shine detection step for detecting shine on the necessary information area image data, and the shine detection in the shine detection step. A non-shiny area obtaining step of obtaining non-shiny area image data that is the necessary information area image data that has not been detected.
　また、本発明に係るプログラムは、撮影されたフレームを取得するフレーム取得ステップと、前記フレームから被写体領域の被写体領域画像データを取得する被写体領域取得ステップと、前記被写体領域における必要情報領域を検出し、前記必要情報領域の必要情報領域画像データを取得する必要情報領域取得ステップと、前記必要情報領域画像データに対してテカリを検出するテカリ検出ステップと、前記テカリ検出ステップにて前記テカリが検出されなかった前記必要情報領域画像データである非テカリ領域画像データを取得する非テカリ領域取得ステップと、をコンピュータに実行させることを特徴とする。 The program according to the present invention detects a frame acquisition step for acquiring a captured frame, a subject region acquisition step for acquiring subject region image data of a subject region from the frame, and a necessary information region in the subject region. The necessary information area acquisition step for acquiring the necessary information area image data of the necessary information area, the shine detection step for detecting shine on the necessary information area image data, and the shine detection in the shine detection step. A non-shine area acquisition step of acquiring non-shine area image data that is the necessary information area image data that has not been included is executed by a computer.
　この発明によれば、画像合成の繋ぎ目として予め文字部分等の必要な領域が含まれない境界を選択できるため、合成による文字部分等の画質劣化に繋がる要因がなく、高いＯＣＲ精度を実現することができる。 According to the present invention, since a boundary that does not include a necessary area such as a character portion can be selected in advance as a joint for image composition, there is no factor that leads to image quality degradation of the character portion due to composition, and high OCR accuracy is realized. be able to.
　また、この発明によれば、オフィス内の蛍光灯などの照明により、被写体がテカってしまう状況下でも、ＯＣＲ実施向けの画像データの取得が可能となる。 Further, according to the present invention, it is possible to acquire image data for performing OCR even under a situation where the subject is taken up by illumination such as a fluorescent lamp in the office.
　このため、この発明によれば、従来使用できなかった環境下においてもＯＣＲが実施することができるようになり、利用用途が拡大する。 For this reason, according to the present invention, OCR can be carried out even in an environment that could not be used conventionally, and the usage applications are expanded.
　また、この発明によれば、被写体がテカってしまう状況で、テカリの無い画像データの取得ができるまで動画像撮影を繰り返す必要性がなくなるため、処理が完了するまでの時間を短縮することができる。 In addition, according to the present invention, it is not necessary to repeat moving image shooting until the image data without shining can be acquired in a situation where the subject is shining, so the time until the processing is completed can be shortened. it can.
　また、この発明によれば、取得された部分画像に対して必ずしも合成処理を施さなくても、ユーザが必要とする被写体に記載された情報を取得することができる。 Further, according to the present invention, it is possible to acquire the information described in the subject required by the user without necessarily performing the synthesis process on the acquired partial image.
図１は、本実施形態に係る画像処理装置の構成の一例を示すブロック図である。FIG. 1 is a block diagram illustrating an example of the configuration of the image processing apparatus according to the present embodiment. 図２は、本実施形態の画像処理装置における処理の一例を示すフローチャートである。FIG. 2 is a flowchart illustrating an example of processing in the image processing apparatus according to the present embodiment. 図３は、本実施形態における必要情報領域の一例を示す図である。FIG. 3 is a diagram illustrating an example of a necessary information area in the present embodiment. 図４は、本実施形態における必要情報領域分割の一例を示す図である。FIG. 4 is a diagram showing an example of necessary information area division in the present embodiment. 図５は、本実施形態における必要情報領域分割の一例を示す図である。FIG. 5 is a diagram showing an example of necessary information area division in the present embodiment. 図６は、本実施形態における必要情報領域分割の一例を示す図である。FIG. 6 is a diagram illustrating an example of necessary information area division in the present embodiment. 図７は、本実施形態における必要情報領域分割の一例を示す図である。FIG. 7 is a diagram showing an example of necessary information area division in the present embodiment. 図８は、本実施形態における必要情報領域分割の一例を示す図である。FIG. 8 is a diagram showing an example of necessary information area division in the present embodiment. 図９は、本実施形態における画像合成の一例を示す図である。FIG. 9 is a diagram illustrating an example of image composition in the present embodiment. 図１０は、本実施形態におけるＯＣＲ処理結果の一例を示す図である。FIG. 10 is a diagram illustrating an example of an OCR processing result in the present embodiment.
　以下に、本発明に係る画像処理装置、画像処理方法、および、プログラムの実施形態を図面に基づいて詳細に説明する。なお、この実施形態により本発明が限定されるものではない。 Hereinafter, embodiments of an image processing device, an image processing method, and a program according to the present invention will be described in detail with reference to the drawings. In addition, this invention is not limited by this embodiment.
［本実施形態の構成］
　以下、本発明の実施形態に係る画像処理装置１００の構成の一例について図１を参照して説明し、その後、本実施形態の処理等について詳細に説明する。図１は、本実施形態に係る画像処理装置１００の構成の一例を示すブロック図である。 [Configuration of this embodiment]
Hereinafter, an example of the configuration of the image processing apparatus 100 according to the embodiment of the present invention will be described with reference to FIG. 1, and then the processing and the like of the present embodiment will be described in detail. FIG. 1 is a block diagram illustrating an example of the configuration of the image processing apparatus 100 according to the present embodiment.
　但し、以下に示す実施形態は、本発明の技術思想を具体化するための画像処理装置１００を例示するものであって、本発明をこの画像処理装置１００に特定することを意図するものではなく、請求の範囲に含まれるその他の実施形態の画像処理装置１００にも等しく適用し得るものである。 However, the embodiment described below exemplifies the image processing apparatus 100 for embodying the technical idea of the present invention, and is not intended to specify the present invention to the image processing apparatus 100. The present invention is equally applicable to the image processing apparatus 100 of other embodiments included in the scope of claims.
　また、本実施形態で例示する画像処理装置１００における機能分散の形態は以下に限られず、同様の効果や機能を奏し得る範囲において、任意の単位で機能的または物理的に分散・統合して構成することができる。 In addition, the form of function distribution in the image processing apparatus 100 exemplified in the present embodiment is not limited to the following, and may be configured to be functionally or physically distributed / integrated in arbitrary units within a range where similar effects and functions can be achieved. can do.
　ここで、画像処理装置１００は、例えば、タブレット端末、携帯電話、スマートフォン、ＰＨＳ、ＰＤＡ、ノート型のパーソナルコンピュータ、または、メガネ型もしくは時計型などのウェアラブルコンピュータ等の可搬性を有する携帯型の情報処理装置（モバイル端末）であってもよい。 Here, the image processing apparatus 100 is, for example, portable information such as a tablet terminal, a mobile phone, a smartphone, a PHS, a PDA, a notebook personal computer, or a wearable computer such as a glasses type or a watch type. It may be a processing device (mobile terminal).
　まず、図１に示すように、画像処理装置１００は、概略的に、制御部１０２と記憶部１０６と撮影部１１０と入出力部１１２とセンサ部１１４と通信部１１６とを備えて構成される。 First, as shown in FIG. 1, the image processing apparatus 100 is generally configured to include a control unit 102, a storage unit 106, a photographing unit 110, an input / output unit 112, a sensor unit 114, and a communication unit 116. .
　なお、図１において、画像処理装置１００は、筐体内に撮影部１１０を備えたモバイル端末として示しているが、筐体内に撮影部１１０を備えておらず、外部の撮影装置から撮影画像データを受信する構成（例えば、デスクトップ型のパーソナルコンピュータ等）であってもよい。 In FIG. 1, the image processing apparatus 100 is illustrated as a mobile terminal including a photographing unit 110 in a housing. However, the image processing apparatus 100 does not include the photographing unit 110 in a housing, and captures captured image data from an external photographing device. It may be configured to receive (for example, a desktop personal computer).
　ここで、図１では省略しているが、本実施形態において、更に、入出力部１１２と制御部１０２とを接続する入出力インターフェース部（図示せず）を備えていてもよい。これら画像処理装置１００の各部は任意の通信路を介して通信可能に接続されている。 Here, although omitted in FIG. 1, in this embodiment, an input / output interface unit (not shown) for connecting the input / output unit 112 and the control unit 102 may be further provided. Each unit of the image processing apparatus 100 is connected to be communicable via an arbitrary communication path.
　ここで、通信部１１６は、有線通信および／または無線通信（ＷｉＦｉ等）によりＩＰデータを送受信するためのネットワークインターフェース（ＮＩＣ（Ｎｅｔｗｏｒｋ　Ｉｎｔｅｒｆａｃｅ　Ｃｏｎｔｒｏｌｌｅｒ）等）、Ｂｌｕｅｔｏｏｔｈ（登録商標）、または、赤外線通信等によって無線通信を行うインターフェースであってもよい。 Here, the communication unit 116 is a network interface (NIC (Network Interface Controller), etc.), Bluetooth (registered trademark), infrared communication, etc. for transmitting and receiving IP data by wired communication and / or wireless communication (WiFi, etc.) May be an interface for performing wireless communication.
　ここで、画像処理装置１００は、通信部１１６を用いて、ネットワークを介して外部装置と通信可能に接続されていてもよい。 Here, the image processing apparatus 100 may be communicably connected to an external apparatus via a network using the communication unit 116.
　また、センサ部１１４は、物理量を検出して別媒体の信号（デジタル信号）に変換する。ここで、センサ部１１４は、近接センサ、方角センサ、磁場センサ、直線加速センサ、輝度センサ、ジャイロセンサ、圧力センサ、重力センサ、加速度センサ、気圧センサ、および／または、温度センサ等を含んでいてもよい。 Also, the sensor unit 114 detects a physical quantity and converts it into a signal (digital signal) of another medium. Here, the sensor unit 114 includes a proximity sensor, a direction sensor, a magnetic field sensor, a linear acceleration sensor, a luminance sensor, a gyro sensor, a pressure sensor, a gravity sensor, an acceleration sensor, an atmospheric pressure sensor, and / or a temperature sensor. Also good.
　また、入出力部１１２は、データの入出力（Ｉ／Ｏ）を行う。ここで、入出力部１１２は、例えば、キー入力部、タッチパネル、コントロールパッド（例えば、タッチパッド、および、ゲームパッド等）、マウス、キーボード、および／または、マイク等であってもよい。 Also, the input / output unit 112 performs data input / output (I / O). Here, the input / output unit 112 may be, for example, a key input unit, a touch panel, a control pad (for example, a touch pad and a game pad), a mouse, a keyboard, and / or a microphone.
　また、入出力部１１２は、アプリケーション等の表示画面を表示する表示部（例えば、液晶または有機ＥＬ等から構成されるディスプレイ、モニタ、または、タッチパネル等）であってもよい。 Also, the input / output unit 112 may be a display unit that displays a display screen of an application or the like (for example, a display, a monitor, a touch panel, or the like configured by liquid crystal or organic EL).
　また、入出力部１１２は、音声情報を音声として出力する音声出力部（例えば、スピーカ等）であってもよい。また、入出力部（タッチパネル）１１２は、物理的接触を検出し、信号（デジタル信号）に変換するセンサ部１１４を含んでいてもよい。 Also, the input / output unit 112 may be an audio output unit (for example, a speaker or the like) that outputs audio information as audio. The input / output unit (touch panel) 112 may include a sensor unit 114 that detects physical contact and converts it into a signal (digital signal).
　また、撮影部１１０は、被写体（例えば、帳票等）を静止画撮影することで、静止画の画像データを取得する。例えば、撮影部１１０は、撮影画像データを取得してもよい。 Also, the image capturing unit 110 acquires still image data by capturing a still image of a subject (for example, a form or the like). For example, the imaging unit 110 may acquire captured image data.
　また、撮影部１１０は、被写体を連続画像撮影（動画撮影）することで、連続（動画）の画像データ（フレーム）を取得してもよい。例えば、撮影部１１０は、映像データを取得してもよい。また、撮影部１１０は、アンシラリデータを取得してもよい。 Further, the photographing unit 110 may acquire continuous (moving image) image data (frames) by continuously capturing images (moving image capturing) of the subject. For example, the imaging unit 110 may acquire video data. The imaging unit 110 may acquire ancillary data.
　ここで、フレームは、非圧縮の画像データであってもよい。また、フレームは、高解像度の画像データであってもよい。ここで、高解像度とは、フルハイビジョン、４Ｋ解像度、または、スーパーハイビジョン（８Ｋ解像度）等であってもよい。 Here, the frame may be non-compressed image data. The frame may be high-resolution image data. Here, the high resolution may be full high vision, 4K resolution, super high vision (8K resolution), or the like.
　また、撮影部１１０は、２４ｆｐｓまたは３０ｆｐｓ等で動画撮影してもよい。ここで、撮影部１１０は、ＣＣＤ（Ｃｈａｒｇｅ　Ｃｏｕｐｌｅｄ　Ｄｅｖｉｃｅ）、および／または、ＣＭＯＳ（Ｃｏｍｐｌｅｍｅｎｔａｒｙ　Ｍｅｔａｌ　Ｏｘｉｄｅ　Ｓｅｍｉｃｏｎｄｕｃｔｏｒ）等の撮像素子を備えたカメラ等であってもよい。 Further, the photographing unit 110 may shoot moving images at 24 fps or 30 fps. Here, the image capturing unit 110 may be a camera including an image sensor such as a CCD (Charge Coupled Device) and / or a CMOS (Complementary Metal Oxide Semiconductor).
　記憶部１０６は、ストレージ手段であり、例えばＲＡＭ・ＲＯＭ等のメモリ、ハードディスクのような固定ディスク装置、ＳＳＤ（Ｓｏｌｉｄ　Ｓｔａｔｅ　Ｄｒｉｖｅ）、および／または、光ディスク等の有形の記憶装置、または、記憶回路を用いることができる。 The storage unit 106 is storage means, for example, a memory such as RAM / ROM, a fixed disk device such as a hard disk, an SSD (Solid State Drive), and / or a tangible storage device such as an optical disk, or a storage circuit. Can be used.
　また、記憶部１０６は、各種のデータベース、テーブル、バッファ、および／または、ファイル（必要情報領域ファイル１０６ａ、および、画像データファイル１０６ｂ等）を格納する。ここで、記憶部１０６には、ＣＰＵ（Ｃｅｎｔｒａｌ　Ｐｒｏｃｅｓｓｉｎｇ　Ｕｎｉｔ）に命令を与え各種処理を行うためのコンピュータプログラム等が記録されていてもよい。 The storage unit 106 stores various databases, tables, buffers, and / or files (necessary information area file 106a, image data file 106b, etc.). Here, the storage unit 106 may store a computer program or the like for giving a command to a CPU (Central Processing Unit) and performing various processes.
　これら記憶部１０６の各構成要素のうち、必要情報領域ファイル１０６ａは、被写体領域における必要情報領域の境界データを記憶する。ここで、必要情報領域は、ホワイトボード（全体）に相当する画像であってもよい。 Of the components of the storage unit 106, the necessary information area file 106a stores boundary data of the necessary information area in the subject area. Here, the necessary information area may be an image corresponding to the whiteboard (entire).
　また、必要情報領域は、被写体領域において必要な情報が視認可能な領域であってもよい。ここで、必要な情報が視認可能な領域とは、文字、数字、記号、図形、写真、および／または、印影等を含む領域であってもよい。 The necessary information area may be an area where necessary information can be visually recognized in the subject area. Here, the area where necessary information can be visually recognized may be an area including letters, numbers, symbols, figures, photographs, and / or seals.
　また、被写体領域は、フレームに基づく読取画像に含まれる原稿（帳票）の原稿画像であってもよい。また、帳票は、運転免許証を含む各種免許証、各種身分証明書、または、健康保険証等の規定帳票であってもよい。 Further, the subject area may be a document image of a document (form) included in a read image based on a frame. The form may be a prescribed form such as various licenses including a driver's license, various identification cards, or a health insurance card.
　このように、必要情報領域ファイル１０６ａは、被写体となる既知の原稿について、必要情報領域の繋ぎ目となる境界の境界データを事前に記憶している。 As described above, the necessary information area file 106a stores boundary data of boundaries that are joints of necessary information areas in advance for a known document that is a subject.
　また、画像データファイル１０６ｂは、画像データ（フレーム等）を記憶する。ここで、画像データファイル１０６ｂは、被写体領域画像データ、必要情報領域画像データ、非テカリ領域画像データ、合成画像データ、分割領域画像データ、非テカリ分割領域画像データ、撮影画像データ、および／または、原稿画像データを記憶していてもよい。 Also, the image data file 106b stores image data (such as a frame). Here, the image data file 106b includes subject area image data, necessary information area image data, non-shiny area image data, composite image data, divided area image data, non-shiny divided area image data, captured image data, and / or Document image data may be stored.
　また、制御部１０２は、画像処理装置１００を統括的に制御するＣＰＵ、ＧＰＵ（Ｇｒａｐｈｉｃｓ　Ｐｒｏｃｅｓｓｉｎｇ　Ｕｎｉｔ）、ＤＳＰ（Ｄｉｇｉｔａｌ　Ｓｉｇｎａｌ　Ｐｒｏｃｅｓｓｏｒ）、ＬＳＩ（Ｌａｒｇｅ　Ｓｃａｌｅ　Ｉｎｔｅｇｒａｔｉｏｎ）、ＡＳＩＣ（Ａｐｐｌｉｃａｔｉｏｎ　Ｓｐｅｃｉｆｉｃ　Ｉｎｔｅｇｒａｔｅｄ　Ｃｉｒｃｕｉｔ）、および／または、ＦＰＧＡ（Ｆｉｅｌｄ－Ｐｒｏｇｒａｍｍｉｎｇ　Ｇａｔｅ　Ａｒｒａｙ）等を含む有形のコントローラ、または、制御回路から構成されてもよい。 The control unit 102 is a CPU, GPU (Graphics Processing Unit), DSP (Digital Signal Processor), LSI (Large Scale Integration / Integration Specified and ASIC), which controls the image processing apparatus 100 in an integrated manner. Alternatively, it may be composed of a tangible controller including a FPGA (Field-Programming Gate Array) or a control circuit.
　また、制御部１０２は、制御プログラムと各種の処理手順等を規定したプログラムと所要データとを格納するための内部メモリを有し、これらプログラムに基づいて種々の処理を実行するための情報処理を行う。 In addition, the control unit 102 has an internal memory for storing a control program, a program defining various processing procedures, and necessary data, and performs information processing for executing various processes based on these programs. Do.
　ここで、制御部１０２は、機能概念的に、フレーム取得部１０２ａ、被写体領域取得部１０２ｂ、必要情報領域取得部１０２ｃ、テカリ検出部１０２ｄ、非テカリ領域取得部１０２ｅ、分割領域取得部１０２ｆ、画像合成部１０２ｇ、分割合成部１０２ｈ、および、ＯＣＲ部１０２ｉを備える。 Here, the control unit 102 functionally conceptually includes a frame acquisition unit 102a, a subject region acquisition unit 102b, a necessary information region acquisition unit 102c, a shine detection unit 102d, a non-shine region acquisition unit 102e, a divided region acquisition unit 102f, and an image. A synthesis unit 102g, a division synthesis unit 102h, and an OCR unit 102i are provided.
　フレーム取得部１０２ａは、フレーム（撮影画像の撮影画像データ）を取得する。ここで、フレーム取得部１０２ａは、撮影部１１０または外部の撮影装置にて撮影されたフレームを取得してもよい。 The frame acquisition unit 102a acquires a frame (captured image data of a captured image). Here, the frame acquisition unit 102a may acquire a frame imaged by the imaging unit 110 or an external imaging device.
　また、フレーム取得部１０２ａは、被写体領域を含む撮影画像の撮影画像データを取得してもよい。また、フレーム取得部１０２ａは、撮影部１１０による静止画撮影を制御して、撮影画像データを取得してもよい。 Further, the frame acquisition unit 102a may acquire captured image data of a captured image including a subject area. The frame acquisition unit 102a may acquire still image data by controlling still image shooting by the shooting unit 110.
　また、フレーム取得部１０２ａは、撮影部１１０による連続画像撮影または動画撮影を制御して、１コマに相当するフレームを取得してもよい。 Also, the frame acquisition unit 102a may acquire a frame corresponding to one frame by controlling continuous image shooting or moving image shooting by the shooting unit 110.
　被写体領域取得部１０２ｂは、フレームから被写体領域の被写体領域画像データを取得する。例えば、被写体領域取得部１０２ｂは、フレームから原稿画像の原稿画像データを取得してもよい。 The subject area acquisition unit 102b acquires subject area image data of the subject area from the frame. For example, the subject area acquisition unit 102b may acquire document image data of a document image from a frame.
　必要情報領域取得部１０２ｃは、被写体領域における必要情報領域を検出し、必要情報領域の必要情報領域画像データを取得する。 The necessary information area acquisition unit 102c detects a necessary information area in the subject area and acquires necessary information area image data of the necessary information area.
　ここで、必要情報領域取得部１０２ｃは、必要情報領域ファイル１０６ａに記憶された必要情報領域の境界データに基づいて、必要情報領域を検出し、必要情報領域画像データを取得してもよい。 Here, the necessary information area acquisition unit 102c may detect the necessary information area based on the boundary data of the necessary information area stored in the necessary information area file 106a and acquire the necessary information area image data.
　テカリ検出部１０２ｄは、必要情報領域画像データに対してテカリを検出する。ここで、テカリ検出部１０２ｄは、各分割領域画像データに対してテカリを検出してもよい。 The shine detection unit 102d detects shine on the necessary information area image data. Here, the shine detection unit 102d may detect shine on each divided region image data.
　また、テカリ検出部１０２ｄは、必要情報領域画像データの輝度と所定の閾値との比較に基づいて、テカリを検出してもよい。 Further, the shine detection unit 102d may detect the shine based on a comparison between the luminance of the necessary information area image data and a predetermined threshold value.
　非テカリ領域取得部１０２ｅは、テカリ検出部１０２ｄによりテカリが検出されなかった必要情報領域画像データである非テカリ領域画像データを取得する。 The non-shine area acquisition unit 102e acquires non-shine area image data that is necessary information area image data in which no shine is detected by the shine detection section 102d.
　ここで、非テカリ領域取得部１０２ｅは、テカリ検出部１０２ｄによりテカリが検出されなかった分割領域画像データである非テカリ分割領域画像データを取得してもよい。 Here, the non-shine area acquisition unit 102e may acquire non-shine division area image data that is division area image data in which no shine has been detected by the shine detection section 102d.
　分割領域取得部１０２ｆは、テカリ検出部１０２ｄによりテカリが検出された必要情報領域画像データに対する文字検出処理により非文字領域を検出し、必要情報領域を非文字領域で分割した分割領域の分割領域画像データを取得する。 The divided area acquisition unit 102f detects a non-character area by character detection processing on the necessary information area image data in which the shine is detected by the shine detection unit 102d, and divides the necessary information area into the non-character areas. Get the data.
　ここで、分割領域取得部１０２ｆは、テカリ検出部１０２ｄによりテカリが検出された必要情報領域画像データに対してエッジ検出法により非文字領域を検出し、分割領域の分割領域画像データを取得してもよい。 Here, the divided region acquisition unit 102f detects the non-character region by the edge detection method for the necessary information region image data in which the shine is detected by the shine detection unit 102d, and acquires the divided region image data of the divided region. Also good.
　画像合成部１０２ｇは、非テカリ領域取得部１０２ｅにより取得された複数の非テカリ領域画像データを合成した合成画像データを取得する。 The image composition unit 102g acquires composite image data obtained by combining a plurality of non-shine area image data acquired by the non-shine area acquisition unit 102e.
　ここで、画像合成部１０２ｇは、非テカリ領域画像データと、必要情報領域を除く被写体領域の領域外部画像データと、を合成した合成画像データを取得してもよい。 Here, the image composition unit 102g may acquire composite image data obtained by combining the non-shiny region image data and the region external image data of the subject region excluding the necessary information region.
　また、画像合成部１０２ｇは、被写体領域に含まれる全ての必要情報領域の非テカリ領域画像データが取得された場合、複数の非テカリ領域画像データを合成した合成画像データを取得してもよい。 In addition, when the non-shine area image data of all necessary information areas included in the subject area is acquired, the image composition unit 102g may acquire composite image data obtained by combining a plurality of non-shine area image data.
　分割合成部１０２ｈは、非テカリ領域取得部１０２ｅにより取得された非テカリ分割領域画像データを合成することで、非テカリ領域画像データを取得する。 The division synthesis unit 102h acquires non-shine area image data by synthesizing the non-shine division area image data acquired by the non-shine area acquisition unit 102e.
　ＯＣＲ部１０２ｉは、画像データに対してＯＣＲ処理を行い、文字データを取得する。ここで、ＯＣＲ部１０２ｉは、非テカリ領域画像データに対してＯＣＲ処理を行い、文字データを取得してもよい。 The OCR unit 102i performs OCR processing on the image data and acquires character data. Here, the OCR unit 102i may perform the OCR process on the non-shine area image data to acquire character data.
　また、ＯＣＲ部１０２ｉは、合成画像データに対してＯＣＲ処理を行い、文字データを取得してもよい。 Further, the OCR unit 102i may perform character image data by performing OCR processing on the composite image data.
［本実施形態の処理］
　上述した構成の画像処理装置１００（モバイル端末）で実行される処理の一例について、図２から図１０を参照して、本実施形態におけるＯＣＲ処理の一例について説明する。図２は、本実施形態の画像処理装置１００における処理の一例を示すフローチャートである。 [Process of this embodiment]
With respect to an example of processing executed by the image processing apparatus 100 (mobile terminal) configured as described above, an example of OCR processing according to the present embodiment will be described with reference to FIGS. FIG. 2 is a flowchart illustrating an example of processing in the image processing apparatus 100 according to the present embodiment.
　図２に示すように、まず、フレーム取得部１０２ａは、被写体に応じて撮影部１１０の設定を初期化し、撮影部１１０による動画撮影開始を制御して、フレームを取得する（ステップＳＡ－１）。 As shown in FIG. 2, first, the frame acquisition unit 102a initializes the setting of the imaging unit 110 according to the subject, controls the start of moving image shooting by the imaging unit 110, and acquires a frame (step SA-1). .
　すなわち、本実施形態においては、カメラデバイスによる被写体の動画撮影を開始する。 That is, in this embodiment, moving image shooting of the subject by the camera device is started.
　そして、被写体領域取得部１０２ｂは、フレームに基づく撮影画像内の被写体領域を検出し、被写体領域の被写体領域画像データを取得し、画像データファイル１０６ｂに格納（記録）する（ステップＳＡ－２）。 Then, the subject area acquisition unit 102b detects a subject area in the captured image based on the frame, acquires subject area image data of the subject area, and stores (records) it in the image data file 106b (step SA-2).
　すなわち、本実施形態においては、動画撮影の間、撮影画像内の被写体領域を検出する。ここで、撮影画像内の被写体領域の検出は、エッジ検出、および／または、特徴点検出等の処理を用いて行ってもよい。 That is, in this embodiment, the subject area in the captured image is detected during moving image shooting. Here, the detection of the subject region in the captured image may be performed using processing such as edge detection and / or feature point detection.
　そして、必要情報領域取得部１０２ｃは、被写体領域における必要情報領域を検出し、必要情報領域の必要情報領域画像データを取得する（ステップＳＡ－３）。 Then, the necessary information area acquisition unit 102c detects the necessary information area in the subject area and acquires the necessary information area image data of the necessary information area (step SA-3).
　ここで、図３を参照して、既知の原稿種における必要情報領域、および、繋ぎ目の一例について説明する。図３は、本実施形態における必要情報領域の一例を示す図である。 Here, with reference to FIG. 3, an example of necessary information areas and joints in known document types will be described. FIG. 3 is a diagram illustrating an example of a necessary information area in the present embodiment.
　図３に示す運転免許証においては、点線で囲まれた領域１（氏名）、領域２（生年月日）、領域３（住所）、領域４（交付日）、領域５（有効期限日）、領域６（免許証番号）および領域７（顔写真）が、必要情報領域であり、点線部が、繋ぎ目（境界）であってもよい。 In the driver's license shown in FIG. 3, area 1 (name), area 2 (date of birth), area 3 (address), area 4 (delivery date), area 5 (expiration date) surrounded by dotted lines, The area 6 (license number) and the area 7 (face photograph) may be necessary information areas, and the dotted line portion may be a joint (boundary).
　ここで、本実施形態においては、被写体画像において、必要情報領域が被覆しない領域を、領域外部としてもよい。なお、本実施形態において、領域外部は、画像合成前の最後に取得された被写体画像から特定してもよい。 Here, in the present embodiment, a region that is not covered by the necessary information region in the subject image may be outside the region. In the present embodiment, the outside of the area may be specified from the subject image acquired last before the image composition.
　また、本実施形態においては、必要情報領域が被写体画像全域を被覆する場合、領域外部は存在しない。 In the present embodiment, when the necessary information area covers the entire subject image, there is no area outside.
　図２に戻り、テカリ検出部１０２ｄは、必要情報領域画像データの輝度と所定の閾値との比較に基づいて、必要情報領域画像データ毎にテカリを検出する（ステップＳＡ－４）。 Referring back to FIG. 2, the shine detection unit 102d detects the shine for each necessary information area image data based on the comparison between the luminance of the necessary information area image data and a predetermined threshold (step SA-4).
　すなわち、本実施形態においては、被写体領域内にてテカリ検出を実施することで、被写体内のテカリの無い必要情報領域を特定する。 That is, in the present embodiment, the necessary information area without the shine in the subject is specified by detecting the shine in the subject area.
　ここで、本実施形態において、被写体領域内のテカリ検出としては、輝度の閾値判定として、画像データのＲＧＢについて、（０．２９９＊Ｒ＋０．５８７＊Ｇ＋０．１１４＊Ｂ）＞２５０が成立する場合、テカリが検出されたと判定してもよい。 Here, in the present embodiment, as the detection of the shine in the subject area, as a threshold determination of the luminance, when (0.299 * R + 0.587 * G + 0.114 * B)> 250 holds for RGB of the image data. It may be determined that the shine has been detected.
　また、本実施形態において、被写体領域内のテカリ検出としては、背景推定値の閾値判定として、窓内平均値＞２００が成立する場合、テカリが検出されたと判定してもよい。 Further, in the present embodiment, as the detection of the shine in the subject area, it may be determined that the shine has been detected when the average value in the window> 200 is established as the threshold value determination of the background estimation value.
　そして、非テカリ領域取得部１０２ｅは、テカリ検出部１０２ｄによりテカリが検出されなかった必要情報領域画像データである非テカリ領域画像データを取得し、画像データファイル１０６ｂに記録する（ステップＳＡ－５）。 Then, the non-shine area acquisition unit 102e acquires non-shine area image data, which is necessary information area image data in which no shine has been detected by the shine detection section 102d, and records it in the image data file 106b (step SA-5). .
　すなわち、本実施形態においては、テカリ無しの必要情報領域の画像データを記録する。 That is, in the present embodiment, image data of a necessary information area without shine is recorded.
　ここで、図４および図５を参照して、本実施形態における必要情報領域分割による非テカリ領域画像データ取得処理の一例について説明する。図４および図５は、本実施形態における必要情報領域分割の一例を示す図である。 Here, with reference to FIG. 4 and FIG. 5, an example of the non-shine area image data acquisition processing by necessary information area division in the present embodiment will be described. 4 and 5 are diagrams showing an example of necessary information area division in the present embodiment.
　本実施形態においては、テカリが検出された際に、必要情報領域内部の一部のみがテカっている場合、必要情報領域を分割することで、非テカリ領域画像データを取得してもよい。 In the present embodiment, when only a part of the necessary information area is detected when the shine is detected, the necessary information area may be divided to acquire the non-shine area image data.
　すなわち、図４に示すように、本実施形態においては、運転免許証の交付日欄である必要情報領域Ａを、図５に示すように、分割領域Ｂおよび分割領域Ｃへ分割してもよい。 That is, as shown in FIG. 4, in the present embodiment, the necessary information area A that is the date of driver license is divided into a divided area B and a divided area C as shown in FIG. 5. .
　ここで、本実施形態においては、分割領域Ｂと分割領域Ｃとの境界が新たな繋ぎ目となるが、当該繋ぎ目が文字を含まないように文字検出を実施し、文字でないと判断された非文字領域を、分割のための繋ぎ目として設定してもよい。 Here, in the present embodiment, the boundary between the divided region B and the divided region C becomes a new joint, but character detection is performed so that the joint does not include a character, and it is determined that the character is not a character. A non-character area may be set as a joint for division.
　ここで、文字検出手法としては、エッジ検出法等を用いてもよい。 Here, an edge detection method or the like may be used as a character detection method.
　そして、本実施形態においては、分割領域Ｂと分割領域Ｃとのそれぞれについて、非テカリ分割領域画像データを取得して、再合成することで、必要情報領域Ａの非テカリ領域画像データを取得してもよい。 In the present embodiment, the non-shiny divided region image data is obtained for each of the divided region B and the divided region C, and the non-shiny region image data of the necessary information region A is obtained by recombining. May be.
　また、図６から図８を参照して、本実施形態におけるホワイトボードにおける必要情報領域分割の一例について説明する。図６から図８は、本実施形態における必要情報領域分割の一例を示す図である。 In addition, an example of necessary information area division in the whiteboard in the present embodiment will be described with reference to FIGS. 6 to 8 are diagrams showing an example of necessary information area division in the present embodiment.
　図６に示すように、本実施形態においては、ホワイトボードを被写体として撮影された画像を必要情報領域Ｄとして検出してもよい。 As shown in FIG. 6, in the present embodiment, an image captured using a whiteboard as a subject may be detected as the necessary information area D.
　そして、図７に示すように、本実施形態においては、動画撮影を開始の後、ホワイトボードに相当する画像の一部にテカリＧが生じた場合、必要情報領域Ｄの分割手法を使用して、エッジが検出されていない「＞議題」の文字列の下に分割領域Ｅと分割領域Ｆとの境界を繋ぎ目として設定してもよい。 As shown in FIG. 7, in this embodiment, after the start of moving image shooting, when a shine G occurs in a part of the image corresponding to the whiteboard, the division method of the necessary information area D is used. The boundary between the divided area E and the divided area F may be set as a joint under the character string “> Agenda” where no edge is detected.
　そして、図８に示すように、本実施形態においては、カメラ撮影位置の変更により、室内照明によるテカリが生じる箇所をテカリＪに変動させている。 Then, as shown in FIG. 8, in this embodiment, the location where shine occurs due to room lighting is changed to shine J due to the change of the camera shooting position.
　そして、図７で全体的にテカリＧが生じていた分割領域Ｆに対して、更に、エッジが検出されていない「２．工程」の文字列の下に分割領域Ｈと分割領域Ｉとの境界を繋ぎ目として設定してもよい。 Then, with respect to the divided area F in which the shine G has occurred as a whole in FIG. 7, the boundary between the divided area H and the divided area I below the character string of “2. Step” in which no edge is detected. May be set as a joint.
　このように、ホワイトボードの光沢は、オフィスの照明を良く反射してしまい、テカリの影響を大きく受けるという課題があったが、本実施形態を活用することで、テカリの影響を回避したホワイトボードスキャニングによる非テカリ領域画像データの取得を実現している。 In this way, the gloss of the whiteboard reflects the office lighting well, and there is a problem that it is greatly affected by the shine, but by utilizing this embodiment, the whiteboard that avoids the influence of the shine is a problem. The acquisition of non-shine area image data by scanning is realized.
　更に、本実施形態においては、テカリが無い分割領域画像データを収集することが可能であるため、最終的には、テカリのないホワイトボード全体の画像データを合成することができる。 Furthermore, in the present embodiment, it is possible to collect the divided area image data without the shine, so that finally the image data of the entire whiteboard without the shine can be synthesized.
　これにより、本実施形態においては、オフィスでの会議でホワイトボードを使用し打ち合わせ内容を記載する場合等に、議事内容が記載されたホワイトボードをモバイルカメラにて撮影し、ＯＣＲを実施することでテキストデータ化できるため、会議の議事録化等のオフィスワーク上の効率化が促進される。 As a result, in the present embodiment, when a meeting is described using a whiteboard at a meeting in an office, the whiteboard on which the contents of the proceedings are recorded is taken with a mobile camera and OCR is performed. Since it can be converted into text data, efficiency in office work such as meeting minutes is promoted.
　図２に戻り、非テカリ領域取得部１０２ｅは、被写体領域に含まれる全ての必要情報領域の非テカリ領域画像データが取得されたか否かを判定する（ステップＳＡ－６）。 Returning to FIG. 2, the non-shine area acquisition unit 102e determines whether non-shine area image data of all necessary information areas included in the subject area has been acquired (step SA-6).
　すなわち、本実施形態においては、ステップＳＡ－２からステップＳＡ－５までの処理を繰り返し、被写体全域のテカリ無しの必要情報領域の画像データを取得されたかを判定する。 In other words, in the present embodiment, the processing from step SA-2 to step SA-5 is repeated to determine whether image data of the necessary information area without the shine of the entire subject has been acquired.
　そして、非テカリ領域取得部１０２ｅは、被写体領域に含まれる全ての必要情報領域の非テカリ領域画像データが取得されていないと判定した場合（ステップＳＡ－６：Ｎｏ）、処理をステップＳＡ－２に移行させる。 If the non-shine area acquisition unit 102e determines that non-shine area image data of all necessary information areas included in the subject area has not been acquired (step SA-6: No), the process proceeds to step SA-2. To migrate.
　一方、非テカリ領域取得部１０２ｅは、被写体領域に含まれる全ての必要情報領域の非テカリ領域画像データが取得されたと判定した場合（ステップＳＡ－６：Ｙｅｓ）、処理をステップＳＡ－７に移行させる。 On the other hand, if the non-shine area acquisition unit 102e determines that non-shine area image data of all necessary information areas included in the subject area has been acquired (step SA-6: Yes), the process proceeds to step SA-7. Let
　そして、画像合成部１０２ｇは、被写体領域に含まれる全ての必要情報領域の非テカリ領域画像データと、必要情報領域を除く被写体領域の領域外部画像データと、を合成した合成画像データを取得する（ステップＳＡ－７）。 Then, the image composition unit 102g obtains composite image data obtained by combining the non-shiny region image data of all necessary information regions included in the subject region and the region external image data of the subject region excluding the necessary information region ( Step SA-7).
　すなわち、本実施形態においては、全領域が取得された場合、画像同士を合成し、テカリの無い被写体画像を作成する。 That is, in the present embodiment, when all the areas are acquired, the images are combined to create a subject image without shine.
　ここで、図９を参照して、本実施形態における画像合成の一例について説明する。図９は、本実施形態における画像合成の一例を示す図である。 Here, an example of image composition in the present embodiment will be described with reference to FIG. FIG. 9 is a diagram illustrating an example of image composition in the present embodiment.
　図９に示すように、本実施形態においては、運転免許証の住所欄と交付日欄との間を境界（図９の点線）として、上部と下部とをそれぞれ必要情報領域として設定してもよい。 As shown in FIG. 9, in the present embodiment, even if the upper part and the lower part are respectively set as necessary information areas with the boundary (dotted line in FIG. 9) between the address field of the driver's license and the date of issue as a boundary. Good.
　そして、図９に示すように、本実施形態においては、動画撮影中に被写体（運転免許証）画像内の上部の生年月日欄Ｋおよび住所欄ＬにテカリＭが発生した画像と、下部の免許証番号欄ＮにテカリＰが発生した画像との画像データを取得してもよい。 As shown in FIG. 9, in the present embodiment, an image in which shine M is generated in the upper date of birth column K and address column L in the subject (driver's license) image during moving image shooting, You may acquire the image data with the image which the shine P generate | occur | produced in the license number column N. FIG.
　そして、両画像データのテカリが検出されない方の必要情報領域画像データを非テカリ領域画像データとして記録してもよい。 Then, the necessary information area image data in which the shine of both image data is not detected may be recorded as the non-shine area image data.
　そして、図９に示すように、非テカリ領域画像データ同士を合成し、合成画像データを取得してもよい。 And as shown in FIG. 9, you may synthesize | combine non-shine area image data and acquire synthetic | combination image data.
　図２に戻り、ＯＣＲ部１０２ｉは、合成画像データに対してＯＣＲ処理を実施して文字データを取得し、文字データを被写体領域画像データに対応付け画像データファイル１０６ｂに記録し（ステップＳＡ－８）、処理を終了する。 Returning to FIG. 2, the OCR unit 102i performs OCR processing on the composite image data to acquire character data, and records the character data in the image data file 106b in association with the subject area image data (step SA-8). ), The process is terminated.
　このように、本実施形態においては、取得された合成画像データにＯＣＲ処理を実施することで、高精度なＯＣＲ成功率を達成することができる。 Thus, in this embodiment, a highly accurate OCR success rate can be achieved by performing the OCR process on the acquired composite image data.
　ここで、図１０を参照して、本実施形態におけるＯＣＲ処理結果の一例について説明する。図１０は、本実施形態におけるＯＣＲ処理結果の一例を示す図である。 Here, an example of the OCR processing result in the present embodiment will be described with reference to FIG. FIG. 10 is a diagram illustrating an example of an OCR processing result in the present embodiment.
　図１０に示すように、図９におけるテカリＭのある生年月日欄Ｋおよび住所欄Ｌに対してＯＣＲ処理をしようとした場合、適切に認識できないことがある。 As shown in FIG. 10, when an OCR process is performed on the date of birth column K and address column L with the shine M in FIG. 9, it may not be properly recognized.
　また、図１０に示すように、図９におけるテカリＰのある免許証番号欄Ｎに対してＯＣＲ処理しようとした場合も、適切に認識できないことがある。 Also, as shown in FIG. 10, even when an OCR process is attempted on the license number column N with the shine P in FIG. 9, it may not be properly recognized.
　そこで、図１０に示すように、本実施形態においては、図９にて合成された合成画像データに対しＯＣＲ処理を実施することで、テカリのある画像データ自体にＯＣＲ処理を実施する場合と比較して、高いＯＣＲ精度を実現している。 Therefore, as shown in FIG. 10, in the present embodiment, the OCR process is performed on the synthesized image data synthesized in FIG. 9, so that the OCR process is performed on the shining image data itself. Thus, high OCR accuracy is realized.
　ここで、店舗窓口等の実際の利用場面においては、照明が一様でなく、運転免許証の撮影画像がテカってしまうことで、精度高い文字認識が阻害されるような場合がある。 Here, in an actual usage scene such as a store window, there is a case where illumination is not uniform, and a photographed image of a driver's license is obscured, so that character recognition with high accuracy is hindered.
　そのような場合であっても、本実施形態においては、運転免許証をモバイルカメラによりスキャニングしてＯＣＲ処理を実施することで、運転免許証記載の個人情報（名前、生年月日、住所、交付日および免許証番号等）を精度よく取得することができる。 Even in such a case, in this embodiment, the personal information (name, date of birth, address, delivery) described in the driver's license is obtained by scanning the driver's license with a mobile camera and performing OCR processing. Date and license number, etc.).
　これにより、本実施形態においては、ＯＣＲ処理により取得された文字データを個人情報データベースと比較することによる個人認証等の処理を容易に実施することができる。 Thereby, in the present embodiment, processing such as personal authentication by comparing character data acquired by OCR processing with a personal information database can be easily performed.
　このように、本実施形態においては、ドキュメント媒体にテカリが生じている状況でも、迅速かつ高精度なＯＣＲを実施することができる。 As described above, in the present embodiment, it is possible to perform OCR quickly and with high accuracy even in a situation where shine is generated in the document medium.
　特に、本実施形態においては、既知の原稿種（各種免許証、または、請求書等）において、取得される必要のある情報領域（例えば、テキスト欄、または、顔写真等）を予め記憶して、画像合成の繋ぎ目となる箇所を特定することができる。 In particular, in this embodiment, an information area (for example, a text field or a face photograph) that needs to be acquired is stored in advance in a known document type (such as various licenses or invoices). , It is possible to specify a location that is a joint for image synthesis.
　すなわち、本実施形態においては、精度の高いＯＣＲを実施するために、テキストが存在する箇所では、画像合成を実施せず、テキストが無い繋ぎ目を合成の対象とすることで、高いＯＣＲ精度を実現することができる。 In other words, in the present embodiment, in order to perform high-precision OCR, image synthesis is not performed in a place where text exists, and a high OCR accuracy is achieved by using a joint having no text as a synthesis target. Can be realized.
　このように、本実施形態においては、事前に知られている原稿種について、必要情報領域の境界を繋ぎ目として使用している。 As described above, in this embodiment, the boundary of the necessary information area is used as a joint for a document type known in advance.
　もし、テキスト部分を横切る繋ぎ目にて画像合成を行った場合、テキスト部分にノイズ成分が混じる等、ＯＣＲ精度劣化の原因となるため、本実施形態においては、テキスト部分を合成の繋ぎ目として使用しないことで、高いＯＣＲ精度を実現している。 If image synthesis is performed at a joint that crosses the text portion, noise components may be mixed into the text portion, which may cause deterioration in OCR accuracy. In this embodiment, the text portion is used as a synthesis joint. By not doing so, high OCR accuracy is achieved.
　これにより、本実施形態においては、テカリ検出を利用して、テカリが無い合成画像データに対するＯＣＲ処理を実行することができるため、テカリによる悪影響を回避したＯＣＲ処理が実現され、文字認識の高精度化を図ることができる。 Thereby, in this embodiment, since the OCR process can be performed on the synthetic image data without the shine using the shine detection, the OCR process avoiding the adverse effect due to the shine is realized, and the character recognition has a high accuracy. Can be achieved.
　また、本実施形態においては、テカリ検出と動画処理技術との組み合わせによる、テカリの無い画像データを迅速に合成することができる。 In this embodiment, image data without shine can be quickly synthesized by combining shine detection and moving image processing technology.
　また、本実施形態においては、撮影画像中の被写体領域における必要情報領域毎にテカリ検出を実行するため、被写体領域全域がテカっていない状況になるまで撮影を繰り返す必要がない。 Further, in the present embodiment, since the shine detection is executed for each necessary information area in the subject area in the photographed image, it is not necessary to repeat photographing until the entire subject area is not shining.
　このため、本実施形態においては、動画撮影中のリトライ回数を低減することができ、迅速なＯＣＲ処理を実施することができる。 For this reason, in the present embodiment, the number of retries during moving image shooting can be reduced, and quick OCR processing can be performed.
　近年、スマートフォン等に組み込まれたカメラデバイスの解像度向上により、モバイルカメラ撮影画像を用いＯＣＲを実施することができるようになってきている。 In recent years, it has become possible to perform OCR using mobile camera images by improving the resolution of camera devices incorporated in smartphones and the like.
　特に、各種免許証または健康保険証といったドキュメント媒体を手軽に撮影し、ＯＣＲにかけることにより、個人認証の手段として使用できるため、商業窓口または店舗での契約手続き等を簡素化することができるようになってきている。 In particular, since document media such as various licenses or health insurance cards can be easily taken and applied to OCR, it can be used as a means of personal authentication, so that contract procedures at commercial counters or stores can be simplified. It is becoming.
　しかしながら、これらドキュメント媒体の多くは照明下にて光を反射し、撮影画像内にテカリ箇所が生じてしまうため、ＯＣＲ認識性能を低下させてしまっている。 However, many of these document media reflect light under illumination, resulting in a shining spot in the photographed image, thus reducing the OCR recognition performance.
　また、顔写真といった個人認証に必要とされるコンテンツを含む画像部分にテカリが生じてしまうと、証明書としての役割を果たさなくなる。 Also, if the image portion containing the content required for personal authentication such as a face photo is brilliant, it will no longer serve as a certificate.
　このため、テカリが生じてしまうシーンでも精度高くＯＣＲを実施可能とし、必要となる部分画像を取得するためには、テカリの影響を除去、もしくは、回避する手法が必要であった。 For this reason, it is possible to perform OCR with high accuracy even in scenes where shine occurs, and in order to acquire the necessary partial images, a technique for removing or avoiding the influence of shine is necessary.
　そこで、本実施形態においては、テカリへの対処技術として、モバイルカメラが撮影位置を移動させながら動画撮影が可能なモバイルカメラの特性を利用した、テカリへの対処技術を開示している。 Therefore, in the present embodiment, as a technique for dealing with shine, a technique for dealing with shine using the characteristics of a mobile camera that allows a mobile camera to shoot a movie while moving the shooting position is disclosed.
　具体的には、本実施形態においては、リアルタイム撮影時にテカリ位置検出を実施することで、必要なコンテンツを含む、テカリのない部分画像を抽出し、合成することで、テカリの無い被写体画像の画像データの生成を実現している。 Specifically, in the present embodiment, by performing shine position detection at the time of real-time shooting, a partial image that does not have shine is extracted and synthesized by including a necessary content, so that an image of a subject image without shine is obtained. Data generation is realized.
　そして、本実施形態においては、この合成画像データに対してＯＣＲ処理を実施することで、ＯＣＲの高精度化を実現している。 In the present embodiment, OCR processing is performed on the composite image data to achieve high accuracy of OCR.
　また、従来から、撮影画像内の輝度分布の平均化手法等によりテカリ輝度分布を推定し、撮影画像からテカリ成分を差し引くことで、テカリの影響を取り除く手法はあった。 Also, conventionally, there has been a method of removing the influence of the shine by estimating the shine brightness distribution by a method of averaging the brightness distribution in the shot image and subtracting the shine component from the shot image.
　しかしながら、テカリ分布を正確に推定することが困難なことから、テカリ成分が差し引かれた画像内に推定誤差起因のノイズが生じてしまい、ＯＣＲ実行時の誤認識要因となっていた。 However, since it is difficult to accurately estimate the shine distribution, noise caused by an estimation error is generated in the image from which the shine component is subtracted, which is a cause of erroneous recognition during OCR execution.
　また、従来から、画像内の色情報を元にテカリ位置を検出する手法があり、主に、画像全体の輝度値ヒストグラムに基づく大域的な手法と、画像内の局所的な輝度値に基づく局所的な手法と、があった。 Conventionally, there is a method for detecting a shine position based on color information in an image, mainly a global method based on a luminance value histogram of the entire image and a local method based on a local luminance value in the image. There was a typical method.
　しかしながら、いずれの手法も、撮影画像がテカっているか否かを判定するためだけに用いられていた。 However, both methods are used only to determine whether or not the photographed image is shining.
　特に、ＯＣＲ処理の前にテカリ検出を行うことで、テカっている画像の場合、ＯＣＲ実行を拒否し、テカリが無い画像が得られるまで、撮影を再度要求するような、リトライ処理が実用されていた。 In particular, retry processing is practically performed by detecting shine before OCR processing, so that, in the case of an image that has been lost, OCR execution is rejected, and until an image without shine is obtained, photographing is requested again. It was.
　すなわち、いずれの手法においても、画像中の極一部にテカリが生じていれば、撮影を再要求するため、ＯＣＲが実施されるまでに、多くの撮影を繰り返す必要性があった。 That is, in any of the methods, if there is a shine in a very small part of the image, it is necessary to repeat many shootings before the OCR is performed because the shooting is requested again.
　そこで、本実施形態においては、ＯＣＲ実行前に、画像全面にテカリが無いと判定されるまで試行を繰り返す必要がないようにしている。 Therefore, in the present embodiment, it is not necessary to repeat trials until it is determined that there is no shine on the entire image before OCR is executed.
　そのため、本実施形態においては、迅速にＯＣＲを実施する上で、画像中の一部にテカリが生じていたとしても、テカっていない箇所に対してＯＣＲを実施することで、可能な限り撮影のリトライ回数を少なくすることができている。 Therefore, in this embodiment, when performing OCR quickly, even if shine occurs in a part of the image, the OCR is performed on a portion that is not shining, so that photographing is performed as much as possible. The number of retries can be reduced.
　また、本実施形態においては、従来行われていたテカリ除去の画像処理を被写体領域に実行する必要が無いため、ＯＣＲの誤認識を減らすことができ、高精度のＯＣＲを実現している。 Further, in the present embodiment, since it is not necessary to perform the image processing for removing shine that has been conventionally performed on the subject area, erroneous recognition of OCR can be reduced, and high-precision OCR is realized.
［他の実施形態］
　さて、これまで本発明の実施形態について説明したが、本発明は、上述した実施形態以外にも、請求の範囲に記載した技術的思想の範囲内において種々の異なる実施形態にて実施されてよいものである。 [Other Embodiments]
The embodiments of the present invention have been described so far, but the present invention may be implemented in various different embodiments other than the above-described embodiments within the scope of the technical idea described in the claims. Is.
　例えば、画像処理装置１００は、スタンドアローンの形態で処理を行ってもよく、クライアント端末（画像処理装置１００とは別筐体である）からの要求に応じて処理を行い、その処理結果を当該クライアント端末に返却するようにしてもよい。 For example, the image processing apparatus 100 may perform processing in a stand-alone form, performs processing in response to a request from a client terminal (which is a separate housing from the image processing apparatus 100), and the processing result is You may make it return to a client terminal.
　また、実施形態において説明した各処理のうち、自動的に行われるものとして説明した処理の全部または一部を手動的に行うこともでき、あるいは、手動的に行われるものとして説明した処理の全部または一部を公知の方法で自動的に行うこともできる。 In addition, among the processes described in the embodiment, all or a part of the processes described as being automatically performed can be manually performed, or all of the processes described as being manually performed can be performed. Alternatively, a part can be automatically performed by a known method.
　このほか、明細書中および図面中で示した処理手順、制御手順、具体的名称、各処理の登録データもしくは検索条件等のパラメータを含む情報、画面例、または、データベース構成については、特記する場合を除いて任意に変更することができる。 In addition, the processing procedure, control procedure, specific name, information including parameters such as registration data or search conditions for each processing, screen examples, or database configuration shown in the description and drawings are specially noted. It can be changed arbitrarily except for.
　また、画像処理装置１００に関して、図示の各構成要素は機能概念的なものであり、必ずしも物理的に図示の如く構成されていることを要しない。 Further, with respect to the image processing apparatus 100, the illustrated components are functionally conceptual, and need not be physically configured as illustrated.
　例えば、画像処理装置１００の各装置が備える処理機能、特に制御部１０２にて行われる各処理機能については、その全部または任意の一部を、ＣＰＵおよび当該ＣＰＵにて解釈実行されるプログラムにて実現してもよく、また、ワイヤードロジックによるハードウェアとして実現してもよい。 For example, all or some of the processing functions provided in each device of the image processing apparatus 100, particularly the processing functions performed by the control unit 102, are executed by the CPU and a program interpreted and executed by the CPU. You may implement | achieve and may implement | achieve as hardware by a wired logic.
　なお、プログラムは、後述する、コンピュータに本発明に係る方法を実行させるためのプログラム化された命令を含む、一時的でないコンピュータ読み取り可能な記録媒体に記録されており、必要に応じて画像処理装置１００に機械的に読み取られる。すなわち、ＲＯＭまたはＨＤＤなどの記憶部１０６などには、ＯＳ（Ｏｐｅｒａｔｉｎｇ　Ｓｙｓｔｅｍ）と協働してＣＰＵに命令を与え、各種処理を行うためのコンピュータプログラムが記録されている。このコンピュータプログラムは、ＲＡＭにロードされることによって実行され、ＣＰＵと協働して制御部を構成する。 The program is recorded on a non-transitory computer-readable recording medium including a programmed instruction for causing a computer to execute the method according to the present invention, which will be described later, and an image processing apparatus as necessary. 100 mechanically read. That is, in the storage unit 106 such as a ROM or an HDD, computer programs for performing various processes by giving instructions to the CPU in cooperation with an OS (Operating System) are recorded. This computer program is executed by being loaded into the RAM, and constitutes a control unit in cooperation with the CPU.
　また、このコンピュータプログラムは、画像処理装置１００に対して任意のネットワークを介して接続されたアプリケーションプログラムサーバに記憶されていてもよく、必要に応じてその全部または一部をダウンロードすることも可能である。 The computer program may be stored in an application program server connected to the image processing apparatus 100 via an arbitrary network, and may be downloaded in whole or in part as necessary. is there.
　また、本発明に係るプログラムを、コンピュータに読み取り可能な記録媒体に格納してもよく、また、プログラム製品として構成することもできる。ここで、この「記録媒体」とは、メモリーカード、ＵＳＢメモリ、ＳＤカード、フレキシブルディスク、光磁気ディスク、ＲＯＭ、ＥＰＲＯＭ、ＥＥＰＲＯＭ、ＣＤ－ＲＯＭ、ＭＯ、ＤＶＤ、および、Ｂｌｕ－ｒａｙ（登録商標）Ｄｉｓｃ等の任意の「可搬用の物理媒体」を含むものとする。 Further, the program according to the present invention may be stored in a computer-readable recording medium, or may be configured as a program product. Here, the “recording medium” includes a memory card, USB memory, SD card, flexible disk, magneto-optical disk, ROM, EPROM, EEPROM, CD-ROM, MO, DVD, and Blu-ray (registered trademark). It includes any “portable physical medium” such as Disc.
　また、「プログラム」とは、任意の言語や記述方法にて記述されたデータ処理方法であり、ソースコードやバイナリコード等の形式を問わない。なお、「プログラム」は必ずしも単一的に構成されるものに限られず、複数のモジュールやライブラリとして分散構成されるものや、ＯＳに代表される別個のプログラムと協働してその機能を達成するものも含む。なお、実施形態に示した各装置において記録媒体を読み取るための具体的な構成、読み取り手順、あるいは、読み取り後のインストール手順等については、周知の構成や手順を用いることができる。 In addition, “program” is a data processing method described in an arbitrary language or description method, and may be in any form such as source code or binary code. Note that the “program” is not necessarily limited to a single configuration, and functions are achieved in cooperation with a separate configuration such as a plurality of modules and libraries or a separate program represented by the OS. Including things. In addition, a well-known structure and procedure can be used about the specific structure for reading a recording medium in each apparatus shown in embodiment, a reading procedure, or the installation procedure after reading.
　記憶部１０６に格納される各種のデータベース等は、ＲＡＭもしくはＲＯＭ等のメモリ装置、ハードディスク等の固定ディスク装置、フレキシブルディスク、および／または、光ディスク等のストレージ手段であり、各種処理やウェブサイト提供に用いる各種のプログラム、テーブル、データベース、および／または、ウェブページ用ファイル等を格納してもよい。 Various databases and the like stored in the storage unit 106 are storage means such as a memory device such as a RAM or a ROM, a fixed disk device such as a hard disk, a flexible disk, and / or an optical disk. Various programs, tables, databases, and / or web page files used may be stored.
　また、画像処理装置１００は、既知のパーソナルコンピュータ等の情報処理装置として構成してもよく、また、該情報処理装置に任意の周辺装置を接続して構成してもよい。また、画像処理装置１００は、該情報処理装置に本発明の方法を実現させるソフトウェア（プログラム、データ等を含む）を実装することにより実現してもよい。 The image processing apparatus 100 may be configured as an information processing apparatus such as a known personal computer, or may be configured by connecting an arbitrary peripheral device to the information processing apparatus. The image processing apparatus 100 may be realized by installing software (including programs, data, and the like) that causes the information processing apparatus to realize the method of the present invention.
　更に、装置の分散・統合の具体的形態は図示するものに限られず、その全部または一部を、各種の付加等に応じて、または、機能負荷に応じて、任意の単位で機能的または物理的に分散・統合して構成することができる。すなわち、上述した実施形態を任意に組み合わせて実施してもよく、実施形態を選択的に実施してもよい。 Furthermore, the specific form of distribution / integration of the devices is not limited to that shown in the figure, and all or a part of them may be functional or physical in arbitrary units according to various additions or according to functional loads. Can be distributed and integrated. That is, the above-described embodiments may be arbitrarily combined and may be selectively implemented.
　以上のように、画像処理装置、画像処理方法、および、プログラムは、産業上の多くの分野、特にカメラで読み込んだ画像を扱う画像処理分野で実施することができ、極めて有用である。 As described above, the image processing apparatus, the image processing method, and the program can be implemented in many industrial fields, particularly in the image processing field that handles images read by a camera, and are extremely useful.
　１００　画像処理装置
　１０２　制御部
　１０２ａ　フレーム取得部
　１０２ｂ　被写体領域取得部
　１０２ｃ　必要情報領域取得部
　１０２ｄ　テカリ検出部
　１０２ｅ　非テカリ領域取得部
　１０２ｆ　分割領域取得部
　１０２ｇ　画像合成部
　１０２ｈ　分割合成部
　１０２ｉ　ＯＣＲ部
　１０６　記憶部
　１０６ａ　必要情報領域ファイル
　１０６ｂ　画像データファイル
　１１０　撮影部
　１１２　入出力部
　１１４　センサ部
　１１６　通信部 DESCRIPTION OF SYMBOLS 100 Image processing apparatus 102 Control part 102a Frame acquisition part 102b Subject area acquisition part 102c Necessary information area acquisition part 102d Detective detection part 102e Non-shine area acquisition part 102f Division area acquisition part 102g Image composition part 102h Division composition part 102i OCR part 106 Storage 106a Necessary information area file 106b Image data file 110 Imaging unit 112 Input / output unit 114 Sensor unit 116 Communication unit

Claims (19)

  1. 　撮影されたフレームを取得するフレーム取得手段と、
    　前記フレームから被写体領域の被写体領域画像データを取得する被写体領域取得手段と、
    　前記被写体領域における必要情報領域を検出し、前記必要情報領域の必要情報領域画像データを取得する必要情報領域取得手段と、
    　前記必要情報領域画像データに対してテカリを検出するテカリ検出手段と、
    　前記テカリ検出手段により前記テカリが検出されなかった前記必要情報領域画像データである非テカリ領域画像データを取得する非テカリ領域取得手段と、
    　を備えたことを特徴とする、画像処理装置。     Frame acquisition means for acquiring a photographed frame;
    Subject region acquisition means for acquiring subject region image data of the subject region from the frame;
    Necessary information area acquisition means for detecting a necessary information area in the subject area and acquiring necessary information area image data of the necessary information area;
    Shine detection means for detecting shine on the necessary information area image data;
    Non-shine area acquisition means for acquiring non-shine area image data that is the necessary information area image data in which the shine is not detected by the shine detection means;
    An image processing apparatus comprising:
  2. 　前記非テカリ領域取得手段により取得された複数の前記非テカリ領域画像データを合成した合成画像データを取得する画像合成手段、
    　を更に備えた、請求項１に記載の画像処理装置。     Image combining means for acquiring composite image data obtained by combining the plurality of non-shine area image data acquired by the non-shine area acquisition means;
    The image processing apparatus according to claim 1, further comprising:
  3. 　前記テカリ検出手段により前記テカリが検出された前記必要情報領域画像データに対する文字検出処理により非文字領域を検出し、前記必要情報領域を前記非文字領域で分割した分割領域の分割領域画像データを取得する分割領域取得手段、
    　を更に備え、
    　前記テカリ検出手段は、
    　前記各分割領域画像データに対してテカリを検出し、
    　前記非テカリ領域取得手段は、
    　前記テカリ検出手段により前記テカリが検出されなかった前記分割領域画像データである非テカリ分割領域画像データを取得し、
    　前記非テカリ領域取得手段により取得された前記非テカリ分割領域画像データを合成することで、前記非テカリ領域画像データを取得する分割合成手段、
    　を更に備えた、請求項１または２に記載の画像処理装置。     A non-character area is detected by character detection processing on the necessary information area image data in which the shine is detected by the shine detection means, and divided area image data of a divided area obtained by dividing the necessary information area by the non-character area is obtained. Divided region acquisition means to perform,
    Further comprising
    The shine detection means is
    Detecting shine for each of the divided region image data,
    The non-shine area acquisition means includes
    Non-shiny divided area image data that is the divided area image data in which the shine was not detected by the shine detection means,
    A dividing and synthesizing unit that acquires the non-shiny region image data by synthesizing the non-shining region image data obtained by the non-shiny region obtaining unit;
    The image processing apparatus according to claim 1, further comprising:
  4. 　前記必要情報領域の境界データを記憶する必要情報領域記憶手段、
    　を更に備え、
    　前記必要情報領域取得手段は、
    　前記境界データに基づいて、前記必要情報領域を検出し、前記必要情報領域画像データを取得する、請求項１から３のいずれか一つに記載の画像処理装置。     Necessary information area storage means for storing boundary data of the necessary information area;
    Further comprising
    The necessary information area acquisition means includes
    The image processing apparatus according to claim 1, wherein the necessary information area is detected based on the boundary data, and the necessary information area image data is acquired.
  5. 　前記テカリ検出手段は、
    　前記必要情報領域画像データの輝度と所定の閾値との比較に基づいて、前記テカリを検出する、請求項１から４のいずれか一つに記載の画像処理装置。     The shine detection means is
    5. The image processing apparatus according to claim 1, wherein the shine is detected based on a comparison between a luminance of the necessary information area image data and a predetermined threshold value. 6.
  6. 　前記分割領域取得手段は、
    　前記テカリ検出手段により前記テカリが検出された前記必要情報領域画像データに対してエッジ検出法により前記非文字領域を検出し、前記分割領域の前記分割領域画像データを取得する、請求項３に記載の画像処理装置。     The divided area acquisition means includes
    4. The divided area image data of the divided area is acquired by detecting the non-character area by an edge detection method for the necessary information area image data in which the shine is detected by the shine detection means. Image processing apparatus.
  7. 　前記非テカリ領域画像データに対してＯＣＲ処理を行い、文字データを取得するＯＣＲ手段、
    　を更に備えた、請求項１から６のいずれか一つに記載の画像処理装置。     OCR means for obtaining character data by performing OCR processing on the non-shine area image data;
    The image processing apparatus according to claim 1, further comprising:
  8. 　前記必要情報領域は、
    　ホワイトボードに相当する画像である、請求項１から７のいずれか一つに記載の画像処理装置。     The necessary information area is:
    The image processing apparatus according to claim 1, wherein the image processing apparatus is an image corresponding to a whiteboard.
  9. 　前記必要情報領域は、
    　前記被写体領域において必要な情報が視認可能な領域である、請求項１から８のいずれか一つに記載の画像処理装置。     The necessary information area is:
    The image processing apparatus according to claim 1, wherein necessary information in the subject area is a visible area.
  10. 　撮影されたフレームを取得するフレーム取得ステップと、
    　前記フレームから被写体領域の被写体領域画像データを取得する被写体領域取得ステップと、
    　前記被写体領域における必要情報領域を検出し、前記必要情報領域の必要情報領域画像データを取得する必要情報領域取得ステップと、
    　前記必要情報領域画像データに対してテカリを検出するテカリ検出ステップと、
    　前記テカリ検出ステップにて前記テカリが検出されなかった前記必要情報領域画像データである非テカリ領域画像データを取得する非テカリ領域取得ステップと、
    　を含むことを特徴とする、画像処理方法。     A frame acquisition step for acquiring a photographed frame;
    A subject area acquisition step of acquiring subject area image data of a subject area from the frame;
    A necessary information area obtaining step of detecting a necessary information area in the subject area and obtaining necessary information area image data of the necessary information area;
    A shine detection step for detecting shine on the necessary information area image data;
    A non-shine area acquisition step of acquiring non-shine area image data that is the necessary information area image data in which the shine is not detected in the shine detection step;
    An image processing method comprising:
  11. 　前記非テカリ領域取得ステップにて取得された複数の前記非テカリ領域画像データを合成した合成画像データを取得する画像合成ステップ、
    　を更に含む、請求項１０に記載の画像処理方法。     An image combining step of acquiring composite image data obtained by combining the plurality of non-shine area image data acquired in the non-shine area acquisition step;
    The image processing method according to claim 10, further comprising:
  12. 　前記テカリ検出ステップにて前記テカリが検出された前記必要情報領域画像データに対する文字検出処理により非文字領域を検出し、前記必要情報領域を前記非文字領域で分割した分割領域の分割領域画像データを取得する分割領域取得ステップ、
    　を更に含み、
    　前記テカリ検出ステップにて、
    　前記各分割領域画像データに対してテカリを検出し、
    　前記非テカリ領域取得ステップにて、
    　前記テカリ検出ステップにて前記テカリが検出されなかった前記分割領域画像データである非テカリ分割領域画像データを取得し、
    　前記非テカリ領域取得ステップにて取得された前記非テカリ分割領域画像データを合成することで、前記非テカリ領域画像データを取得する分割合成ステップ、
    　を更に含む、請求項１０または１１に記載の画像処理方法。     A non-character area is detected by character detection processing on the necessary information area image data in which the shine is detected in the shine detection step, and divided area image data of a divided area obtained by dividing the necessary information area by the non-character area is obtained. A divided region acquisition step to be acquired;
    Further including
    In the shine detection step,
    Detecting shine for each of the divided region image data,
    In the non-shine area acquisition step,
    Obtaining non-shiny divided area image data that is the divided area image data in which the shine was not detected in the shine detection step;
    A dividing and synthesizing step of acquiring the non-shine area image data by combining the non-shine area image data acquired in the non-shine area acquisition step,
    The image processing method according to claim 10 or 11, further comprising:
  13. 　前記必要情報領域取得ステップにて、
    　記憶された前記必要情報領域の境界データに基づいて、前記必要情報領域を検出し、前記必要情報領域画像データを取得する、請求項１０から１２のいずれか一つに記載の画像処理方法。     In the necessary information area acquisition step,
    The image processing method according to any one of claims 10 to 12, wherein the necessary information area is detected and the necessary information area image data is acquired based on the stored boundary data of the necessary information area.
  14. 　前記テカリ検出ステップにて、
    　前記必要情報領域画像データの輝度と所定の閾値との比較に基づいて、前記テカリを検出する、請求項１０から１３のいずれか一つに記載の画像処理方法。     In the shine detection step,
    The image processing method according to claim 10, wherein the shine is detected based on a comparison between a luminance of the necessary information area image data and a predetermined threshold value.
  15. 　前記分割領域取得ステップにて、
    　前記テカリ検出ステップにて前記テカリが検出された前記必要情報領域画像データに対してエッジ検出法により前記非文字領域を検出し、前記分割領域の前記分割領域画像データを取得する、請求項１２に記載の画像処理方法。     In the divided region acquisition step,
    The divided area image data of the divided area is obtained by detecting the non-character area by an edge detection method for the necessary information area image data in which the shine is detected in the shine detection step. The image processing method as described.
  16. 　前記非テカリ領域画像データに対してＯＣＲ処理を行い、文字データを取得するＯＣＲステップ、
    　を更に含む、請求項１０から１５のいずれか一つに記載の画像処理方法。     An OCR step of performing character character data by performing an OCR process on the non-shine area image data;
    The image processing method according to claim 10, further comprising:
  17. 　前記必要情報領域は、
    　ホワイトボードに相当する画像である、請求項１０から１６のいずれか一つに記載の画像処理方法。     The necessary information area is:
    The image processing method according to claim 10, wherein the image processing method is an image corresponding to a whiteboard.
  18. 　前記必要情報領域は、
    　前記被写体領域において必要な情報が視認可能な領域である、請求項１０から１７のいずれか一つに記載の画像処理方法。     The necessary information area is:
    The image processing method according to claim 10, wherein necessary information is visible in the subject area.
  19. 　撮影されたフレームを取得するフレーム取得ステップと、
    　前記フレームから被写体領域の被写体領域画像データを取得する被写体領域取得ステップと、
    　前記被写体領域における必要情報領域を検出し、前記必要情報領域の必要情報領域画像データを取得する必要情報領域取得ステップと、
    　前記必要情報領域画像データに対してテカリを検出するテカリ検出ステップと、
    　前記テカリ検出ステップにて前記テカリが検出されなかった前記必要情報領域画像データである非テカリ領域画像データを取得する非テカリ領域取得ステップと、
    　をコンピュータに実行させるためのプログラム。     A frame acquisition step for acquiring a photographed frame;
    A subject area acquisition step of acquiring subject area image data of a subject area from the frame;
    A necessary information area obtaining step of detecting a necessary information area in the subject area and obtaining necessary information area image data of the necessary information area;
    A shine detection step for detecting shine on the necessary information area image data;
    A non-shine area acquisition step of acquiring non-shine area image data that is the necessary information area image data in which the shine is not detected in the shine detection step;
    A program that causes a computer to execute.
PCT/JP2016/069528 2016-06-30 2016-06-30 Image processing device, image processing method, and program WO2018003090A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/JP2016/069528 WO2018003090A1 (en) 2016-06-30 2016-06-30 Image processing device, image processing method, and program

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2016/069528 WO2018003090A1 (en) 2016-06-30 2016-06-30 Image processing device, image processing method, and program

Publications (1)

Publication Number Publication Date
WO2018003090A1 true WO2018003090A1 (en) 2018-01-04

Family

ID=60785183

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2016/069528 WO2018003090A1 (en) 2016-06-30 2016-06-30 Image processing device, image processing method, and program

Country Status (1)

Country Link
WO (1) WO2018003090A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113194253A (en) * 2021-04-28 2021-07-30 维沃移动通信有限公司 Shooting method and device for removing image reflection and electronic equipment

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0757045A (en) * 1993-08-17 1995-03-03 Matsushita Electric Ind Co Ltd Driver's license recognition device
JPH07210628A (en) * 1994-01-20 1995-08-11 Matsushita Electric Ind Co Ltd License reader
JP2005130326A (en) * 2003-10-27 2005-05-19 Konica Minolta Photo Imaging Inc Digital camera
JP2008071076A (en) * 2006-09-13 2008-03-27 Oki Electric Ind Co Ltd Image extraction device

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0757045A (en) * 1993-08-17 1995-03-03 Matsushita Electric Ind Co Ltd Driver's license recognition device
JPH07210628A (en) * 1994-01-20 1995-08-11 Matsushita Electric Ind Co Ltd License reader
JP2005130326A (en) * 2003-10-27 2005-05-19 Konica Minolta Photo Imaging Inc Digital camera
JP2008071076A (en) * 2006-09-13 2008-03-27 Oki Electric Ind Co Ltd Image extraction device

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113194253A (en) * 2021-04-28 2021-07-30 维沃移动通信有限公司 Shooting method and device for removing image reflection and electronic equipment

Similar Documents

Publication Publication Date Title
US9609221B2 (en) Image stabilization method and electronic device therefor
WO2016112704A1 (en) Method and device for adjusting focal length of projector, and computer storage medium
US7450756B2 (en) Method and apparatus for incorporating iris color in red-eye correction
US11736792B2 (en) Electronic device including plurality of cameras, and operation method therefor
JP2015012480A (en) Image processing apparatus and image processing method
US11790583B2 (en) Image processing apparatus, image processing method, and non-transitory computer-readable storage medium
JP2010211255A (en) Imaging apparatus, image processing method, and program
US10810743B2 (en) Image processing device, image processing method, and computer program product
JP2013106284A (en) Light source estimation device, light source estimation method, light source estimation program, and imaging apparatus
US20110235866A1 (en) Motion detection apparatus and method
JP5510287B2 (en) Subject detection apparatus, subject detection method, and program
JP6581288B2 (en) Mobile terminal, image processing method, and program
WO2018003090A1 (en) Image processing device, image processing method, and program
JP6777507B2 (en) Image processing device and image processing method
JP6503478B2 (en) Mobile terminal, image processing method, and program
JP2011135227A (en) Object tracking device and control method thereof
JP6785930B2 (en) Mobile devices, image processing methods, and programs
WO2017208368A1 (en) Image processing device, image processing method, and program
JP6613378B2 (en) Mobile terminal, image processing method, and program
JP5807695B2 (en) Subject detection apparatus, subject detection method, and program
JP5740934B2 (en) Subject detection apparatus, subject detection method, and program
JP6697829B2 (en) Mobile terminal, image processing method, and program
JP6335064B2 (en) IMAGING DEVICE, IMAGING DEVICE CONTROL METHOD, AND COMPUTER PROGRAM
JP6668646B2 (en) Image processing apparatus, image processing method, and program
WO2017158814A1 (en) Mobile terminal, image processing method, and program

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16907323

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16907323

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: JP