JP2018060392A - Image synthesis device, image synthesis method, and image synthesis system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To create an image allowing precise recognition of characters by enlargement radiography of a necessary region.SOLUTION: A character area size calculation unit 305 calculates a character area size for each character area included in a photographic image acquired by photographing a prescribed imaging range. An enlargement radiography determination unit 307 determines necessity of enlargement radiography in the prescribed imaging range based on the character area size for each calculated character area. An imaging range determination unit 308 determines an enlargement radiography range based on a character area size for each calculated character area and a position of each character area when the enlargement radiography determination unit 307 determines that enlargement radiography is necessary. An image synthesis unit 309 creates a synthesis image by synthesizing an enlargement radiography image acquired by enlargement radiography of the enlargement radiography area with a photographing image.SELECTED DRAWING: Figure 3

Description

  The present invention relates to an image composition device, an image composition method, and an image composition system.

  2. Description of the Related Art Document camera devices are known in which a document placed on a document table 105, an object such as a book or a three-dimensional object is photographed by a camera located above the document table 105 and output as image data such as a pdf file or a jpeg file. ing.

  The document camera apparatus captures a wide range of objects using a camera, and can acquire objects existing within a certain range as image data in a lump. Therefore, it is possible to project a real-time image by connecting the document camera device to a projection device as another form of use of the document camera device and projecting image data obtained by photographing on the screen sequentially and sequentially. Become.

  As described above, since the document camera apparatus captures an object in a wide range, it is assumed that the position of the camera and the position of the object are separated to some extent. For this reason, for example, in order to capture a small character on a manuscript so that it can be clearly read, it is necessary to provide an expensive camera with high resolution, and there is a problem of increasing the overall cost.

  Furthermore, in recent years, there has been an increasing use of OCR that performs character recognition on image data obtained by photographing and converts it to text data. When character recognition is performed, the recognition rate changes depending on the size of the character to be photographed. Therefore, when a small character is photographed with a low-resolution camera, there is a problem in that character recognition accuracy decreases. Therefore, the user has to adjust the magnification of the camera manually.

  Therefore, Japanese Patent Laid-Open No. 2006-094082 (Patent Document 1) compares the size of the character to be photographed with the recognizable character size, automatically adjusts the magnification and resolution, and shoots a plurality of times. A technique for synthesizing the results and outputting an image capable of character recognition with sufficient accuracy is disclosed.

However, since Patent Document 1 captures the entire area to be photographed at the same magnification, even if characters in only a part of the area are smaller than a recognizable size when photographed without using the zoom. Therefore, it was necessary to magnify the entire area to be photographed so that the characters can be recognized.
For this reason, there is a problem in that the number of times of photographing is wasted by enlarging photographing even in an area where characters can be recognized without using zoom.
The present invention has been made in view of the above, and an object thereof is to enable accurate character recognition by enlarging a necessary area.

  In order to solve the above problem, the invention according to claim 1 is a character area size calculation unit that calculates a character area size for each character area included in a captured image acquired by capturing a predetermined capturing range, and the calculation Based on the character area size for each of the character areas thus obtained, enlargement shooting is required by the enlargement shooting necessity determination unit that determines the necessity of enlargement shooting within the predetermined shooting range and the enlargement shooting necessity determination unit. When determined, an imaging range determination unit that determines an enlarged imaging range based on the calculated character area size for each of the character areas and a position for each of the character areas, and enlarges the imaging of the enlarged imaging range. And an image composition unit that composes an enlarged photographed image acquired in this manner and the photographed image to create a composite image.

  According to the present invention, it is possible to recognize characters with high accuracy by enlarging a necessary area.

It is a figure which shows an example of the whole structure of the document camera apparatus which concerns on the 1st Embodiment of this invention. It is a figure which shows an example of the hardware constitutions of the document camera apparatus which concerns on the 1st Embodiment of this invention. It is a figure which shows an example of a function structure of the document camera apparatus which concerns on the 1st Embodiment of this invention. It is a figure which shows an example of the imaging | photography condition table which concerns on the 1st Embodiment of this invention. It is a figure which shows an example of the area | region division which concerns on the 1st Embodiment of this invention. It is a figure which shows an example of the calculation method of the pixel number information which concerns on the 1st Embodiment of this invention. It is a figure which shows an example of the magnification table which concerns on the 1st Embodiment of this invention. It is a figure which shows an example of a function structure of the expansion photographing necessity determination part which concerns on the 1st Embodiment of this invention. It is a figure which shows an example of a function structure of the imaging condition determination part which concerns on the 1st Embodiment of this invention. It is a figure which shows the determination method of the imaging | photography range which concerns on the 1st Embodiment of this invention. It is a figure which shows an example of the imaging | photography condition table which concerns on the 1st Embodiment of this invention. It is a figure which shows an example of the flow of the character recognition which concerns on the 1st Embodiment of this invention. It is a figure which shows an example of the flow of the necessity determination of the expansion photography which concerns on the 1st Embodiment of this invention. It is a figure which shows an example of the preparation flow of the imaging condition table which concerns on the 1st Embodiment of this invention. It is a figure which shows an example of a function structure of the imaging condition determination part which concerns on the 2nd Embodiment of this invention. It is a figure which shows the determination method of the imaging | photography range which concerns on the 2nd Embodiment of this invention. It is a figure which shows an example of the imaging | photography condition table which concerns on the 2nd Embodiment of this invention. It is a figure which shows an example of the preparation flow of the imaging condition table which concerns on the 2nd Embodiment of this invention. It is a figure which shows an example of the whole structure of the document camera apparatus which concerns on the 3rd Embodiment of this invention.

  Hereinafter, with reference to the drawings, a specific embodiment of a photographing technique in the document camera apparatus according to the embodiment of the present invention will be described.

<First embodiment>
This embodiment improves the accuracy of character recognition in the subsequent stage by replacing a partial area including a small character string in image data obtained by photographing with enlarged image data obtained by enlarged photographing. Enable.

<Document camera device>
FIG. 1 is a diagram illustrating the overall configuration of a document camera device 100 according to the first embodiment. As shown in FIG. 1, a document camera apparatus 100 according to the present embodiment supports a photographing apparatus 101, an illumination 102, a document table 105 that is a table for placing a photographing object, the photographing apparatus 101, and the illumination 102. A support member 103 and an input I / F 104 are provided.

  The photographing apparatus 101 is an apparatus for photographing an object placed on the document table 105. The imaging device 101 has an imaging device such as a CCD sensor or a CMOS sensor as an imaging mechanism, a zoom lens or mirror as a lens mechanism, and a memory for temporarily storing an image obtained by imaging an object. The photographing apparatus 101 can freely rotate in a hemispherical shape.

  The illumination 102 is an LED lamp, and a plurality of illumination members 102 are provided on the support member. The input I / F 104 is an interface that accepts input from the user through a keyboard, a touch panel, or the like.

  FIG. 2 is a diagram illustrating a hardware configuration of the document camera apparatus 100 according to the first embodiment. In the document camera apparatus 100 according to the present embodiment, a photographing apparatus 101, an illumination 102, an input I / F 104, a CPU 201, a RAM 202, a ROM 203, a storage device 204, and an external I / F 205 are connected via a bus.

  The CPU 201 is an arithmetic processing unit in the document camera apparatus 100, controls the overall operation of the document camera apparatus 100 according to a control program, recognizes characters for image data received from the photographing apparatus 101, performs image processing for image data, Performs processing such as image data format conversion. In the present embodiment, the CPU 201 performs character recognition and image processing, but some of the processing may be performed using hardware such as an ASIC, or via the external I / F 205. It may be performed by an external device.

  A RAM 202 is a volatile storage medium for reading and writing information at high speed, and functions as a work area when the CPU 201 executes a control program. The ROM 203 is a read-only nonvolatile storage medium in which a control program is stored.

  The storage device 204 is an HDD (Hard Disk Drive), for example, and is a large-capacity nonvolatile storage medium that can read and write information. The storage device 204 stores control programs, applications, and the like, and is output from various devices. Image data to be stored. The storage device 204 may be another type of storage medium such as an SSD (Solid State Drive).

  The external I / F 205 is a communication interface for connecting to a network such as a LAN, and is connected to an external terminal such as a PC or a smartphone. The external I / F 205 has a reception function and a transmission function. In addition to receiving information sent from an external terminal, the external I / F 205 transmits image data stored in the storage device 204.

  Based on information input by the user via the input I / F 104, the CPU 201 reads a corresponding control program from the ROM 203 and controls a target device. The CPU 201 can also receive information from an external terminal such as a PC or a smartphone through the external I / F 205 and read a corresponding control program from the ROM 203 to control a target device.

  The imaging device 101 is rotatably mounted and images an object under conditions in accordance with instructions from the CPU 201. Information sent from the CPU 201 to the photographing apparatus 101 will be described in detail later. The photographed image data is temporarily stored in a memory in the photographing apparatus 101 and then output to the RAM 202 through a bus. The illumination 102 is switched on / off in accordance with an instruction from the CPU 201.

  FIG. 3 is a diagram illustrating a functional configuration of the document camera apparatus 100 according to the first embodiment. The processing units surrounded by broken lines in the figure are hardware, and the respective processing units surrounded by solid lines in the figure functionally indicate processing executed by the CPU 201, ROM 203, and RAM 202.

  The mediation unit 301 creates an imaging condition table according to a start instruction input via the input I / F 104 or the external I / F 205 and outputs the imaging condition table to the imaging apparatus 101. This start instruction is an instruction to start or end shooting. In addition, the mediation unit 301 outputs an end flag for ending the shooting to the image capturing apparatus 101 in accordance with an end instruction from the input I / F 104 or the external I / F 205.

  The imaging device 101 refers to an imaging condition table received from the mediation unit 301 or the imaging condition determination unit 308 described later, and images a printed material placed on the document table 105, and sets the data obtained by imaging to a predetermined value. Output to the device.

<Shooting condition table>
FIG. 4 is a diagram for explaining the photographing condition table.
The operation mode indicates a series of processing contents. “Character recognition mode” refers to a series of processes for reconstructing part or all of image data obtained by photographing an object including characters into a text file. The process described in this embodiment It is. In addition to the “character recognition mode” as the operation mode, the “normal mode” in which the image data obtained by shooting is stored in the storage device 204 as it is, the shot video is connected to the projection device via the external I / F 205, There is a “projection mode” that projects in real time. The photographing apparatus 101 performs photographing in a corresponding mode. The enlargement magnification indicates the enlargement magnification when enlarging shooting, and the shooting range indicates the area to be shot.

  When the operation mode input to the shooting condition table is “normal mode”, the shot image data is output to the storage device 204. When the operation mode input to the imaging condition table is “projection mode”, the captured moving image data is sequentially transmitted to an external projection device via the external I / F 205.

  When the operation mode input in the shooting condition table is “character recognition mode” and the input magnification is 1 × (no zoom is used), the image data obtained by shooting is pre-image data. Is output to the temporary storage unit 302. When the operation mode input to the shooting condition table is “character recognition mode” and the input enlargement magnification is greater than 1, the enlarged image data is output to the temporary storage unit 302 as enlarged image data. The

<Shooting area>
FIG. 5 is a diagram for explaining the photographing region in detail.
A solid line frame in the figure indicates a shooting area when shooting without zooming. The dotted line in the figure shows a state in which the vertical and horizontal directions of the shooting area are divided into two when shooting at a magnification of 2 times. At this time, (x1, y1), (x2, y1), (x1, y2), and (x2, y2) correspond to region information indicating a photographing range when performing magnified photographing. As the region information is located from left to right, the value of n of xn (n is a natural number of 1 or more) increases, and as it is located from top to bottom, the value of n of yn (n is a natural number of 1 or more) is increased. To increase. The maximum number of n in the area information = enlargement magnification.

  The photographing device 101 stores in advance a photographing location corresponding to the enlargement magnification and the region information. The photographing device 101 rotates the direction of itself to photograph the portion indicated by the region information, and the photographed data is stored in a predetermined data. Output to the device.

  The temporary storage unit 302 temporarily stores data sent from the photographing apparatus 101. The temporary storage unit 302 is a storage medium that can be rewritten at high speed, and corresponds to the RAM 202 in the present embodiment.

The character region determination unit 304 refers to the pre-image data stored in the temporary storage unit 302 and extracts a character region using a method described in Patent Document 2, for example. The character area is an area surrounding the outside of the character string, and is an area surrounded by a rectangle surrounding the outside of the character string ABC as shown in FIG. 6, for example.
The character area size calculation unit 305 calculates the size of the detected character area as pixel number information for each target character area.

<Calculation method of pixel number information>
FIG. 6 is a diagram for explaining a method for calculating pixel number information by the character area size calculation unit 305. A region X surrounded by a rectangle in the drawing is a character region determined by the character region determination unit 304. The character area size calculation unit 305 compares the number of pixels used for the sides of the rectangle forming the character area, and calculates the number of pixels on the side having the smaller number of pixels as the character area size of the area X. In the figure, the character area size calculation unit 305 compares the number of pixels used for the a side (12 dots) and the b side (36 dots), and the number of pixels used for the a side is smaller. The number of used pixels of 12 dots is calculated as the character area size of the area X, and the calculated character area size is assigned to the area X. When the character area size has been assigned to all the character areas included in the pre-image data, the character area size calculation unit 305 outputs the pre-image data to which the character area size has been assigned to the enlarged shooting necessity determination unit 307. .

  The size information storage unit 306 stores an enlargement ratio suitable for each number of pixels as a magnification table. The enlargement magnification suitable for each number of pixels is an enlargement magnification at the time of photographing necessary for obtaining sufficient accuracy when a character included in the character area on the pre-image data is recognized later. The accuracy of character recognition depends on the number of vertical and horizontal pixels that make up the character that is to be recognized. The higher the magnification, the narrower the shooting area, but the larger the number of vertical and horizontal pixels that make up the character that is to be recognized. . When photographing pre-image data, zoom is not used, and when photographing a small character, a sufficient number of pixels cannot be obtained, so that sufficient accuracy cannot be obtained at the subsequent recognition processing. A character region composed of such small characters can be secured with a sufficient number of pixels for one character by shooting at a higher magnification.

<Magnification table>
FIG. 7 shows an example of the magnification table. In the present embodiment, it is assumed that the photographing apparatus 101 has a resolution equivalent to 200 dpi. The horizontal row in the figure shows the enlargement magnification at the time of shooting, and the minimum character area size that can be recognized with sufficient accuracy in a later process when shooting at that magnification. The minimum character area size here is the character area size when the image is taken without using the zoom.
In this embodiment, it is assumed that the minimum character area size that can be recognized with sufficient accuracy in subsequent processing is 30 dots or more, and sufficient accuracy cannot be obtained when characters included in a character area size of 29 dots or less are recognized. Assume that The reason why the minimum character area size is smaller as the enlargement magnification is higher is that shooting with a higher magnification reduces the shooting area and increases the number of vertical and horizontal pixels of the character to be recognized.
For example, when a character area that was 6 dots when photographed without using the zoom was photographed at a magnification of 5 times, the character area size was 30 dots, and the minimum character area size that can be recognized with sufficient accuracy It can be seen that the value exceeds the lower limit of.

The enlargement shooting necessity determination unit 307 refers to the magnification table and determines whether or not enlargement shooting is necessary for the character area that is the target of the pre-image data.
<Operation of Enlarged Shooting Necessity Determination Unit>
FIG. 8 is a block diagram illustrating the function of the enlarged shooting necessity determination unit 307.
The comparison unit 307a refers to the magnification table and assigns magnification information indicating an enlargement magnification as necessary to all the character regions in the pre-image data. A method for determining whether or not to apply magnification information and a method for determining the magnification information to be applied will be specifically described.
The comparison unit 307a compares the character area size of the target character area with the minimum character area size (30 dots) that can be recognized with sufficient accuracy, and if the character area size of the target character area is smaller , Magnification information is given to the target character area. The comparison unit 307a assigns, as magnification information, an enlargement magnification that is equal to or smaller than the character area size of the target character area and has the smallest enlargement magnification.
For example, when the character area size of the target character area is 8 dots, the comparison unit 307a reduces the minimum character area size that can be recognized with sufficient accuracy to 8 dots or less and reduces the enlargement magnification to 4 times. It is given as magnification information.

  The determination unit 307b determines whether enlargement shooting is necessary based on the presence / absence of magnification information in the pre-image data. Specifically, the enlargement shooting necessity determination unit 307 determines that the enlargement shooting is not necessary when there is no area having magnification information in the character area in the pre-image data, and the character area in the pre-image data. If there is an area having magnification information, it is determined that enlarged shooting is necessary.

  Returning to FIG. 3, when the enlargement shooting necessity determination unit 307 determines that the enlargement shooting is necessary, the shooting condition determination unit 308 receives the enlargement magnification and the shooting region to be enlarged based on the magnification information. Is output to the photographing apparatus 101.

<Shooting condition determination unit>
FIG. 9 is a block diagram illustrating functions of the imaging condition determination unit 308.
The area dividing unit 308a divides the shooting area when shooting is performed without using the zoom, with the number of divisions based on the highest magnification among the magnification information given to the pre-image data. Next, when there is a pixel including a character area to which magnification information is added in the divided area, the shooting range determination unit 308b determines that the corresponding area is an area that requires enlarged shooting. Next, the creation unit 308c creates an imaging condition table so that enlarged imaging is performed on an area that is determined to be an area requiring enlarged imaging by the imaging range determination unit 308b.

<How to determine the shooting range>
FIG. 10 is a diagram for explaining a method of determining a shooting range.
A halftone dot region indicates a character region. Although the assigned magnification information is described in the character area to which the magnification information is assigned, nothing is described about the character area to which the magnification information is not given. The enlargement magnification of the character area indicating the highest enlargement magnification in the magnification information in the figure is 3 times, and when shooting at a magnification of 3 times, the shooting area is 1/3 as long as when the zoom is not used. Since the width is 1/3, the number of divisions is set to 9, and the area dividing unit 308a divides a shooting area shot without using zoom into nine areas divided by dotted lines.
In the figure, (x3, y1), (x3, y2), (x1, y3), (x2, y3), (x3, y3) have character areas in the area, and the shooting range determination unit 308b This area is determined to be an area that requires enlarged shooting. The creation unit 308c inputs (x3, y1), (x3, y2), (x1, y3), (x2, y3), (x3, y3) indicating the magnification information and the shooting range to the shooting condition table, and operates the operation mode. Enter "Character recognition mode". FIG. 11 shows a photographing condition table showing information output to the photographing apparatus 101 at this time.

  Returning to FIG. 3, the image composition unit 309 enlarges the pre-image data stored in the temporary storage unit 302 when the enlarged shooting necessity determination unit 307 determines that the enlarged shooting is necessary. Next, the partial area of the enlarged pre-image data is replaced with the enlarged image data or the partial area of the enlarged image data stored in the temporary storage unit 302 from the photographing apparatus 101, and the pre-image data and the enlarged image data To create composite image data. Therefore, a higher-resolution image can be obtained in the enlarged image data area than in the original pre-image data area.

  When the enlargement photographing necessity determination unit 307 determines that enlargement photographing is necessary, the character recognition unit 310 performs character recognition processing on the composite image data to create text data. When the enlargement shooting necessity determination unit 307 determines that enlargement shooting is not necessary, the character recognition unit 310 performs character recognition processing on the pre-image data to create text data. Character recognition processing is performed by, for example, OCR (Optical Character Reader). The character recognition unit 310 may perform character recognition processing on all pixels of the composite image data or the pre-image data, but performs character recognition processing only on the character region determined by the character region determination unit 304. It is possible to improve the processing speed.

  The image conversion unit 311 combines one of the composite image data or the pre-image data that has been subjected to character recognition by the character recognition unit 310 and the text data created by the character recognition unit 310, and then converts the data into a predetermined file format. And stored in the storage device 204. Data synthesis is performed by replacing a part of synthesized image data or pre-image data with reference to text data. The file format of the conversion destination can be specified by the user, such as a pdf file or a jpeg file.

<Operation of document camera device>
FIG. 12 is a flowchart of processing of the document camera apparatus 100 according to the present embodiment.
When an instruction to start photographing is input via the input I / F 104 or the external I / F 205 in S101, the mediation unit 301 creates a photographing condition table based on the instruction and outputs the photographing condition table to the photographing apparatus 101 in S102. The photographing apparatus 101 photographs the object placed on the document table 105 with reference to the photographing condition table created in S102.
Next, the mediation unit 301 determines an operation mode at the start of shooting (S103).
When shooting is started in S103, image data or video data obtained by shooting is transmitted to a different place depending on the operation mode in the shooting condition table.
When the received operation mode in the imaging condition table is “normal mode” (S103, normal mode), the process proceeds to S113, and in S113, the captured image data is stored in the storage device 204, and the process ends. When the operation mode in the received shooting condition table is “projection mode” (S103, projection mode), the process proceeds to S114, and in S114, moving image data being shot is transmitted in real time via the external I / F 205. When the imaging apparatus 101 receives an end flag from the mediation unit 301 in S115, the projection apparatus ends imaging and ends the process. If the photographing apparatus 101 has not received an end flag from the mediation unit 301 in S115, transmission of video data is continued.

  When the operation mode is “character recognition mode” (S103, character recognition mode), the captured image data is output to the temporary storage unit 302 as pre-image data, and the process proceeds to S105. In S105, the character area determination unit 304 refers to the pre-image data stored in the temporary storage unit 302, and determines a character area in the pre-image data.

In S106, the character area size calculation unit 305 assigns pixel number information to all the character areas in the pre-image data determined in S105, and the process proceeds to S107. In step S107, the enlargement shooting necessity determination unit 307 compares the magnification table and all the pixel number information added to the pre-image data, and determines whether enlargement shooting for accurate character recognition in the subsequent stage is necessary. To do. If it is determined in S107 that enlargement shooting is not required (S107, No), the process proceeds to S111. If it is determined in S107 that enlarged shooting is necessary (S107, Yes), the process proceeds to S108.
In step S <b> 108, the shooting condition determination unit 308 creates a shooting condition table for enlarged shooting and outputs the table to the shooting apparatus 101. In step S <b> 109, the photographing apparatus 101 performs enlarged photographing with reference to the photographing condition table received in step S <b> 108, and outputs the obtained enlarged image data to the temporary storage unit 302.
In S110, the image composition unit 309 composes pre-image data and enlarged image data to create composite image data. In S111, the character recognition unit 310 performs character recognition on the composite image data or the pre-image data, and creates text data. In S112, the image conversion unit 311 synthesizes the composite image data or pre-image data that has been subjected to character recognition processing in S111 and the text data created in S111, converts the data into a designated file format, and converts the converted data. Create

<Operation of Enlarged Shooting Necessity Determination Unit>
FIG. 13 is a flowchart for more specifically explaining the flow of S107 by the enlargement photographing necessity determination unit 307.
In S201, the comparison unit 307a refers to the magnification table. In S202, the comparison unit 307a compares the character area size of the target character area with the minimum character area size that can be recognized with sufficient accuracy, and if it is equal to or larger than the minimum character area size (S202, Yes). The process proceeds to S205, and if it is less than the minimum character area size (S202, No), the process proceeds to S203.
In step S203, the comparison unit 307a assigns an enlargement magnification that is equal to or smaller than the character area size of the target character area and has the smallest enlargement magnification as magnification information of the target character area, and proceeds to step S204.
In S204, it is determined whether or not the process of S202 has been completed for all character areas in the pre-image data. If the process has been completed for all character areas (Yes in S204), the process proceeds to S205 and the process is still performed. If there is an unfinished character area (S204, No), the process returns to S202.
In S205, the determination unit 307b determines whether there is a character area to which the magnification information is added. If there is no area having the magnification information in the character area in the pre-image data (S205, No), the process proceeds to S206, and the determination unit 307b determines that the enlarged shooting is unnecessary and ends the process. If there is an area having magnification information in the character area in the pre-image data (S205, Yes), the process proceeds to S207, and the determination unit 307b determines that enlargement shooting is necessary and ends the process.

<Operation of shooting condition determination unit>
FIG. 14 is a flowchart for more specifically explaining the flow of S108 by the imaging condition determination unit 308.
In S301, the area dividing unit 308a divides the shooting area by the number of divisions based on the maximum magnification among the magnification information given to the pre-image data.
In S302, when there is a pixel including a character area to which magnification information is added in the divided area, the imaging range determination unit 308b determines that the area is an area that needs to be enlarged.
In S303, the creation unit 308c sets the “character recognition mode” as the operation mode of the shooting condition table, the maximum magnification determined in S301 as the magnification, and the area that requires the magnification determined in S302 as the shooting range. The information is input, the photographing condition table is output to the photographing apparatus 101, and the process ends.

  As described above, according to the document camera device 100 according to the present embodiment, the area that needs to be enlarged is determined from the pre-image data that is taken without using the zoom, and the enlarged image data obtained by enlarging the corresponding area is obtained. By combining with image data, it is possible to create a composite image that can accurately recognize characters.

<Second Embodiment>
In the document camera apparatus 100 according to the first embodiment, when the operation mode is the “character recognition mode”, the designated area is magnified and photographed in a predetermined area at a constant magnification as necessary. The obtained enlarged image data is used for character recognition processing in the subsequent stage, thereby performing character recognition with high accuracy.
On the other hand, in the document camera device 100 according to the present embodiment, a mode in which enlarged image data obtained by enlarging an area designated at one or more types of enlargement magnifications is used in the subsequent character recognition processing will be described. Note that items different from the first embodiment will be described below.

<Shooting condition determination unit>
FIG. 15 is a diagram illustrating a functional configuration of the imaging condition determination unit 308 according to the present embodiment.
The area extracting unit 308d extracts a character area to which magnification information indicating the highest magnification is given from the magnification information given to the pre-image data. The number of character areas extracted here is one, and when there are a plurality of character areas to which the highest magnification is given, the character area closer to the origin coordinate is extracted.
The shooting range determination unit 308e includes a shooting range that includes the largest number of constituent pixels of the extracted character area and, if possible, includes a large number of constituent pixels of other character areas to which magnification information is added. Determine the center position coordinates. The shooting range at this time is a range when shooting is performed at an enlargement magnification assigned to the extracted character area. When all the extracted character areas do not fit within the shooting range, a plurality of position coordinates are determined so that the shooting range is continuous.

Further, the region extraction unit 308d identifies a character region to which magnification information indicating the highest magnification is given from among character regions having constituent pixels in the region excluding the region already determined as the shooting range from the pre-image data. .
Thereafter, the area identification unit 308d and the shooting range determination unit 308e repeatedly perform the same processing flow until all the constituent pixels of the character area to which the magnification information is added fall within the shooting range.

<Shooting range determination unit>
FIG. 16 is a diagram illustrating a method for determining the position coordinates of the center of the shooting range by the shooting range determination unit 308e.
In FIG. 16, among the magnification information given to the pre-image data, there are two character areas to which the magnification information indicating the highest magnification is given, the character area 2 and the character area 3, and the area extraction unit 308d 1 is extracted. Since there is no character area that is provided with magnification information and is not included in the shooting range around the character area 2, the shooting range determination unit 308 e has an A area that is a shooting range with a magnification of 3 × that includes only the character area 2. Is determined as the position coordinate of the center of the imaging range.
Here, the periphery of the character area 2 is a set of areas including the character area 2. For example, the A area when the upper left coordinates of the A area are aligned with the upper left coordinates of the character area 2, and the upper right of the character area 2. The A area when the upper right coordinates of the A area are aligned with the coordinates of the A area, the A area when the lower left coordinates of the A area are aligned with the lower left coordinates of the character area 2, and the lower right coordinates of the character area 2 An area represented by a set with the A area when the lower right coordinates of the A area are combined.

  Next, the area extracting unit 308d extracts the character area 3 that shows the highest magnification among the areas excluding the A area. Since the magnification information is given around the character area 3 and there is a character area 4 which is a character area that is not included in the already determined shooting range, the area extraction unit 308d determines that the character area 3 and the character area 4 is determined as the position coordinates of the center of the shooting range. Since the magnification information given to the character area 2 and the character area 3 is the same magnification, whichever comes first in the extraction order. As described above, the shooting range determination unit 308e determines the shooting range based on the positional relationship between the plurality of character areas.

  Next, the character area 5 showing the highest magnification is extracted from the areas excluding the A area and the B area. Since there is no character area around the character area 5 to which magnification information is assigned and which is not included in the already determined shooting range, the shooting range determination unit 308e has a shooting range with a magnification ratio of 2 including only the character area 5. Is determined as the position coordinate of the center of the imaging range.

  As described above, the area extracting unit 308d extracts the character area to which the maximum magnification information is assigned from the areas excluding the area that has already been determined as the shooting range, and then around the area including the extracted character area. It is determined whether or not there is a character area to which magnification information is assigned and which is not included in the already determined shooting range, and if it exists, the shooting range is determined so as to include the character area. The area extraction unit 308d repeats a series of processes until all the character areas to which the magnification information is added are included in the shooting range. In the present embodiment, when only a part of the target character area is included in the shooting range, it is assumed that the remaining part of the target character area is not included in the shooting range.

  In addition, as a method of determining the shooting range in the area extraction unit 308d, a character area including magnification information that is not included in the already determined shooting range is searched around the character area to which the maximum magnification information is assigned. The state in which the content ratio of the maximum value may be determined as the imaging range.

The creation unit 308f displays the center coordinates of the A area (200, 100), the center coordinates of the B area (500, 100), and the center coordinates of the C area (550, 300). The magnification information indicating the shooting magnification is input to the shooting condition table, and “character recognition mode” is input as the operation mode.
<Shooting condition table>
FIG. 17 is a photographing condition table showing information output to the photographing apparatus 101 at this time.
The imaging apparatus 101 refers to the imaging condition table, rotates the lens so that the input coordinates become the center point, and enlarges the imaging range with the input magnification.

<Processing of shooting condition determination unit>
FIG. 18 is a flowchart for explaining the processing flow of the imaging condition determination unit 308 according to this embodiment. FIG. 18 corresponds to the processing in S108 shown in FIG.
In S401, the region extraction unit 308d extracts a character region to which the maximum enlargement factor is assigned. In S402, the shooting range determination unit 308e includes the extracted character area, and if possible, the position coordinates of the center of the shooting range in which many constituent pixels of other character areas to which magnification information is added are included in the shooting range. To decide.
In step S403, the area extraction unit 308d confirms whether the pixels included in all the character areas to which the magnification information is added are included in the shooting range. If there is a constituent pixel of the character area to which the magnification information is added outside the shooting range in S403 (S403, No), the process proceeds to S404.
In step S404, the area extraction unit 308d extracts a character area to which the maximum enlargement magnification is given, other than the area determined as the shooting range. When the extraction is completed in S404, the process returns to S402.
In S403, when there is no constituent pixel of the character area to which the magnification information is added outside the shooting range (S403, Yes), the process proceeds to S405, and the creation unit 308f creates a shooting condition table, and the shooting apparatus 101 stores the shooting condition. Output the table and finish the process.

  As described above, according to the document camera device 100 according to the present embodiment, the area that needs to be enlarged is determined from the pre-image data that is taken without using the zoom, and the enlarged image data obtained by enlarging the corresponding area is obtained. By combining with image data, it is possible to create a composite image that can accurately recognize characters.

  Compared to the first embodiment, by shooting at a plurality of magnifications according to the conditions, a composite image can be created that can recognize characters more accurately than pre-image data with a lower number of shootings more efficiently. It becomes possible to do.

  Compared with the first embodiment, since the shooting range is determined so as to include a plurality of character areas to which magnification information is added, the character recognition is performed more efficiently than the pre-image data with a smaller number of shooting times. A composite image that can be obtained can be obtained.

<Third Embodiment>
In the document camera device 100 according to the first embodiment and the second embodiment, the designated portion is magnified by rotating the photographing device 101.
On the other hand, the document camera apparatus 900 according to the present embodiment will be described with respect to a mode in which a designated portion is magnified by using the movable document table 106 that is freely movable in the plane direction. In the following, items different from those in the first embodiment and the second embodiment will be described.

FIG. 19 illustrates the overall configuration of the third embodiment in which a specified portion is enlarged and photographed using a movable document table 106 that is freely movable in the plane direction instead of the fixed document table 105 in FIG. It is a figure to do.
In the present embodiment, the CPU 201 refers to the shooting range input in the shooting condition table, and controls the movable document table 106 so that the center of the input shooting range is located directly below the shooting apparatus 101. The image capturing apparatus 101 captures an image with an enlargement magnification input in the image capturing condition table.

  As described above, according to the document camera apparatus 900 according to the present embodiment, the area that needs to be enlarged is determined from the pre-image data that is taken without using the zoom, and the enlarged image data obtained by enlarging the corresponding area is obtained. By combining with image data, it is possible to create a composite image that can accurately recognize characters.

  Compared with the first embodiment and the second embodiment, the movable original table 106 is movably controlled so that the designated portion is located directly below the photographing apparatus 101, so that an enlarged image with less distortion is obtained. And the accuracy of character recognition is improved.

  While specific embodiments of the present invention have been described above, the above-described embodiments are examples of the present invention. The present invention is not limited to the above-described embodiments, and can be embodied with various modifications and changes without departing from the scope of the invention in the implementation stage.

<Configuration, operation and effect of exemplary embodiment of the present invention>
<First aspect>
The document camera device 100 (image composition device) of this aspect is calculated with a character region size calculation unit 305 that calculates a character region size for each character region included in a captured image acquired by capturing a predetermined capturing range. Based on the character area size for each character area, an enlarged shooting necessity determination unit 307 that determines whether or not enlargement shooting is necessary within a predetermined shooting range, and an enlargement shooting necessity determination unit 307 determine that enlargement shooting is necessary. In this case, based on the calculated character area size for each character area and the position for each character area, the image capturing range determination unit 308b that determines the enlargement image capturing range, and the enlargement acquired by enlarging the image capturing area. An image composition unit 309 that composes the photographed image and the photographed image to create a composite image;
According to this aspect, the character region size calculation unit 305 calculates the character region size for each character region included in the captured image acquired by capturing a predetermined capturing range. Based on the calculated character area size for each character area, the magnified shooting necessity determination unit 307 determines whether or not expansion shooting within a predetermined shooting range is necessary. When the shooting range determining unit 308b determines that the enlarged shooting is necessary by the enlarged shooting necessity determination unit 307, the enlarged shooting range is determined based on the calculated character region size for each character region and the position for each character region. To decide. The image composition unit 309 synthesizes an enlarged photographed image acquired by enlarging the enlarged photographing range and the photographed image to create a composite image.
This makes it possible to recognize characters with high accuracy by enlarging a necessary area.

<Second aspect>
The magnified shooting necessity determination unit 307 according to this aspect determines that the magnified shooting is necessary when the calculated minimum character area size is smaller than a predetermined value.
According to this aspect, since the magnified shooting necessity determination unit 307 determines that the magnified shooting is necessary when the calculated minimum character area size is smaller than the predetermined value, the necessary area is magnified. This enables accurate character recognition.

<Third aspect>
The shooting range determination unit 308b according to this aspect determines the divided area as the enlarged shooting range when a character area having a character area size smaller than a predetermined value is included in the area obtained by dividing the predetermined shooting range.
According to this aspect, the shooting range determination unit 308b determines the divided area as the enlarged shooting range when a character area having a character area size smaller than a predetermined value is included in the area obtained by dividing the predetermined shooting range. The necessary area can be enlarged and the character recognition can be performed with high accuracy.

<4th aspect>
The shooting condition determination unit 308 of this aspect determines the number of divisions of a predetermined shooting range based on the minimum character region size calculated by the character region size calculation unit 305.
According to this aspect, the shooting condition determination unit 308 determines the number of divisions of the predetermined shooting range based on the minimum character region size calculated by the character region size calculation unit 305. Accordingly, the divided area is determined as the enlarged photographing range, so that the necessary area can be enlarged and the character recognition can be performed with high accuracy.

<5th aspect>
The shooting range determination unit 308e of this aspect determines an enlarged shooting range based on the positional relationship between a plurality of character areas.
According to this aspect, since the imaging range determination unit 308e determines the enlarged imaging range based on the positional relationship between the plurality of character areas, it is possible to enlarge the necessary area according to the determined enlarged imaging range. This enables accurate character recognition.

<Sixth aspect>
The image composition unit 309 of this aspect replaces a partial area of the captured image while maintaining the resolution of the enlarged captured image corresponding to the partial area of the captured image or the partial area of the enlarged captured image. Create a composite image.
According to this aspect, the image composition unit 309 holds a partial area of the captured image as an enlarged captured image corresponding to the partial area of the captured image or the resolution of the partial area of the enlarged captured image. Since the composite image is created by replacing the image as it is, an enlarged image data area can be obtained with a higher resolution than the original pre-image data area.

<Seventh aspect>
The document camera apparatus 100 (image composition apparatus) of this aspect further includes a character recognition unit 310 that performs character recognition processing on the composite image.
According to this aspect, since the character recognition unit 310 performs character recognition processing on the composite image, the accuracy of character recognition is improved.

<Eighth aspect>
The image composition method of this aspect includes a character area size calculating step (S106) for calculating a character area size for each character area included in a captured image acquired by capturing a predetermined capturing range, and for each calculated character area. Based on the character area size, an enlarged shooting necessity determination step (S107) for determining whether or not enlargement shooting is required within a predetermined shooting range and an enlargement shooting necessity determination step (S107) determine that enlargement shooting is necessary. In this case, an imaging range determination step (S302) for determining an enlarged imaging range based on the calculated character area size for each character area and a position for each character area, and acquisition by enlarging the enlarged imaging range. An image composition step (S110) is performed in which the enlarged photographed image and the photographed image are synthesized to create a composite image.
According to this aspect, in the character area size calculating step (S106), the character area size is calculated for each character area included in the photographed image acquired by photographing the predetermined photographing range. In the enlargement shooting necessity determination step (S107), the necessity of enlargement shooting within a predetermined shooting range is determined based on the calculated character area size for each character area. When it is determined in the shooting range determination step (S302) that enlargement shooting is necessary in the enlargement shooting necessity determination step (S107), based on the calculated character area size for each character area and the position for each character area. To determine the magnified shooting range. In the image composition step (S110), a magnified image obtained by enlarging the magnified photographing range is combined with the photographed image to create a composite image.
This makes it possible to recognize characters with high accuracy by enlarging a necessary area.

<Ninth aspect>
The document camera device 100 (image composition system) of this aspect includes a photographing device 101 (photographing unit) that acquires a photographed image obtained by photographing a predetermined photographing range or an enlarged photographed image obtained by enlarging a predetermined photographing range
A character area size calculation unit 305 that calculates a character area size for each character area included in the captured image;
Based on the calculated character area size for each character area, the enlarged shooting necessity determination unit 307 for determining whether or not the enlarged shooting is necessary within the predetermined shooting range and the enlarged shooting necessity determination unit 307 require the enlarged shooting. If a certain determination is made, an image capturing range determination unit 308b that determines an enlarged image capturing range based on the calculated character region size for each character region and a position for each character region, and an enlarged image capturing range are acquired by performing an image capturing operation. And an image composition unit 309 that synthesizes the enlarged image and the photographed image to create a composite image.
According to this aspect, the imaging device 101 (imaging unit) acquires a captured image obtained by photographing a predetermined photographing range or an enlarged photographed image obtained by enlarging a predetermined photographing range. A character area size calculation unit 305 calculates a character area size for each character area included in the captured image. Based on the calculated character area size for each character area, the magnified shooting necessity determination unit 307 determines whether or not expansion shooting within a predetermined shooting range is necessary. When the shooting range determining unit 308b determines that the enlarged shooting is necessary by the enlarged shooting necessity determination unit 307, the enlarged shooting range is determined based on the calculated character region size for each character region and the position for each character region. To decide. The image composition unit 309 synthesizes an enlarged photographed image acquired by enlarging the enlarged photographing range and the photographed image to create a composite image.
This makes it possible to recognize characters with high accuracy by enlarging a necessary area.

DESCRIPTION OF SYMBOLS 100, 900 ... Document camera apparatus, 101 ... Shooting apparatus, 102 ... Illumination, 103 ... Supporting member, 104 ... Input I / F, 105 ... Original plate, 106 ... Movable original plate, 201 ... CPU, 202 ... RAM, 203 ... ROM, 204 ... storage device, 205 ... external I / F, 301 ... mediation section, 302 ... temporary storage section, 304 ... character area determination section, 305 ... character area size calculation section, 306 ... size information storage section, 307 ... Enlarged shooting necessity determination unit, 308... Shooting condition determination unit, 309... Image composition unit, 310... Character recognition unit, 311.

JP 2006-094082 A JP 2005-301672 A

Claims (9)

A character area size calculating unit that calculates a character area size for each character area included in a captured image acquired by photographing a predetermined photographing range;
Based on the calculated character area size for each of the character areas, an enlarged shooting necessity determination unit that determines whether enlargement shooting is required within the predetermined shooting range;
Shooting that determines an enlarged shooting range based on the calculated character area size for each character area and the position for each character area when the enlarged shooting necessity determination unit determines that enlargement shooting is necessary. A range determination unit;
An image composition unit that composes an enlarged photographed image obtained by enlarging the magnified photographing range and the photographed image to create a composite image;
An image synthesizing apparatus.   The image synthesizing apparatus according to claim 1, wherein the enlargement photographing necessity determination unit determines that enlargement photographing is necessary when the calculated minimum character area size is smaller than a predetermined value.   The said imaging | photography range determination part determines the said divided | segmented area | region as said enlarged imaging | photography range, when the area | region which divided | segmented the said predetermined | prescribed imaging | photography range contains the character area of a character area size smaller than predetermined value. Image synthesizer.   The image synthesizing apparatus according to claim 3, further comprising a shooting condition determining unit that determines the number of divisions of the predetermined shooting range based on a minimum character region size calculated by the character region size calculating unit.   The image synthesizing apparatus according to claim 1, wherein the shooting range determination unit determines the enlarged shooting range based on a positional relationship between a plurality of character areas.   The image composition unit replaces a part of the photographed image with the resolution of the enlarged photographed image corresponding to the part of the photographed image or the part of the magnified photographed image being retained, and composes it. The image composition device according to any one of claims 1 to 5, which creates an image.   The image synthesizing apparatus according to claim 1, further comprising a character recognition unit that performs character recognition processing on the synthesized image. A character region size calculating step for calculating a character region size for each character region included in a captured image acquired by capturing a predetermined capturing range;
An enlarged shooting necessity determination step for determining whether or not enlargement photographing within the predetermined photographing range is necessary based on the calculated character region size for each character region;
Shooting for determining an enlarged shooting range based on the calculated character area size for each character area and the position for each character area when it is determined in the enlargement shooting necessity determination step that enlargement shooting is necessary. A range determination step;
An image synthesis step of creating a synthesized image by synthesizing the enlarged photographed image acquired by enlarging the magnified photographing range and the photographed image;
The image composition method to execute. A photographing unit that obtains a photographed image obtained by photographing a predetermined photographing range or an enlarged photographed image obtained by enlarging the predetermined photographing range;
A character area size calculation unit for calculating a character area size for each character area included in the captured image;
Based on the calculated character area size for each of the character areas, an enlarged shooting necessity determination unit that determines whether enlargement shooting is required within the predetermined shooting range;
Shooting that determines an enlarged shooting range based on the calculated character area size for each character area and the position for each character area when the enlarged shooting necessity determination unit determines that enlargement shooting is necessary. A range determination unit;
An image composition unit that composes an enlarged photographed image obtained by enlarging the magnified photographing range and the photographed image to create a composite image;
An image composition system comprising:

Images