JP2012243120A - Image processing program, image processing device and image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make it possible to determine precisely whether or not an image of an object other than a subject is included in a captured image.SOLUTION: An image processing device 10 includes: a captured image storage section 61 for storing a captured image of a mounting table captured by an image capturing section 71 of a stand-type image scanner 50; a contour extraction processing section 63 for extracting a contour image from the captured image stored in the captured image storage section 61 to store the contour image in a contour image storage section 62; a circularity and straight line number calculating section 64 for calculating circularity of the contour image and the number of straight lines constituting the contour image on the basis of the contour image stored in the contour image storage section 62; and a hand reflection determination section 65 for determining whether or not a hand reflection is included in a business form image on the basis of the calculated circularity and the calculated number of the straight lines.

Description

  The present invention relates to image processing, and more particularly to an image processing program, an image processing method, and an image processing apparatus for processing an image captured by an image sensor.

  In a financial institution or the like, a form reading device is used for reading an image of a form. This form reading apparatus includes a stand-type image scanner as an apparatus for acquiring a form image. This stand-type image scanner includes a table for placing a form, a form abutting guide for helping to correctly place the form on the stage, a CCD camera as an image sensor, and a distance from the table to the CCD camera. A supporting column is provided (see, for example, Patent Documents 1 and 2). The CCD camera included in the stand-type image scanner is a two-dimensional area sensor that captures a two-dimensional image, and captures a form image by photographing the form placed on the table. The stand-type image scanner transmits an image captured by itself to a host device (for example, a personal computer) via a communication interface such as USB (Universal Serial Bus). The host device generates a form image (form image) based on the image data received from the stand-type image scanner. The generated form image is subjected to character recognition processing, and necessary character information is extracted from the form image.

  As a feature of the CCD element used in the stand-type image scanner, a field readout method is used in which one image is divided into a plurality of partial images for high-definition reading. That is, the stand-type image scanner generates a whole form image by synthesizing partial images (divided images) of the form.

  In a form reading apparatus, an instruction to start reading an image of a form with respect to a stand-type image scanner is generally performed when an operator presses a photographing button (reading key) provided in a host apparatus. In this case, the operator needs to press the shooting button after confirming that his / her hand is out of the shooting area of the CCD camera after the form is correctly placed on the table. However, if the operator rushes to read the form, the timing of pressing the shooting button is too early, and the CCD camera starts shooting while the operator's hand is in the form shooting area. There was a problem of being reflected. In this way, if the hand of the form image is photographed when the form is photographed, the hand image is included in a part of the form image generated by the higher-level device. A recognition error occurs and the form needs to be taken again. As a result, the efficiency of the form processing operation decreases.

Such hand-drawing at the time of reading the image of the form is reduced by the operator's familiarity, but it is a problem because it accounts for most of the factors of the initial reading error of the form reading apparatus.
Further, as described above, since the photographing of the form by the stand-type image scanner is executed a plurality of times, the CCD camera starts to pick up the form image after receiving an instruction to start reading the form image by the operator. Since the host device takes a long time to generate the form image, the time until the host device starts the character recognition processing of the form and the time until the form image is displayed on the screen of the display device. It will be long. For this reason, the stand-type image scanner is configured to start image capturing of the form by the CCD camera before the operator instructs the image reading of the form. This technique is generally called “pre-scan” (see, for example, Patent Document 3).

  However, when pre-scanning is performed, there is a problem that the operator's hand is included in the divided image. This inevitably occurs due to the relationship between the time from when the operator correctly places the form on the table until the operator issues an instruction to start reading the image of the form, and the time from when the operator releases his / her hand from the table. In this way, when pre-scanning is performed, if a hand image is included in the divided image, the hand apparatus is reflected in the form image generated by combining the plurality of divided images, and character recognition of the form image is performed. Doing so is a problem because an error occurs. For this reason, conventionally, there has been proposed a technique for detecting whether a form image obtained by pre-scanning has a hand-drawn image (see, for example, Patent Document 3).

  In Patent Literature 3, by detecting whether or not the average gradation value (gray value) of each pixel in a predetermined area included in the composite image synthesized from the divided image by the higher-level device exists in a predetermined range, or By detecting whether or not each pixel included in the composite image includes a predetermined range of pixels that represent a predetermined range of gradation values (for example, skin tone gradation values) in the composite image, A method for determining whether or not a hand-drawn area (hand-shot image) exists is disclosed. In addition, a method is disclosed in which the time required for hand-shooting determination is shortened by performing the above-described hand-shaking determination for only the hand-shooting area.

JP 2001-78176 A JP-A-8-9102 JP 2010-57079 A

  The method disclosed in Patent Document 3 is based on whether the image processing device (the host device of the stand-type image scanner) determines whether or not a hand-shooting area exists in a composite image obtained by pre-scanning. There is an effect that a form image having no area can be acquired in a short time. In addition, the operator's work of confirming whether or not the hand-drawn area exists in the composite image while visually observing the composite image displayed on the monitor screen is not necessary, so that the operator's work load is reduced. Also have. However, the technique disclosed in Patent Document 3 is based on the premise that “the average gradation value of the outer peripheral area (determination area) of the form becomes thinner when there is a hand-drawing”. However, when actually using a stand-type image scanner, the color of the sleeves of the clothes worn by the operator may be close to the skin color. Sometimes it ends up. Further, according to the method of Patent Document 3, a case where a form is not correctly placed on a form placement table (table) is also determined as a hand-shot. In a stand-type image scanner, a form is routinely placed outside the form setting table. In such a case, according to the method of Patent Document 3, if a form is not correctly placed on the form placement table even after a certain period of time has elapsed, it is regarded that the presence of hand-shooting has been detected a predetermined number of times, and an error has occurred. Output (error notification output indicating that reading is impossible) is performed. Therefore, the technique of Patent Document 3 has a drawback that it is impossible to accurately detect hand-drawing.

  In recent form image character recognition apparatuses, a method of automatically starting character recognition when a form image is acquired is the mainstream. There is a method of rereading the form image when the operator confirms the image and there is a hand-shot, but this method is not performed at present because it is not automated. However, for devices that automatically start character recognition for forms, even if character recognition fails due to hand-drawing, the operator must determine whether hand-printing is the cause of the error or the character shape of the character written on the form. There is a problem that it is not possible to distinguish whether the bad is the cause of the error. For this reason, in an apparatus that automatically starts character recognition of a form, it is very important to automatically and accurately determine hand-drawing on the apparatus side.

  An object of the present invention is to automatically and accurately determine whether or not an object other than a subject to be photographed is shown in a photographed image.

  In order to solve the above problems, an image processing program for processing an image photographed by an image sensor is provided. One aspect of the image processing program is executed by a computer, whereby the computer extracts a contour of a subject portion from a photographed image of the subject and creates a contour image including the subject portion, and the contour Based on the contour image created by the image extraction unit, a feature amount calculation unit that calculates a feature amount necessary to determine whether or not an image of an object other than the subject is included in the contour image; Based on the feature amount obtained by the feature amount calculation means, the contour image functions as a determination means for determining whether an image of an object other than the subject is included.

  Further, another aspect of the image processing program is executed by a computer, so that the computer can acquire a pre-scan unit that captures a subject with an image capturing unit before the operator gives a shooting instruction, and the pre-scan unit. A contour image extracting means for extracting a contour of the subject portion from the captured image and creating a contour image including the subject portion, and an image of an object other than the subject is included in the contour image created by the contour image extracting means. Based on the feature quantity obtained by the feature quantity calculation means for calculating whether or not the feature quantity is necessary for determining whether or not the object is present, an image of an object other than the subject is included in the contour image. A determination unit that determines whether or not the image is included, and when the determination unit determines that an image of an object other than the subject is included in the contour image, Based on at least two captured images obtained by the pre-scan means, calculation means for calculating the time until no object other than the subject appears in the captured image captured by the image-capturing means, and photography by the pre-scan are inherently necessary In the case of shooting at a resolution lower than the resolution to be taken, the imaging unit again takes a picture of the subject again at the necessary resolution at an appropriate timing according to the time calculated by the calculation unit. It functions as a scanning means.

  For example, the calculation means stores data relating to the movement amount for each operator in the movement amount data storage means every time the operator gives a photographing instruction, and the number of times data relating to the movement amount is stored in the movement amount data storage means is constant. For the operator who has reached the number of times, the average value of the data related to the movement amount of the operator so far is calculated, and the average value obtained by the calculation is stored in the movement amount information storage means. When calculating the time until no object other than the subject appears in the captured image taken by the means, the average value of the operator stored in the movement amount data storage means is used.

  According to the image processing apparatus program, the image processing method, and the image processing apparatus, it is possible to automatically and accurately determine whether or not an object other than the subject to be photographed is reflected in the photographed image.

It is a figure which shows the hardware constitutions of the whole system of embodiment of this invention. It is a functional block diagram which shows the function structure of the 1st Embodiment of this invention. It is a figure explaining the principle of the 1st method of determining whether the hand-drawn image is included in the image synthesize | combined by the prescan in embodiment of this invention. It is a figure explaining the principle of the 2nd method of determining whether the hand-drawn image is included in the image synthesize | combined by the prescan in embodiment of this invention. It is a figure explaining the principle of the determination method of the reading start timing of the form in embodiment of this invention. It is a figure which shows the example in the case of producing an outline image from the prescan image without hand-drawing. It is a figure which shows the example in the case of producing an outline image from the prescan image with a hand-drawing. It is a figure which shows the calculation method of the area and perimeter in a prescan image without hand-drawing. It is a figure which shows the number of straight lines detected in the prescan image without a hand-drawing. It is a figure which shows the number of straight lines detected in the prescan image with a hand-drawing. It is a figure which shows the number of straight lines detected by prescribing | regulating the length of the straight line to detect in the prescan image with a hand-drawing. It is a flowchart which shows the algorithm which determines whether a pre-scan image has a hand-shot based on the circularity calculated | required from the outline image, and the number of straight lines. It is a functional block diagram which shows the function structure of the 2nd Embodiment of this invention. It is a table | surface which shows the temporal transition of "circularity" and "the number of straight lines" accompanying the temporal state transition of the form on the stand of the stand-type image scanner after the start of pre-scanning, and an operator's hand movement. It is a figure which shows the form image in each state of six states shown in FIG. It is a figure for demonstrating the method of calculating the time until a hand is no longer image | photographed based on two form images obtained by the prescan. It is a flowchart which shows the method of starting a main scan or post-processing after hand-shot in a prescan image is lost. 10 is a flowchart for explaining a method of predicting and calculating a time when a pre-scanned image disappears and starting main scan or post-processing. It is a figure explaining the case where the 1st and 2nd form of implementation of the image processing apparatus of this invention is applied when the thing put on the form accidentally protrudes from the form covering the outline of the form. . It is a figure explaining the case where the 1st and 2nd form of embodiment of this invention is applied when things like a sticky note are sticking in the form set correctly on the table. It is a figure explaining the case where the 1st and 2nd form of implementation of this invention is applied when the form has protruded from the imaging | photography range. It is a figure explaining the case where the 1st and 2nd form of implementation of this invention is applied when the foreign material is put in the imaging | photography range out of the form. It is a functional block diagram which shows the function structure of the 3rd Embodiment of this invention. It is a figure which shows the structural example of the table provided in the movement speed table memory | storage part shown in FIG.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
FIG. 1 is a diagram illustrating a hardware configuration of the entire system according to the embodiment of this invention.
1 includes an image processing apparatus 10, a monitor 20, a keyboard (KB) 30, a mouse 40, and a stand-type image scanner 50. The image processing apparatus 10 has an OCR (Optical Character Recognition) function for a form image, and needs to acquire a form image having a resolution required for the OCR process by the stand-type image scanner 50.

  The image processing apparatus 10 is a personal computer, for example, and includes a CPU (Central Processing Unit) 11, a RAM (Random Access Memory) 12, an HDD (Hard Disk Drive) 13, and a USB (Universal Serial Interface) interface 14. The CPU 11 is connected to the RAM 12, HDD 13, and USB interface 14 via the bus 15. Although not shown in FIG. 1, the image processing apparatus 10 also includes a NIC (Network Interface Card) for connecting to a network such as a ROM (Read Only Memory) or a LAN (Local Area Network). . The image processing apparatus 10 also includes a timer that measures time as necessary.

  Although not shown in FIG. 1, a display control unit is provided between the CPU 11 and the monitor 20 in the image processing apparatus 10, and the CPU 11 displays the screen of the monitor 20 via the display control unit. Do. Further, the CPU 11 reads / writes data from / to the RAM 12 via a memory controller (not shown) provided in the image processing apparatus 10. The image processing apparatus 10 also includes an HDD controller (not shown) (for example, a SATA (Serial Advanced Technology Attachment) controller), and reads / writes data to / from the HDD 13 via the HDD controller. The image processing apparatus 10 also includes a ROM that stores an IPL (Initial Program Load), a BIOS (Basic Input / Output System), and the like.

  The CPU 11 controls the entire image processing apparatus 10 by executing a program loaded in the RAM 12. The RAM 12 is a semiconductor memory capable of writing / reading data at any time, and stores an area where software such as an OS (Operating System) and application programs is loaded from the HDD 13 and data used when the CPU 11 executes processing. It has an area to do. The RAM 12 also includes an area for storing a captured image transferred from the stand-type image scanner 50 to the USB interface 14 of the image processing apparatus 10 and a work area used when processing the captured image.

  The HDD 13 is a storage device having an area for storing an OS, driver software, application programs, and data necessary for the CPU 11 to execute processing. The application program has, for example, a form image OCR processing function and a function for displaying the form image on the monitor 20. The driver software has a function of acquiring a form image without hand-drawing from an image taken by the stand-type image scanner 50 and passing the form image without hand-drawing to the application program having the OCR processing function. ing. Further, the HDD 13 includes an area for storing a form image with no handwriting acquired by the driver software.

  The USB interface 14 is connected to a USB terminal of each device of the keyboard 30, the mouse 40, and the stand type image scanner 50 through a USB cable, and conforms to the USB standard between the keyboard 30, the mouse 40, and the stand type image scanner 50. Control signals and data communication. An image photographed by the stand-type image scanner 50 is transferred from the stand-type image scanner 50 to the USB interface 14 and written into the RAM 12 via the USB interface 14.

  The monitor 20 is a display device such as a liquid crystal display, for example, and displays a user interface screen for an operator, a form screen read by the stand-type image scanner 50, and the like under the control of the CPU 11.

  The keyboard 30 and the mouse 40 are input devices used for an operation in which an operator instructs the image processing apparatus 10 to read a form. The operator operates the keyboard 30 and the mouse 40 to input the user interface screen displayed on the monitor 20 and instruct to start reading the form to the stand-type image scanner 50 via the user interface screen. To do. The operator operates the form reading start instruction by pressing the reading key provided on the keyboard 30 or clicking the reading key on the user interface screen displayed on the monitor 20 with the mouse 40.

  The stand-type image scanner 50 is a non-contact type image scanner for photographing a form (subject to be photographed) and obtaining a form image (form image), and the photographed image is imaged via a USB cable. Transfer to the USB interface 14 of the processing device 10. The stand-type image scanner 50 has substantially the same configuration as the form imaging device disclosed in FIGS. 1 and 3 of Patent Document 1, and includes a CCD camera 51, a table 52, a form abutment guide 53, and a column 54. Yes. The table 52 and the form match guide 53 have substantially the same configuration as the table 2 and the form match guide 3 disclosed in FIG. That is, the color of the surface of the table 52 is a color (for example, black) that can be clearly identified from the form. The stand-type image scanner 50 shoots a form placed (set) on the table 52 by the CCD camera 51 in a plurality of times. This is because the resolution of the CCD camera 51 is lower than the resolution required for the image processing apparatus 10 to perform OCR processing of the form image. For this reason, the CCD camera 51 captures (captures) one frame of the form image divided into a plurality of fields. The number of fields in one frame is, for example, 3 fields, 4 fields, or 6 fields. By the photographing by the CCD camera 51, a form image having a resolution that enables the image processing apparatus 10 to execute OCR processing of the form image is acquired. The photographing range of the CCD camera 51 is wider than the entire stage 52. The stand-type image scanner 50 cuts out an image having a size of “form + α” from the image taken by the CCD camera 51 and transfers the cut-out image to the image processing apparatus 10. Here, α represents a predetermined width (for example, 5 cm). For this reason, when the form is correctly placed on the table 52, the photographed image transferred from the stand-type image scanner 50 to the image processing apparatus 10 is a form image (form image) in the center and the entire periphery surrounds the entire form. The image of the outer edge of the mounting table 52 (a black background image composed of black pixels) is obtained (see FIG. 3 of Patent Document 1). Since the image of the outer edge portion of the form is a white pixel area, the form image can be extracted from the photographed image by scanning the photographed image from the outside to the inside and detecting the boundary where the black pixel changes to the white pixel.

  The image processing apparatus 10 synthesizes partial images (field images) acquired by the stand-type image scanner 50 by pre-scanning to generate a composite image having a predetermined resolution (hereinafter referred to as a pre-scan image for convenience), and It is determined whether or not the pre-scan image is included in the hand-shot.

  The stand-type image scanner 50 has two shooting modes of “moving image mode” and “still image mode” as shooting functions. Pre-scanning by the stand-type image scanner 50 is executed in the moving image mode or the still image mode. The moving image mode is a mode in which pre-scanning is performed at a resolution (low resolution) lower than that required for post-processing (such as OCR processing of a form image or screen display). The still image mode is a mode for performing pre-scanning at a resolution (high resolution) required for post-processing of a form image. One shooting time in the moving image mode (shooting time for one frame) is, for example, 30 ms. One shooting time in the still image mode is, for example, 500 ms. The post-processing is executed after “processing for correcting image magnification / distortion / skew” (image conversion) is performed on a high-resolution form image acquired by the stand-type image scanner 50. Therefore, in order to execute the post-processing, an image of one frame acquired by photographing in the still image mode is necessary.

  In the embodiment of the present invention, it is determined whether or not a form image without hand-drawing is acquired by pre-scanning, and when it is determined that a form image without hand-drawing is acquired by pre-scanning in the moving image mode. The main scan is started. That is, when the pre-scan is performed in the moving image mode, if it is determined that a form image without hand-drawing is acquired, the image stand type image scanner 50 starts re-imaging of the entire table 52 in the still image mode. The image acquired by the re-shooting is transferred from the stand-type image scanner 50 to the image processing apparatus 10. On the other hand, when the pre-scan is performed in the still image mode, when it is determined that a form image without hand-drawing has been acquired, a form image without hand-drawing with a resolution required for post-processing has already been obtained. Since the image is taken into the image processing apparatus 10, the stand-type image scanner 50 does not re-photograph the table 52. In the embodiment of the present invention, the main scan is performed by “photographing the entire table 52 on which the form is correctly placed at a resolution required for post-processing, and capturing the photographed image obtained by the photographing into the image processing apparatus 10.・ Process to memorize ”. For this reason, when the pre-scan is in the still image mode, a pre-scan obtained by capturing a form image without hand-drawing and capturing and storing the captured image in the image processing apparatus 10 corresponds to the main scan. Become.

[First Embodiment]
FIG. 2 is a functional block diagram showing a functional configuration of the image processing apparatus according to the first embodiment of the present invention having the hardware configuration shown in FIG.

The image processing apparatus 10-1 illustrated in FIG. 2 includes a captured image storage unit 61, a contour image storage 62, a contour extraction processing unit 63, a circularity / straight line number calculation unit 64, a handwriting determination unit 65, and an output unit 66. Yes. The storage areas of the photographed image storage unit 61 and the contour image storage unit 62 are provided in the RAM 12, for example. The CPU 11 loads the image processing apparatus program according to the embodiment of the present invention from the HDD 13 to the RAM 12 and executes it.
The stand type image scanner 50 includes an image photographing unit 71. The image capturing unit 71 captures the range including the pedestal 52 with the CCD camera 51 in a plurality of fields. The image photographing unit 71 cuts out a photographed image having a size of “form + α” from the image composed of a plurality of fields photographed by the CCD camera 51, divides the photographed image into a plurality of blocks, and stores the photographed image in units of blocks. Transfer to the unit 61. For convenience, this block-unit image is referred to as a block image. The image photographing unit 71 has two photographing modes of “moving image mode” and “still image mode” described above. The image capturing unit 71 captures an image of one frame (an image including a form and a surrounding black background image) in 30 ms in the moving image mode and in 500 ms in the still image mode.

  The captured image storage unit 61 includes a storage area for storing captured images transferred from the image capturing unit 71 in units of blocks. The captured image storage unit 61 receives and stores the block image transferred from the image capturing unit 71. When the captured image storage unit 61 receives images of all the blocks constituting one frame from the image capturing unit 71, the captured image storage unit 61 combines the block images. An image of one frame is generated, and the generated frame image is converted into a color image. Then, the color image is stored in a predetermined storage area. Of these frame images, the frame image acquired by the prescan is referred to as a “prescan image” as described above. When storing the frame image, the captured image storage unit 61 adds the acquisition time (frame image generation time) to the frame image and stores the frame image. The acquisition time is acquired from a timer (not shown) in the image processing apparatus 10.

  The contour image storage unit 62 includes a storage area for storing the contour image extracted by the contour extraction processing unit 63. Acquisition time information may be added to the contour image in the same manner as the pre-scan image.

  The contour extraction processing unit 63 reads a prescan image from the captured image storage unit 61, extracts a contour image from the prescan image by a method described later, and writes the contour image in the contour image storage unit 62. The contour extraction processing unit 63 extracts the contour image after converting the color pre-scan image stored in the photographed image storage unit 61 into a monochrome binary frame image. The contour extraction processing unit 63 extracts the contour of the image portion of the form and the contour of the hand image portion from the binary frame image, and extracts the image portion surrounded by the contour from the binary frame image. Extract images. This contour image is an image of the entire form in the case of a frame image without hand-drawing. On the other hand, the outline image in the case of a frame image with hand-drawing is an image obtained by deleting the hand-drawn image portion from the image of the entire form. Then, the extracted contour image is stored in the contour image storage unit 62.

  The circularity and straight line number calculation unit 64 reads the contour image from the contour image storage unit 62, and calculates the circularity described later based on the normally captured form image and contour image by a method described later. Further, the number of straight lines constituting the contour image is detected.

  Based on the degree of circularity, the degree of circularity and / or the number of straight lines obtained by the number-of-straights calculation unit 64, the hand-shot determination unit 65 determines whether or not there is a hand-shot in the captured image stored in the captured image storage unit 61. Determine whether. The hand-shot determination unit 65 includes a storage unit (not shown) that holds the pre-scan shooting mode (moving image mode or still image mode) of the image shooting unit 71. When the hand-shooting determination unit 65 determines that there is no hand-shooting in the captured image in the hand-shooting determination, if the pre-scan is in the moving image mode, it instructs the image capturing unit 71 to perform re-shooting. On the other hand, when the pre-scan is in the still image mode, the output unit 66 is notified of a form image output instruction.

  When the output unit 66 receives the notification of the output instruction of the form image from the handwriting determination unit 65, the output unit 66 reads out the form image with no handwriting from the photographed image storage unit 61 and outputs the form image to the monitor 20.

{Principle of hand-drawing judgment method}
Next, the principle of a method for determining whether or not the form image acquired by pre-scanning according to the embodiment of the present invention includes a hand-shot (hand-shooting determination method) will be described.

(1) Method Using Circularity FIG. 3 is a diagram for explaining a first method for determining whether or not a pre-scan image includes a hand-shot. FIG. 3A shows a prescan image including a hand-shot, and FIG. 3B shows a pre-scan image without a hand-shoot (a pre-scan image in which a form is normally captured). In FIGS. 3A and 3B, the outline of the form image is indicated by a dotted line.

The first technique uses circularity. Circularity is defined as follows.
Circularity = 4π × area / (periphery length × perimeter length) (1)
Area: Area of the form image (area of the contour image)
Perimeter: Perimeter of the form image (contour image contour length)
In the formula (1), the circularity is “1” in the case of a circle. The contour image will be described later.

  In the pre-scan images 100 and 200 shown in FIGS. 3A and 3B, black frame images (black background images) 101 and 201 become outer frames of the form when the form is correctly placed on the table 52. This is an image of the outer periphery of the pedestal 52, and white areas (white pixel areas) 102 and 202 are form images. 3A and 3B, dotted line portions 103 and 203 drawn around the white areas 102 and 202 indicate the outline of the form image. That is, the lengths of the dotted line portions 103 and 203 indicate the lengths of the outlines (peripheral lengths) of the form images 102 and 202. In the pre-scan image 100 shown in FIG. 3A, a black frame image and a part of the form image are covered with the hand image 104. The hand image 104 is an image of the operator's hand that was present at the top of the form when the CCD camera of the stand-type image scanner 50 took a picture.

  When the circularity is calculated using the above equation (1) for each of the pre-scanned image 100 with hand-drawing shown in FIG. 3A and the pre-scanned image 200 without hand-drawing shown in FIG. The pre-scan image 200 having no hand-drawing shown in FIG. 3B has a higher circularity than the pre-scan image 100 having the hand-drawing shown in FIG. This is because the area of the contour image 102 in the pre-scan image 100 with hand-drawing is smaller than the area of the contour image 202 in the pre-scan image 200 without hand-shooting, and the contour in the pre-scan image with hand-shaking This is because the peripheral length 103 of the image 102 is longer than the peripheral length 203 of the contour image 202 in the pre-scan image 200 with no hand-drawing. Therefore, by calculating the circularity of the pre-scanned image using the above formula (1) and comparing the circularity calculated by the calculation with an appropriate reference value, whether or not the hand-scanned image exists in the pre-scanned image. It becomes possible to determine. That is, when the circularity is lower than the reference value, it can be determined that there is a hand image.

(2) Method of using hand-drawn image captured in a line as a feature value In the stand-type image scanner 50, as described above, the resolution of the form image acquired by one shooting with the CCD camera 51 is required for post-processing. Since the resolution of the form image is lower, the table 52 on which the form is placed is shot in a plurality of times. In this case, one image (one frame) is generated from images of a plurality of fields having different exposure timings. For this reason, if the CCD camera 51 captures an image of the pedestal 52 while the operator's hand is moving above the pedestal 52, images of the hands appear in several field images.

  FIGS. 4A to 4C show an example in which the CCD camera 51 performs a pre-scan while shifting pixels in units of one field in the horizontal direction of the drawing. FIG. 4A is a diagram illustrating an example of a pre-scan image acquired when pre-scanning with the CCD camera 51 while the operator's hand is moving on the placement table 52. In this example, a vertically striped hand image 114 is shown below the right side of the form image area 112 of the pre-scan image 110. FIG. 4B is an enlarged view of a part of the hand image 114. FIG. 4B shows an example in which the CCD camera 51 shoots the table 52 in four steps. In this case, the pre-scan image 112 is composed of images of four fields taken at different times. In the example shown in FIG. 4B, a hand is captured in one of the four fields, and the reflected hand is captured as a vertical striped linear image.

  FIG. 4C shows an example of a pre-scan image 210 that is acquired when a black fill having the same shape as the hand shown in FIG. In this case, since the black-filled portion is stationary, it is copied when photographing all the fields. Therefore, as shown in FIG. 4C, the black-filled portion 215 that appears in the lower right side of the form image area 212 of the pre-scan image 210 is not a vertically striped linear image but an entire black image. It has become. Therefore, in the embodiment of the present invention, even when an image having a shape similar to the hand is shown in the form image area of the pre-scan image, it is detected whether or not the image includes a linear straight line. Thus, “hand image” and “filling of a form similar to the shape of the hand” are discriminated.

  Note that if the operator's hand remains stationary on the table 52 during the period in which the CCD camera 51 captures an image of one frame of the form, the hand image will not appear in a striped pattern in the pre-scan image. , The whole hand is reflected evenly. In such a case, it is possible to detect the hand image in the pre-scanned image by using the skin color tone value disclosed in the known document 1. Therefore, by combining the technique of the embodiment of the present invention that uses the feature that the hand image appears in a striped pattern and the hand detection technique of known literature, the image of the hand reflected in the pre-scan image can be reliably obtained. Can be detected.

{Principle of judgment method of image reading start timing for forms without hand-drawing}
Next, a method for determining the reading start timing of the form in the embodiment of the present invention will be described with reference to the timing sequence diagram of FIG.

  In the timing sequence diagram of FIG. 5, time advances in the direction of the arrow, that is, from the left side to the right side in the drawing. FIG. 5 shows “before the set operation” (a state before the operation of setting (mounting) the form on the stand 52 of the stand-type image scanner 50) and “during setting” (the form being placed on the stand-type image scanner 50). 52), “after setting” (state after the form is correctly set on the stand 52 of the stand-type image scanner 50), and “readable form” (when the stand-type image scanner 50 is Four states are shown in a state where the entire form can be read). The states of “form” and “operator's hand” in the above four types of form image states are “present” (present on the table 52), “no” (not present on the table 52), The symbols are “moving” (moving on the pedestal 52) and “stop” (still on the pedestal 52). In addition, a pre-scan image synthesized from partial images photographed by the CCD camera 51 in each state of the four kinds of wheels is shown. The prescan image 310 is an image photographed before the setting operation, the prescan image 320 is an image photographed during the setting, the prescan image 330 is an image photographed after the setting, and the prescan image 340 is in a readable state (manual image) This is an image that was taken when

Hereinafter, a method for determining the image reading start timing of a form in the embodiment of the present invention will be described in order of time.
(1) Before the operator sets a form on the stage 52 of the stand-type image scanner 50, the stage 52 is continuously pre-scanned by the CCD camera 51 at regular time intervals. Before the set operation, there is no form or operator's hand on the table 52, and therefore, in this state, the pre-scan image 310 shown in FIG. 5 is acquired. Since the pre-scan image 310 is an image obtained by photographing the entire upper surface of the mounting table 52, the entire image is a black image.

  (2) When the operator is setting a form on the table 52, both the form and the operator's hand are moving, so in this state, the pre-scan image 320 shown in FIG. 5 is acquired. The pre-scanned image 320 is an image in which a form in the middle of setting on the pedestal 52 and an operator's hand trying to set the form are projected as linear stripes.

  (3) When the operator finishes setting the form on the stand 52 of the stand-type image scanner 50, the operator presses the read key at time T0 and instructs the stand-type image scanner 50 to start reading the form image. In this example, the time T0 is before the operator finishes setting the form on the table 52, and the operator's hand is moving to set the form correctly on the table 52 at the time T0. Therefore, the prescan image acquired when the operator presses the read key is an image similar to the prescan image 320.

  (4) Immediately after the form is correctly set on the pedestal 52, the form is stationary, but the operator's hand is still moving on the pedestal 52. At this point, the pre-scan image 330 shown in FIG. 5 is acquired. . The pre-scan image 330 is an image in which a vertical stripe hand image is reflected in a form image.

  (5) Thereafter, when the operator's hand disappears from the form, a pre-scan image 340 shown in FIG. 5 is acquired thereafter. The pre-scan image 340 is an image composed of a form image with no hand-drawing and a black background image surrounding the outer edge of the form image.

  In the embodiment of the present invention, when the pre-scan image 340 is first acquired by pre-scan, it is automatically determined that the pre-scan image 340 has no hand-shot, and when the pre-scan is in the moving image mode, The main scan is started at the determined timing (time T1). On the other hand, when the pre-scan is in the still image mode, the post-processing is started based on the pre-scan image 340 having no hand-shooting stored in the captured image storage unit 61.

The main scan may be started not by the method (4) but by the following method (5).
(5) Since the hand is completely removed from the pre-scan image somewhere during the period from the acquisition of the pre-scan image 330 to the acquisition of the pre-scan image 340, the two acquired before the pre-scan image 330 is acquired By using the pre-scanned image, the timing until the pre-scan image without hand-shooting can be obtained is calculated, and scanning is performed at the timing (time Ti2) obtained by the prediction calculation (stand-type image scanner 50). (Shooting in still image mode or movie mode). In this prediction calculation, for example, the amount of movement of the hand image between two pre-scan images acquired at different times, the amount of change in circularity, or the like is used. Details of the prediction calculation method will be described later.

  Further, for each operator, each time a form is set on the table 52, the movement speed of the hand after the form is set on the table 52 (the amount of movement of the hand per unit time in the pre-scan image). Alternatively, by calculating and storing the circularity, the average moving speed or the average circularity for each operator is learned, and after the learning is completed, the scan is started at a timing suitable for each operator. Details of this learning method will also be described later.

  As described above, in the embodiment of the present invention, it is confirmed that the operator's hand is completely removed from the form image captured by the CCD camera 51 based on the image obtained by the prescan, or the CCD camera. 51 predicts the timing at which the operator's hand is removed from the form image book taken by 51. Then, after the confirmation, the main scan (when the prescan is in the moving image mode) or the post-processing (when the prescan is in the still image mode) is started. Further, the main scan (when the pre-scan is in the moving image mode) or the post-processing (when the pre-scan is in the still image mode) is started at the prediction timing.

{Examples of hand-drawing determination method}
Next, the processing procedure of the embodiment using the principle of the hand-drawing determination method described above will be described.
1. Find the circularity of the form.
Here, the “form” is an image of a form part in an image photographed by the CCD camera 51 in the prescan, and indicates a form image part of the photographed image stored in the photographed image storage unit 61.

  (1) A black-and-white change point (point that changes from a black pixel to a white pixel) is searched from the outer edge to the inside of an image read by pre-scan (pre-scan image), and the change point extracted by the search is defined as a contour. Create a contour image. The contour image creation processing will be described more specifically with reference to FIGS.

FIG. 6 shows an example in which a contour image is created from a pre-scanned image without hand-drawing.
A black and white change point is extracted by the above method for the pre-scanned image 410 having no hand-drawing shown in FIG. 6A, and a contour image 411 shown in FIG. 6B is created. The contour line 413 of the contour image 411 is equal to the contour line of the form. That is, the contour image 411 created from the pre-scan image 410 without hand-drawing is equal to the form image.

FIG. 7 shows an example in which a contour image is created from a pre-scan image with hand-drawing.
A pre-scan image 420 shown in FIG. 7A is an example of a form image with hand-drawing. A hand image 422 is shown on the right side of the pre-scan image 420. By extracting the black and white change points from the pre-scan image 420 by the above method, the contour image 421a shown in FIG. 7B or the contour image 421b shown in FIG. 7C is created. The contour image 421a is an image in which the contour of the three sides of the form (a part of the upper side, the left side, the right side, and the lower side and the part in the form image region of the hand image 422 is the contour line 423a. The image of the three sides of the form (upper side, left side and right side) and part of the lower side and a linear vertical stripe hand is the contour line 423b, that is, the difference between the contour image 421a and the contour image 421b is It is only the shape of the contour line of the hand part.Whether the contour image 421a or the contour image 421b is created can be easily made by changing the setting condition of contour extraction.

(2) The total number of pixels (= area) and the number of pixels (= peripheral length) in the contour image created in (1) above are calculated. The total number of pixels in the contour image includes the contour pixels.
With reference to FIG. 8, the calculation method of the area and the perimeter of a form image without hand-drawing will be described.

  FIG. 8 shows a contour image 411 and a contour line 413 of the pre-scan image 410 without hand-drawing. As shown in FIG. 8, in the case of the pre-scan image 410 with no hand-drawing, the area of the contour image 411 is equal to the area of the form image. The contour line 413 is a line having a width of one pixel. In the case of the contour image 411 of the pre-scan image 410 with no hand-shot, the total number of pixels in the form image 411 is calculated as an area, and the number of pixels in the contour line 413 is calculated as a perimeter.

  In the case of a pre-scanned image with hand-drawn images, the area is determined in the same manner as described above based on the contour images 421a and 421b with hand-drawn images and their contour lines 423a and 423b shown in FIGS. And ask for the perimeter. When calculating the perimeter of the contour line 421b of the contour image 412a, the number of pixels constituting the vertical stripe line is counted for the linear vertical stripe portion.

(3) Calculate the circularity of the form.
Using the above equation (1), the circularity of the form is calculated from the area and perimeter obtained as described above. Here, the circularity of the contour image without hand-drawing is set to “standard circularity” (reference value of circularity). The circularity of the contour image with hand-drawing is lower than the standard circularity. This is because in the case of a contour image with hand-drawing, the perimeter is longer than the perimeter of a contour image without hand-shooting, and the circularity is lower than the circularity of a contour image without hand-shooting.

2. The number of straight lines is obtained from the contour image.
Hough transform is performed on the pixels of the contour line, and a plurality of approximate straight lines constituting the contour line are detected. Then, the number of detected approximate straight lines is obtained. In the case of a contour image 411 with no hand-drawing as indicated by a number with a circle in FIG. 9, the four sides of the form are detected as approximate lines by the Hough transform. Therefore, when the contour image is a form image without hand-drawing, four lines (standard number of straight lines) are detected as the number of straight lines.

  On the other hand, in the case of a contour image with hand-drawn images, the number of detected straight lines is larger than the number of standard straight lines (four). This is because the contour shape of the hand is complicated in the case of the contour image 421a having the hand-drawing shown in FIG. Further, as shown in FIG. 10B, in the case of a contour image 421 having a hand-drawn image acquired when a hand moving on a form is photographed, the image portion of the hand has many linear vertical stripes. Consists of In the case of the contour image 421b, the large number of linear vertical stripes are detected as straight lines, so that a large number of straight lines are detected. In addition, the numbers with ◯ shown in FIGS. 10A and 10B represent the number of straight lines. In the case of FIG. 10B, the number of circles is partially omitted because the number of straight lines is too large.

  In this way, the number of straight lines detected from the contour line of the contour image with hand-shooting is larger than the number of straight lines detected from the contour line of the contour image without hand-shot, so By detecting the number of lines and determining whether or not the number of detected straight lines is greater than a predetermined number of standard straight lines (four in this embodiment), it is determined whether or not the pre-scan image has a hand-drawn image. It becomes possible to do.

{Modification of straight line number detection}
Also, by prescribing the length of the straight line (line segment) detected from the contour image, if the number of straight lines detected from the contour image is smaller than the predetermined standard straight line number, the pre-scan image has a hand-shot. It is also possible to make a determination. This method will be described with reference to FIG. 9 and FIG.

  If the length of the line segment to be detected from the contour image (the length of the straight line) is appropriately defined, the number of straight lines detected from the contour image 411 having no handwriting shown in FIG. 9 is four. That is, four sides of the form are detected as straight lines from the contour image 411. On the other hand, as shown in FIG. 11, in the case of a contour image 431a with hand-drawn, at least one of the four sides (the lower side in the example of FIG. 11) that is the contour of the form is covered with the hand. The side covered with is not detected as a straight line. For this reason, the number of straight lines detected from the contour image is less than four (in the example of FIG. 11, three lines (sides other than the lower side). Therefore, the length of the straight line (line segment) detected from the contour image is reduced. By prescribing, it is possible to determine that the pre-scan image has a hand-shot when the number of straight lines detected from the contour image is smaller than the standard number of straight lines. Therefore, there is no need to detect a straight line constituting the contour of the hand, and there is an advantage that the process of extracting the contour line from the contour image becomes easy.

{Hand-shooting judgment processing flow}
FIG. 12 is a flowchart showing an algorithm of a method for determining the presence / absence of hand-shot of a pre-scan image based on the circularity and the number of straight lines.

A processing procedure of the flowchart of FIG. 12 will be described.
First, it is determined whether “circularity ≧ standard circularity” (step S11). If it is determined Yes in step S11, the process proceeds to step S13, and it is determined in step S13 that “the hand-scanned image is not present in the pre-scan image”.

  If it is determined No in step S11, it is next determined whether “the number of straight lines = the number of standard straight lines” is satisfied (step S12). If “Yes” is determined in step S12, the process proceeds to step S13, and it is determined in step S13 that “the pre-scanned image is not photographed”.

If it is determined No in step S13, the process proceeds to step S14. In step S14, it is determined that “the hand-scanned image exists in the pre-scan image”.
Thus, in the method shown in the flowchart of FIG. 12, using both the circularity and the number of straight lines, when “circularity <standard circularity” and “straight number> standard straight line number”, It is determined that the pre-scan image has a hand-drawn image.

  In addition, it is also possible to determine the presence or absence of hand-shot of the form image by using the circularity or the number of straight lines alone without using both the circularity and the number of straight lines (state (a) to FIG. 14). (Refer to (f)). Moreover, the algorithm of the method using the modified example of the straight line number detection can directly apply the flowchart of FIG.

  According to the first embodiment of the present invention, a feature amount (circularity or number of straight lines) is extracted from a contour image extracted from a pre-scan image, and the extracted feature amount (circularity or number of straight lines) is a standard value. By comparing with (standard circularity or standard circularity), it is possible to accurately determine whether or not the pre-scan image has a hand-shot.

[Second Embodiment]
FIG. 13 is a block diagram showing a functional configuration of the second embodiment of the image processing apparatus of the present invention having the hardware configuration shown in FIG. In FIG. 13, the same components as those included in the image processing apparatus 10-1 of FIG. 2 are assigned the same reference numerals, and detailed descriptions of those components are omitted.

  An image processing apparatus 10-2 illustrated in FIG. 13 includes a movement amount calculation unit 67 and a hand-shake detection start timing calculation unit 68 (68-1) in addition to the components 61 to 66 included in the image processing apparatus 10-1 illustrated in FIG. ).

  The movement amount calculation unit 67 inputs the hand-shooting determination result from the hand-shooting determination unit 65, and determines whether or not a pre-scanned image with a hand-shooting is stored in the captured image storage unit 61 based on the hand-shooting determination result. to decide. Then, when the handshake determination result “with hand-shooting” is input twice from the hand-shooting determination unit 65, the pre-scan image corresponding to the previously input hand-shooting determination result with “hand-shooting” and the subsequent input Two images of the pre-scan image corresponding to the hand-shooting determination result of “with hand-shooting” are read from the captured image storage unit 61, and based on these two pre-scan images, “time interval Tec for checking movement amount” ( Check parameter interval Tec), “the amount of movement of the hand image L2”, and “the distance Lec until the hand-shooting disappears” are obtained, and the values of these three parameters are obtained as the hand-shooting detection start timing calculating unit 68. Output to -1. Note that the movement amount calculation unit 67 may obtain the values of the three parameters from the contour image corresponding to the pre-scan image having the two hand-shots stored in the contour image storage unit 62.

  The hand-shot detection start timing calculation unit 68-1 receives the “time interval Tec for checking the movement amount”, “the movement amount Lec of the hand image during the time interval Tec”, and “hand-shooting” input from the movement amount calculation unit 67. Based on the values of the three parameters “distance until disappearance L2”, a time T2 until hand-shooting disappears from the form image is calculated by a method described later.

  FIG. 14 shows temporal transitions of “circularity” and “number of straight lines” associated with temporal state transitions of the form and the movement of the operator's hand on the stand 52 of the stand-type image scanner 50 after the start of pre-scanning. It is a table | surface which shows. FIG. 14 shows the circularity and the number of straight lines in six states (a) to (f) in time order. FIGS. 15A to 15F show examples of pre-scan images corresponding to the states of FIGS. 14A to 14F, respectively. 15A, 15B, 15C, 15D, 15E, and 15F were acquired at times Ta, Tb, Tc, Td, Te, and Tf, respectively. It is an image. Each of the pre-scan images in FIGS. 15A to 15F is, for example, an image that is continuously captured at regular time intervals by the image capturing unit 71.

Here, the states (a) to (f) shown in FIG. 14 will be described.
The state (a) is a state where the operator is setting the form on the table 52, and both the form and the operator's hand are moving on the table 52. In this state (a), an image acquired by prescanning is a prescanned image 510 in which both a form and a hand to set the form are projected as vertical stripes, as shown in FIG. .

  The state (b) is a state where the operator's hand is stationary on the form while the operator is setting the form on the table 52 (or when the setting is finished), and the form is stationary on the table 52. . In this state (b), the image acquired by the pre-scan is a pre-scan image 520 in which both the form and the hand on the form are properly projected, as shown in FIG.

  The state (c) is a state in which the operator moves his / her hand away from the pedestal 52 after correctly setting the form on the pedestal 52, and the form remains stationary. In this state (c), the image acquired by the pre-scan is a pre-scan image 530 in which only the moving hand is displayed as vertical stripes as shown in FIG.

  The state (d) is a state in which the form is correctly set on the table 52 but the operator's hand is not completely removed from the table 52 and is stopped. In this state (d), the image acquired by the pre-scan is a pre-scan image 540 in which both the form and the hand on the form are projected stationary as shown in FIG.

  The state (e) is a state in which the form is correctly set on the table 52 and the operator's hand is about to come off the table 52. In this state (e), the image acquired by the pre-scan is a pre-scan image 550 in which the hand moving on the form is projected as vertical stripes as shown in FIG.

  The state (f) is a state in which the form is correctly set on the table 52 and the operator's hand is completely removed from the table 52. In this state (f), as shown in FIG. 15 (f), an image acquired by pre-scanning is a form set on the table 52 and a part of the table 52 surrounding the set form. A pre-scan image 560 in which (black frame) is projected.

  By calculating the circularity from the pre-scan images in the states (a) to (f) using the above-described method and comparing the circularity value with the standard circularity, the circularity is “low”, “medium”, When classified into three types of “high”, the circularity in the state (a) is “low”, the circularity in the states (b) to (e) is “medium”, and the circularity in the state (f) is “high”. Become. Therefore, only the state (f) can be selected by setting the standard circularity to “high”. In addition, the number of straight lines is obtained from the form images in the states (a) to (f) using the above-described method, and the value of the number of straight lines is compared with the standard number of straight lines (4). It is classified into two types, “many” (more than 4) and “4”. Then, the number of detected straight lines is “many” in the states (a) to (e), and only “4” is in the state (f). As a result, only in the state (f), the circularity is “high” and the number of straight lines is “4”. Therefore, by detecting a pre-scan image having a circularity of “high” and a straight line number of “4”, it is possible to detect a pre-scan image from which a form image without hand-drawing is obtained during pre-scan. . The detection time can be determined as the start timing of the main scan (when the prescan is in the moving image mode) or the post-processing (when the prescan is in the still image mode).

  By the way, after the operator correctly sets the form on the table 52, the timing at which the operator's hand is completely removed from the table 52 is between the state (e) and the state (f). Therefore, without waiting for detection of the state (f), a pre-scan image 550 having a hand-image acquired at the time of the state (e) and a pre-scan image having a hand-image acquired before that (in this example, for example, By using the pre-scan image 530) acquired in the state (c) and predicting the timing when the hand can be completely removed from the pre-scan image, the pre-scan image in the state (f) is acquired. The main scan (when the prescan is in the moving image mode) or the post-processing (when the prescan is in the still image mode) can be started.

  Next, referring to FIG. 16, a method of calculating the optimum main scan or post-processing start timing based on the above-mentioned idea using the pre-scan images 530 and 550 in the state (c) and the state (e) will be described. I will explain. The pre-scan images shown in FIGS. 16 (a) and 16 (b) are the same as the pre-scan images shown in FIGS. 15 (c) and 15 (e), respectively.

(1) The lengths L1 and L2 of the hand image in the direction parallel to the left and right frames of the black background image are obtained from the pre-scan images 530 and 550 shown in FIGS. In this case, L1 and L2 are set to the distance from the upper end of the hand image to the upper end of the lower frame of the black background image. Then, Lec shown in FIG. 16B is obtained by the following equation (2).
Lec = L1-L2 (2)

Further, the check time interval Tec is obtained from the acquisition time Tc of the prescan image 530 shown in FIG. 15C and the acquisition time Te of the prescan image 550 shown in FIG.
Tec = Te-Tc (3)

And finally, time T2 is calculated | required by following formula (4).
T2 = Tec × L2 / Lec (4)
T2 is the estimated elapsed time from the time Te at which the pre-scan image 550 is acquired to the time at which a form image without hand-drawing can be acquired. Therefore, when the pre-scan is in the moving image mode, it is possible to accelerate the start of the main scan by starting the main scan after the time T2 has elapsed from the time Te. When the pre-scan is in the still image mode, the start of the main scan can be accelerated by starting the main scan after the time T2 has elapsed from the time Te.

It is also possible to calculate the time T2 using the change amount of the circularity. The calculation formula for obtaining T2 in this case is the following formula (5).
T2 = Tec × C2 / Cec (5)
In equation (5),
C2: Circularity of T2 (circularity of contour image without hand-drawing)
Cec = (Circularity of contour image acquired at time Te) − (Circularity of contour image acquired at time Tc)
Tec = Te-Tc
It is.

  As described above, in the embodiment of the present invention, by using a pre-scan image having two hand-shots, the start timing of the main scan capable of obtaining a form image without hand-shooting (when the pre-scan is in the moving image mode) ) Or a timing at which post-processing can be started (when the pre-scan is in the still image mode) is automatically predicted, and main scanning or post-processing can be started at an earlier timing (see time Ti2 in FIG. 5).

{Procedure for starting main scan or post-processing after automatic determination that pre-scan image has no hand-drawing}
FIG. 17 is a flowchart showing a method (first method) of starting main scanning or post-processing after the pre-scan image has no hand-drawn images. The processing shown in this flowchart is executed by the image processing apparatus 10-1.

  In this method, first, pre-scanning for capturing a form by the image capturing unit 71 of the stand-type image scanner 50 is started (step S11). This pre-scan is started when the image processing apparatus 10 (10-1 to 10-3) is started and the image processing program (the driver software) according to the embodiment of the present invention is opened from the application software. . With the start of pre-scanning, the image capturing unit 71 of the stand-type image scanner 50 continuously performs pre-scanning at regular time intervals. The image photographed by the image photographing unit 71 by the prescan is transferred from the image photographing unit 71 to the USB interface 14 of the image processing apparatus 10 (10-1 to 10-3) in units of block images as described above. The CPU 11 generates a pre-scan image from the block image transferred from the image capturing unit 71 and stores the pre-scan image in the captured image storage unit 61. This pre-scan is performed in either a still image mode or a moving image mode. In the moving image mode, although the resolution is inferior, the shooting interval is shorter than that in the still image mode, so that the prescan image can be acquired at a higher speed than in the still image mode. As a result, it is possible to detect a pre-scan image without hand-drawing earlier than in the still image mode.

  Next, the contour extraction processing unit 63 extracts a contour image from the pre-scan image stored in the photographed image storage unit 61 and stores the contour image in the contour image storage unit 62. The circularity and straight line number calculation unit 64 calculates the circularity and the number of straight lines based on the contour image stored in the contour image storage unit 62 (step S12). Based on the circularity and the number of straight lines calculated by the circularity and the number-of-straight lines calculation unit 64, the hand-drawing determination unit 65 performs processing to determine whether there is a hand-shot in the form image in the pre-scan image (step S13). . The hand-shot determination unit 65 determines whether or not the pre-scan image has a hand-shot based on the determination result in step S13 (step S14). In step S14, Yes), the process returns to step S12.

  In this way, the processing from step S12 to step S14 is repeated until the hand-shooting determination unit 65 determines in step S14 that the pre-scan image has no hand-shooting. In other words, the processes in steps S12 to S14 are repeated until a pre-scan image without hand-drawing is acquired.

If the hand-shot determination unit 65 determines in step S14 that the pre-scanned image does not have a hand-shot (No in step S14), the process proceeds to step S15.
In step S15, the hand movement determination unit 65 determines whether or not the pre-scan is in the still image mode. If the pre-scan is in the still image mode (Yes in step S15), the process proceeds to step S17. (Step S15, No) The process proceeds to step S16.

  In step S16, the hand-shooting determination unit 65 instructs the image capturing unit 71 to capture in the still image mode, and the image capturing unit 71 re-captures the range including the entire table 52 in the still image mode. Then, the image photographing unit 71 transfers the image obtained by the re-photographing to the photographed image storage unit 61 in units of block images, and the photographed image storage unit 61 is a color image of one frame from the block image received from the image photographing unit 71. (A color form image and an image including a black background image surrounding the form image) are generated, and the frame image is stored in a predetermined area in the captured image storage unit 61. Then, the captured image storage unit 61 advances the processing to step S17.

  In step S <b> 17, the image conversion processing unit (not shown) performs image conversion processing (processing for correcting image magnification, distortion / skew) on a form image without handwriting stored in the photographed image storage unit 61. The form image that has been subjected to the image conversion process is stored in the captured image storage unit 61. The output unit 66 reads the form image that has been subjected to the image conversion processing from the captured image storage unit 61, and outputs the form image to the monitor 20 via the USB interface 14. An OCR processing unit (not shown) reads the form image subjected to the image conversion from the captured image storage unit 61 and executes character recognition processing of the form image.

As described above, the processing of the flowchart of FIG. 17 ends.
{Processing procedure for calculating the time when hand-shooting disappears and starting main scan or post-processing}
FIG. 18 is a flowchart illustrating a method (second method) for predicting and calculating the time when the pre-scanned image disappears and starting main scan or post-processing. The process shown in this flowchart is executed by the image processing apparatus 10-2.

  In this method, first, pre-scanning is started (step S31), and then the circularity and the number of straight lines are calculated based on the pre-scan image acquired in step S31 (step S32). Then, based on the circularity and the number of straight lines obtained in step S32, it is determined whether or not the pre-scanned form image has a hand-drawn image (step S33). Next, based on the determination result of step S33, it is determined whether or not the pre-scanned form image has a hand-drawn image (step S34). Since the processes of steps S31 to S33 are the same as the processes of steps S11 to S13 in the flowchart of FIG. 18 described above, detailed description thereof is omitted.

  If the hand-shot determination unit 65 determines in step S34 that the pre-scan image has a hand-shot (Yes in step S34), the process proceeds to step S35. On the other hand, if it is determined that the pre-scan image has no hand-drawn (step S34, No), the process proceeds to step S41.

  In step S35, the hand-shooting determination unit 65 determines whether or not it has been determined that the pre-scanned image is still hand-held in the previous time (step S35). The process in step S35 is a process for determining whether or not it is determined that there is a hand-shot in two consecutive pre-scan images. In step S35, if the hand-shooting determination unit 65 determines that the pre-scanned image has been hand-held in the previous time (step S35, Yes), the movement amount calculation unit 67 and the hand-shot detection start timing calculation unit 68-1 Using the above-described method, the time T2 until the operator's hand is removed from the previous and current two pre-scan images captured by the image capturing unit 71 of the stand-type image scanner 50 is calculated, and processing is performed during the time T2. (Step S36).

  When the time T2 elapses, the hand-shooting determination unit 65 switches the shooting mode by the image shooting unit 71 to the still image mode when the prescan is in the moving image mode, and the image shooting unit 71 presses in the still image mode. The can is executed (step S37). Then, the processes in steps S38 to S40 are performed based on the prescan image acquired by the prescan in the still image mode. Since the processes in steps S38 to S39 are the same as the processes in steps S32 to S33 described above, detailed description thereof is omitted.

  If the hand-shooting determination unit 65 determines in step S40 that the pre-scanned image has hand-shaking (step S40, Yes), the process returns to step S36. In this way, the processes in steps S36 to S40 are repeated until a pre-scan image without hand-drawing is acquired. In step S40, if the hand-shooting determination unit 65 determines that the pre-scan image has no hand-shooting (No in step S40), the process proceeds to step S43.

  In step S41, the hand movement determination unit 65 determines whether or not the pre-scan is in the still image mode. If the pre-scan is in the still image mode (Yes in step S41), the process proceeds to step S42. (Step S41, No) The process proceeds to step S43.

The process of step S42 is the same as the process of step S16 in the flowchart of FIG. 17, and when the process of step S42 ends, the process proceeds to step S43.
The process of step S43 is the same process as step S16 of the flowchart of FIG. In step S43, the image conversion processing unit (not shown) performs image conversion processing (processing for correcting image magnification, distortion / skew) on the non-hand-printed form image stored in the photographed image storage unit 61. Then, the form image subjected to the image conversion processing is output to the monitor 20 via the USB interface 14 by the output unit 66, and character recognition processing is executed by the OCR processing unit (not shown).

  As described above, in the second method, the time T2 until the hand is not captured in the pre-scan image captured by the stand-type image scanner 50 is predicted based on the two pre-scan images obtained by pre-scan. If a pre-scan image without hand-drawing is obtained after the predicted time T2 has elapsed (at time Ti2), image conversion processing is immediately performed on the pre-scan image, and then post-processing is started. To do. Thereby, post-processing can be executed earlier than the first method (see FIG. 5).

  In the flowchart of FIG. 18, when the pre-scan is started in the moving image mode, the process of step S37 may be omitted. Such an algorithm is effective when there is a high possibility that the predicted time T2 will deviate because the iterative processing of step S36 to step S40 can be executed at high speed. On the other hand, the algorithm for executing the process of step S37 is effective when the possibility that the predicted time T2 will be off is low. This is because the pre-scan image captured in the still image mode is already stored in the captured image storage unit 61 at the time when the process of step S41 is started (the main scan has been completed). This is because the process of step S41 can be executed immediately without taking a picture.

  If the pre-scan is started in the moving image mode and the process in step S37 is not executed, the processes in steps S38 to S40 are executed in the moving image mode. In this case, when it is determined in step S40 that the pre-scan image has no hand-drawing (No in step S40), the main scan is performed next, and the process of step S43 is executed for the form image acquired by the main scan. To do.

  According to the second embodiment of the present invention, the start timing of the main scan or the post-processing is automatically predicted by using two pre-scan images with hand-drawing, and the main scan or the post-processing is predicted. Can be started at the timing.

{Application example}
Next, a case where the first and second embodiments of the image processing apparatus of the present invention are applied to a state in which a form image other than hand-drawn images cannot be acquired correctly will be described with reference to FIGS. .

  19 to 22, FIG. 19 shows the first and second implementations of the image processing apparatus of the present invention when an object placed on the form protrudes from the form while covering the outline of the form. It is a figure explaining the case to which a form is applied. In FIG. 19A, a broken-line rectangular frame indicates the shooting range 610 of the CCD camera 51 of the stand type image scanner 50. The same applies to the broken-line rectangular frame in FIGS. 20 (a), 21 (a), and 22 (a). Accordingly, the images in the photographing range 610 shown in FIGS. 19A, 20A, 21A, and 22A are images taken by the CCD camera 51. FIG.

  In the case of FIG. 19A, the object 810 is erroneously placed in the lower left corner of the form 710 that is correctly set on the table 52 within the imaging range 610. The image shooting unit 71 cuts out an image in the shooting range 610 shot by the CCD camera 51 shown in FIG. 19A in a cut-out range 910 shown in FIG. 19B, and takes the cut-out image in block image units. Transfer to the image storage unit 61. The contour extraction processing unit 63 extracts a contour image 1010 shown in FIG. 19B from the captured image within the cutout range 910. The contour line 1011 of the contour image 1010 is composed of seven straight lines. Accordingly, since the number of straight lines detected from the contour 1011 by the circularity and straight line number calculation unit 64 exceeds the standard straight line number (= 4), the hand-drawing determination unit 65 is erroneously placed on the form on the table 52. 810 can be determined.

  FIG. 20 is a diagram illustrating a case where the first and second embodiments of the embodiment of the present invention are applied to a case where a sticky note or the like is stuck in a form correctly set on a table. . Since the sticky note cannot be distinguished from printing, the image of the sticky note in the form image cannot be determined from the photographed image.

  As shown in FIG. 20A, when a sticky note 820 is stuck on the form 710 (in this example, the upper left), the image photographing unit 71 takes a picture taken by the CCD camera 51 shown in FIG. The captured image in the range 610 is extracted in the extraction range 920, and the extracted image is transferred to the captured image storage unit 61 in block image units. As shown in FIG. 20B, the contour line 1021 of the contour image 1020 extracted from the clipped image stored in the captured image storage unit 61 by the contour extraction processing unit 63 is the same as the contour line of the form image. . For this reason, the number of straight lines detected from the contour 1021 by the circularity and straight line number calculation unit 64 is four, which is equal to the standard straight line number. Therefore, the handwriting determination unit 65 has a sticky note 820 pasted on the form 710. I can't judge.

FIG. 21 is a diagram for explaining a case in which the first and second embodiments of the present invention are applied when a form protrudes from the shooting range.
As shown in FIG. 21A, when the form 710 protrudes from the shooting range 610 (in this example, protrudes to the right side), a part of the form 710 outside the shooting range 610 is not shot by the CCD camera 51. . The image shooting unit 71 cuts out an image in the cutout range 930 shown in FIG. 21B from the shot image in the shooting range 610 shot by the CCD camera 51 shown in FIG. 21A, and blocks the cutout image. The image data is transferred to the captured image storage unit 61 in units of images. The contour extraction processing unit 63 extracts a contour image 1030 from the image within the cutout range illustrated in FIG. In this case, since the protruding portion of the form covers the table 52, the outline image of the form cannot be extracted by a normal method of detecting a portion where the black pixel changes to the white pixel. Therefore, in such a case, the contour extraction processing unit 63 regards a portion corresponding to the outer periphery of the photographing range 610 of the photographed image as the contour of the form image, and extracts the contour image 1030 shown in FIG. . The contour line 1031 of the contour image 1030 is rectangular. Therefore, since the circularity and the number of straight lines detected by the straight line number calculation unit 64 from the contour line 1013 are the same as the standard straight line number, the hand-shake determination unit 65 determines that the form 710 is placed only when the straight line number detected from the contour line 1013 is used. 52, it cannot be determined that it is protruding from 52. However, since the perimeter of the contour image 1030 (the number of pixels of the contour line) is shorter than the perimeter of the original form image, the circularity is smaller than the standard circularity. For this reason, the hand-drawing determination unit 65 compares the circularity of the contour image 1030 calculated by the circularity and straight line number calculation unit 64 with the standard circularity, so that the form 710 is not correctly set on the table 52. Can be determined. On the other hand, the method disclosed in the above-mentioned known document 3 has a black frame (in this embodiment) in the composite image because the form image protrudes from the form installation table (corresponding to the table 52 in this embodiment). Since a part of the area (equivalent to a black background image) does not exist, it is determined that there is a hand-shot (see paragraph number [0096]). It is not possible to distinguish between “hand-drawing” and “protruding from the table”.

FIG. 22 is a diagram for explaining a case where the first and second embodiments of the present invention are applied when a foreign object is placed outside the form in the imaging range.
As shown in FIG. 22A, when the form 710 is correctly set on the table 52 in the photographing range 610 and the object 810 is mistakenly placed outside the form 710 in the photographing range 610, the image The imaging unit 71 cuts out an image in the cutout range 940 shown in FIG. 22B from the shot image in the shooting range 610 shot by the CCD camera 51 shown in FIG. Then, the clipped image is transferred to the captured image storage unit 61 in block image units. The contour extraction processing unit 63 extracts a contour image 1040 shown in FIG. 22B from the cutout image. The contour image 1040 is a captured image of the form 710 itself. Therefore, the contour line 1041 extracted from the contour image 1040 by the circularity and straight line number calculation unit 64 is the same as the contour line of the captured image of the form 710. As a result, when the object 810 is mistakenly placed in the shooting range 610 outside the form 710, the hand-drawing determination unit 65 cannot determine the object 810.

[Third Embodiment]
In the third embodiment of the present invention, for each operator, the movement speed of the operator's hand after the form is set on the table 52 when the stand-type image scanner 50 is used is recorded each time. The start timing of the main scan (when the pre-scan is in the moving image mode) or post-processing (when the pre-scan is in the still image mode) for each operator is learned.

  FIG. 23 is a diagram showing a functional block configuration according to the third embodiment of the present invention having the hardware configuration shown in FIG. In the image processing apparatus 10-3 illustrated in FIG. 23, the same components as those included in the image processing apparatus 10-2 illustrated in FIG. 13 are denoted by the same reference numerals, and detailed descriptions of these components are provided. Is omitted.

  The image processing apparatus 10-3 includes a moving speed calculation unit 69, a moving speed table storage unit 70, and a logon control unit 71 in addition to the components 61 to 66 and 68 included in the image processing apparatus 10-2. Further, the hand-shake detection start timing calculation unit 68-2 has another function in addition to the function of the component having the same name (the hand-shot detection start timing calculation unit 68-1) shown in FIG.

  The moving speed table storage unit 70 is provided in, for example, the HDD 13 and stores a moving speed table used for learning the moving speed of an operator who reads a form image using the stand-type image scanner 50. It has. Here, the moving speed is the moving speed of the operator's hand after the form is set on the table 52 when the operator performs an image reading operation of the form using the stand-type image scanner 50. Each time an operator performs a form image reading operation using the stand-type image scanner 50, the moving speed of the operator is registered in the moving speed table. The learning of the operator's moving speed ends when the number of times the moving speed is registered in the moving speed table reaches a certain number. Thereafter, the average value of the moving speed (average moving speed) is the operator's moving speed. Used as The moving speed table stores the moving speed and the average moving speed in association with an operator ID (operator identifier) uniquely assigned to each operator.

  The movement speed calculation unit 69 calculates the movement speed of the operator each time the operator performs an operation for reading a form image using the stand-type image scanner 50, and the calculated movement speed can be directly or logon controlled. 70 is registered in the moving speed table.

Here, a calculation formula for the movement speed calculation unit 69 to obtain the movement speed will be described with reference to the example of FIG.
As mentioned above,
T2 = Tec × L2 / Lec (4)
Tec = Te-Tc (3)
Lec = L2-L1 (2)
So
If the moving speed is Vmov,
Vmov = Lec / Tec (6)
It becomes.

Therefore, from Equation (4) and Equation (6),
T2 = L2 / Vmov (7)
Is obtained.

Further, the calculation formula of the time T2 of the operator who has finished learning the moving speed is the following formula (8).
T2 = L / Vmov-av (8)
L: Length of hand image in pre-scan image (captured image) or contour image (corresponding to L1 and L2 in FIGS. 16C and 16E)
Vmov-av: Average moving speed

  When the learning of the moving speed is completed, the moving speed calculating unit 69 obtains the average moving speed, and registers the average moving speed in the moving speed table directly or via the logon control unit 70. As will be described later, the logon control unit 71 may obtain the average moving speed of the operator who has finished learning the moving speed. The movement amount calculation unit 67 stops the calculation of the movement speed for the operator who has finished learning the movement speed. The moving speed calculation unit 69 calculates the L2 of the equation (7) from the pre-scan image with hand-shooting stored in the captured image storage unit 61 or the contour image of the pre-scan image stored in the contour image storage unit 62. And outputs this L2 to the hand-held detection start timing calculator 68-2. The movement speed calculation unit 69 calculates Tec, Lec, L1, and L2 for the operator whose movement speed learning has not been completed in the same manner as the movement amount calculation unit 67, and moves from the expressions (2) and (6). Find the speed Vmov. Then, L2 and the moving speed Vmov are output to the handwriting detection start timing calculation unit 68-2. The movement amount calculation unit 67-3 outputs only L2 to the handwriting detection start timing calculation unit 68-2 for the operator whose learning of the movement speed is completed.

  The hand-shooting detection start timing calculation unit 68-2 calculates the time T2 from the equation (7) using L2 and the movement speed Vmov input from the movement speed calculation unit 69 for an operator who has not learned the movement speed. To do. For the operator who has finished learning the moving speed, the above equation is used by using L2 input from the moving speed calculating unit 69 and the average moving speed Vmov-av read from the moving speed table directly or via the logon control unit 70. The time T2 is calculated by (8). The handwriting detection start timing calculation unit 68-2 notifies the contour extraction processing unit 63 of the time T2 calculated as described above. The contour extraction processing unit 63 waits for the process for the time T2 notified from the hand-shake detection start timing calculation unit 68-2. The hand-shake detection start timing calculation unit 68-2 can also read the movement speed of the operator whose learning of the movement speed has not been completed from the movement speed table directly or via the logon control unit 71.

  The logon control unit 71 displays a logon input screen, which is an authentication screen for using the image processing apparatus 10, on the monitor 20, and performs user authentication based on the user ID and password of the operator input on the logon input screen. When the operator succeeds in user authentication, the operator can perform an image reading operation of the form using the stand-type image scanner 50. When the user authentication is successful, the logon control unit 71 writes the operator's user ID as the “operator ID” in the movement speed table storage unit 70, and writes the operator ID to the movement speed calculation unit 69 and the hand-shot detection start timing calculation unit 68. -2. The moving speed calculation unit 69 and the hand-shake detection start timing calculation unit 68-2 are notified of the operator ID from the logon control unit 71, so that the operator ID of the operator currently using the image processing apparatus 10-3 is obtained. And can access the moving speed table in association with the operator ID. The logon control unit 71 registers the operator ID in the moving speed table, for example, only when each operator logs on for the first time (first form of moving speed table access). In the case of this first mode, the moving speed calculation unit 69 can manage the number of times of registration of the moving speed in the moving speed table for each operator, and therefore can determine the end of learning of the moving speed of each operator. For the operator who has finished learning the moving speed, the average moving speed is obtained, and the average moving speed is registered in the moving speed table.

  Further, instead of the first embodiment, the logon control unit 71 may manage the movement speed table in an integrated manner. In this embodiment, the logon control unit 71 writes the movement speed and average movement speed of each operator in response to a request from the movement speed calculation unit 69. In response to the request from the hand-shooting detection start timing calculation unit 68-2, the logon control unit 71 moves the moving speed Vmov or the average moving speed Vmov of the operator ID of the operator currently using the stand-type image scanner 50. -av is notified to the handwriting detection start timing calculation unit 68-2 (second form of moving speed table access). In this second mode, the logon control unit 71 can manage the number of times each operator has registered the movement speed in the movement speed table, so that the logon control unit 71 determines the average movement speed for the operators whose movement speed has been learned. And the average moving speed is registered in the moving speed table. Further, the operator ID of the operator who has finished learning the moving speed is notified to the moving speed calculating unit 69 and the hand-shot detection start timing calculating unit 68-2. Thereby, the moving speed calculation unit 69 and the hand-shot detection start timing calculation unit 68-2 can know the operator ID of the operator who has finished learning the moving speed.

  The hand-shooting detection start timing calculation unit 68-2 calculates the time T2 using Equation (8) for the operator who has finished learning the moving speed. Therefore, the moving speed calculation unit 69 only needs to obtain L for an operator who has finished learning the moving speed. In other words, after learning of the moving speed, the moving speed calculating unit 69 may obtain L from a pre-scan image (captured image) with one hand-drawn image or its contour image. Thereby, the hand-shake detection start timing calculation unit 68-2 needs to wait for the hand-shake determination unit 65 to determine hand-shooting twice when calculating the time T2 for the operator who has finished learning the moving speed. Will disappear. In other words, even if a pre-scan image having two hand-shots is not used, only a pre-scan image having one hand-shot (for example, a pre-scan image having a hand-shot detected first or its outline image). Can be used to calculate the time T2. Therefore, for the operator who has finished learning the moving speed, the calculation timing of the time T2 can be made earlier than before the learning of the moving speed ends.

As described above, after the learning of the moving speed is completed, it is not necessary to calculate the moving speed by the moving speed calculating unit 69, and the hand-shake detection start timing calculating unit 68-2 is stored in each moving speed table. By reading out the average moving speed of the operators, it is possible to calculate the time T2 of each operator. Thereby, the calculation time of time T2 is shortened after completion of learning. In addition, when the hand-shooting determination unit 65 detects the “pre-scan image with the first hand-shot”, it is possible to immediately calculate the time T2 using the pre-scan image.
Next, a configuration example of the moving speed table will be described.

FIG. 24 is a diagram showing a configuration example of a moving speed table provided in the moving speed table storage unit 70 shown in FIG.
The moving speed table 1100 shown in FIG. 24 has a function of storing and managing the average amount of moving speed of each operator using an ID (operator ID) unique to each operator. Note that Tc, Te, and Lec described in the moving speed table 1100 of FIG. 24 are the same parameters as the parameters of the same notation shown in FIG. 16, and are parameters required for the calculation formula (4) of the time T2. It is. Here, the moving speed stored in the moving speed table 1100 is equal to the moving amount (unit: mm) per unit time (1 s (1 second) in this example), and the unit is mm / s.

  The moving speed table 1100 is a table using an operator ID as a key, and is calculated for each operator when the stand-type image scanner 50 performs an operation for reading a form image and after learning of the moving speed is completed. Store the average moving speed. Each entry in the moving speed table 1100 includes a field for storing an operator ID and a field for storing the moving speed of an operator having the operator ID. The field for storing the moving speed includes a field for storing the moving speed when the operator operates the stand-type image scanner 50 each time and the average moving speed obtained after the learning of the moving speed is completed.

  The moving speed table 1100 shown in FIG. 24 stores the moving speed and average moving speed of each of four operators whose operator IDs are “0001”, “0002”, “0003”, and “0004”, respectively. ing. The movement speed table 1100 shows that the first, second, and third movement speeds of the operator with the operator ID “0001” are 240 mm / s, 220 mm / s, and 230 mm / s, respectively. Information that the speed is 230 mm / s is stored. Similarly, for each of the operators with operator IDs “0002”, “0003”, and “0004”, information on the moving speed and the average moving speed for each time is stored.

  An embodiment in which the circularity change amount and the average circularity change amount are stored in the table instead of the movement speed and the average movement speed is also possible. The image processing apparatus according to the embodiment has a configuration in which the moving speed table storage unit 70 in FIG. 23 is replaced with a circularity storage table storage unit, and the moving speed calculation unit 69 is replaced with a circularity calculation unit, and the circularity storage table storage is performed. This can be realized by providing a circularity storage table in the unit.

  In the method of learning the moving speed, it is necessary to obtain the length L of the hand image every time after learning. However, in the method of learning the amount of change in circularity, since C2 is a constant value, the average after learning is averaged. There is an advantage that the time T2 can be calculated simply by reading the amount of change in circularity from the table.

In addition, the method of obtaining the average moving speed and the amount of change in average circularity is not limited to the method of obtaining the average of all past values, and various forms such as a method of obtaining the most recent predetermined number of average values are conceivable.
According to the third embodiment of the present invention, by learning the average moving speed or circularity change amount of each operator, after the completion of learning, the main scan start timing of each operator is determined. It can be automatically obtained at a higher speed than the second form.

  When obtaining the time T2 from the pre-scan image (or contour image) taken at regular time intervals, the Tec may be obtained from the time stamp only by the method described above. In this case, in the subsequent calculation of time T2, Tec obtained first is used.

  Software (program) to which the image processing method of the present invention is applied may be stored in a portable storage medium and distributed, or may be downloaded from a network and distributed. Furthermore, software may be provided by a cloud service such as Saas (Software as a Service).

  As mentioned above, although this invention was demonstrated based on embodiment, this invention is not limited to embodiment mentioned above. The protection scope of the present invention covers the invention described in the claims and equivalents thereof.

10 (10-1 to 10-3) Image processing apparatus 11 CPU
12 RAM
13 HDD
14 USB interface 20 Monitor 30 Keyboard 40 Mouse 50 Stand-type image scanner 51 CCD camera 52 Table 53 Form collation guide 54 Post 61 Photographed image storage unit 62 Contour image storage unit 63 Contour extraction processing unit 64 Circularity, straight line number calculation unit 65 Manual copy Judgment unit 66 output unit 67 movement amount calculation unit 68 (68-1, 68-2) hand-shooting detection start timing calculation unit 69 movement speed calculation unit 70 movement speed table storage unit 71 logon control unit 1100 movement speed table

Claims (21)

Computer
Contour image extraction means for extracting the contour of the subject portion from the photographed image of the subject and creating a contour image including the subject portion;
Based on the contour image created by the contour image extraction unit, a feature amount calculation unit that calculates a feature amount necessary to determine whether or not an image of an object other than the subject is included in the contour image When,
Determination means for determining whether or not an image of an object other than the subject is included in the contour image based on the feature quantity obtained by the feature quantity calculation means;
Image processing program to function as The feature amount is a circularity calculated using the area of the contour image obtained when the subject is normally photographed and the perimeter of the contour image obtained by actual photographing,
The determination unit determines whether or not an image of an object other than the subject is included in the contour image by comparing the circularity calculated by the feature amount calculation unit with a predetermined standard circularity.
The image processing program according to claim 1. The feature amount is the total number of straight lines constituting the contour of the contour image and straight lines detected from the contour image,
The determination unit determines whether or not an image of an object other than the subject is included in the contour image by comparing the number of straight lines detected by the feature amount calculation unit with a predetermined standard number of straight lines.
The image processing program according to claim 1 or 2, wherein The feature amount is the number of sides constituting the contour of the subject included in the contour of the contour image obtained by photographing,
The determination unit determines whether or not an image of an object other than the subject is included in the contour image by comparing the number of sides detected by the feature amount calculation unit with a predetermined standard number of sides. ,
The image processing program according to any one of claims 1 to 3. Computer
Pre-scan instruction means for instructing the image photographing means to pre-scan the subject before the photographing instruction by the operator is made;
Contour image extraction means for extracting the contour of the subject portion from the captured image acquired by the pre-scan and creating a contour image including the subject portion;
Feature amount calculating means for calculating a feature amount necessary for determining whether or not an image of an object other than the subject is included in the contour image created by the contour image extracting means;
Determination means for determining whether an image of an object other than the subject is included in the contour image based on the feature quantity obtained by the feature quantity calculation means;
When it is determined by the determination means that the contour image includes an image of an object other than the subject, a captured image including at least two objects other than the subject obtained by the pre-scan or those 2 Calculation means for calculating the time until no object other than the subject is captured in the captured image captured by the image capturing means, based on the contour images of the two captured images;
When the pre-scan shooting by the image shooting unit is shooting at a resolution lower than the originally required resolution, the image shooting unit has an appropriate timing according to the time calculated by the calculation unit. Re-scanning instruction means for instructing the object to be scanned again at the originally required resolution,
Image processing program to function as   The calculation means obtains a moving amount of an object between the captured images including objects other than the two subjects photographed at different times or between the contour images of the two captured images, and based on the moving amount, the time is calculated. The image processing program according to claim 5, wherein the image processing program is calculated. The calculation means stores the data related to the movement amount for each operator in the movement amount data storage means every time the operator gives a photographing instruction, and the number of times the data related to the movement amount is stored in the movement amount data storage means is a fixed number of times. For the reached operator, the average value of the data related to the movement amount of the operator so far is calculated, and the average value obtained by the calculation is stored in the movement amount information storage means. When calculating the time until no object other than the subject is captured in the captured image to be captured, using the average value of the operator stored in the movement amount data storage means;
The image processing program according to claim 6.   The calculation means calculates the area of the contour image obtained when the subject is normally photographed and the contour image obtained by the photographing for each photographed image including objects other than the two subjects photographed at different times. The image photographing unit obtains a circularity calculated using a perimeter, and based on a circularity change amount that is a change amount of the circularity between contour images of a captured image including an object other than the two subjects. The image processing program according to any one of claims 5 to 7, further comprising: calculating a time until no object other than the subject is captured in a photographed image taken by. The calculation means stores the circularity change amount for each operator in the circularity change amount storage means every time the operator gives a photographing instruction, and the number of times the circularity change amount is stored in the circularity change amount storage means is constant. For the operator who has reached the number of times, the average value of the circularity change amount of the operator so far is calculated, and the average circularity change amount obtained by the calculation is stored in the circularity change amount storage means. When calculating the time until no object other than the subject appears in the captured image captured by the imaging means for the operator, the average circularity change amount of the operator stored in the circularity change amount storage means is used. Calculating
The image processing program according to claim 8. Contour image extraction means for extracting the contour of the subject portion from the photographed image of the subject and creating a contour image including the subject portion;
Based on the contour image created by the contour image extraction unit, a feature amount calculation unit that calculates a feature amount necessary to determine whether or not an image of an object other than the subject is included in the contour image When,
Determination means for determining whether an image of an object other than the subject is included in the contour image based on the feature quantity obtained by the feature quantity calculation means;
An image processing apparatus comprising: Pre-scan instruction means for instructing the image photographing means to pre-scan the subject before the photographing instruction by the operator is made;
Contour image extraction means for extracting the contour of the subject portion from the captured image acquired by the pre-scan and creating a contour image including the subject portion;
Feature amount calculating means for calculating a feature amount necessary for determining whether or not an image of an object other than the subject is included in the contour image created by the contour image extracting means;
Determination means for determining whether an image of an object other than the subject is included in the contour image based on the feature quantity obtained by the feature quantity calculation means;
When it is determined by the determining means that the contour image includes an image of an object other than the subject, the image capturing means captures based on at least two captured images obtained by the prescan. Calculating means for calculating the time until no object other than the subject is captured in the captured image;
When the pre-scan shooting by the image shooting unit is shooting at a resolution lower than the originally required resolution, the image shooting unit has an appropriate timing according to the time calculated by the calculation unit. Rescan instructing means for instructing the subject to scan again at the resolution originally required;
An image processing apparatus comprising:   The calculation means obtains a moving amount of an object between the captured images including objects other than the two subjects photographed at different times or between the contour images of the two captured images, and based on the moving amount, the time is calculated. The image processing apparatus according to claim 11, wherein: The calculation means stores the data related to the movement amount for each operator in the movement amount data storage means every time the operator gives a photographing instruction, and the number of times the data related to the movement amount is stored in the movement amount data storage means is a fixed number of times. For the reached operator, the average value of the data related to the movement amount of the operator so far is calculated, and the average value obtained by the calculation is stored in the movement amount information storage means. When calculating the time until no object other than the subject is captured in the captured image to be captured, using the average value of the operator stored in the movement amount data storage means;
The image processing apparatus according to claim 12.   The calculation means calculates the area of the contour image obtained when the subject is normally photographed and the contour image obtained by the photographing for each photographed image including objects other than the two subjects photographed at different times. The image photographing unit obtains a circularity calculated using a perimeter, and based on a circularity change amount that is a change amount of the circularity between contour images of a captured image including an object other than the two subjects. The image processing apparatus according to claim 11, further comprising: calculating a time until no object other than the subject appears in the captured image taken by the camera. The calculation means stores the circularity change amount for each operator in the circularity change amount storage means every time the operator gives a photographing instruction, and the number of times the circularity change amount is stored in the circularity change amount storage means is constant. For the operator who has reached the number of times, the average value of the circularity change amount of the operator so far is calculated, and the average circularity change amount obtained by the calculation is stored in the circularity change amount storage means. When calculating the time until no object other than the subject appears in the captured image captured by the imaging means for the operator, the average circularity change amount of the operator stored in the circularity change amount storage means is used. Calculating
The image processing apparatus according to claim 14. A contour image extraction step of extracting a contour of a subject portion from a photographed image of the subject and creating a contour image including the subject portion;
Based on the contour image created by the contour image extraction step, a feature amount calculation step for calculating a feature amount necessary for determining whether or not an image of an object other than the subject is included in the contour image When,
A determination step of determining whether or not an image of an object other than the subject is included in the contour image based on the feature amount obtained by the feature amount calculation step;
An image processing method comprising: A pre-scan instruction step for instructing the image photographing means to pre-scan the subject before the photographing instruction by the operator is made;
A contour image extraction step of extracting a contour of a subject portion from the captured image acquired by the pre-scan, and creating a contour image including the subject portion;
A feature amount calculation step for calculating a feature amount necessary for determining whether or not an image of an object other than the subject is included in the contour image created by the contour image extraction step;
A determination step of determining whether or not an image of an object other than the subject is included in the contour image based on the feature amount obtained by the feature amount calculation step;
When it is determined in the determination step that the contour image includes an image of an object other than the subject, the image capturing unit captures the image based on at least two captured images obtained by the prescan. A calculation step for calculating the time until no object other than the subject is captured in the captured image
When the pre-scan shooting by the image shooting unit is shooting at a resolution lower than the originally required resolution, the image shooting unit has an appropriate timing according to the time calculated by the calculation step. A re-scanning instruction step for instructing the object to be scanned again at the originally required resolution;
An image processing method comprising:   The calculation step obtains a moving amount of an object between the captured images including the objects other than the two subjects photographed at different times or between the contour images of the two captured images, and based on the moving amount, the time The image processing method according to claim 17, wherein: is calculated. In the calculating step, the data relating to the movement amount for each operator is stored in the movement amount data storage means every time the operator gives a photographing instruction, and the number of times the data relating to the movement amount is stored in the movement amount data storage means is a predetermined number of times. For the reached operator, the average value of the data related to the movement amount of the operator so far is calculated, and the average value obtained by the calculation is stored in the movement amount information storage means. When calculating the time until no object other than the subject is captured in the captured image to be captured, using the average value of the operator stored in the movement amount data storage means;
The image processing method according to claim 18.   The calculation step includes an area of a contour image obtained when the subject is normally photographed and a contour image obtained by the photographing for each photographed image including objects other than the two subjects photographed at different times. The image photographing unit obtains a circularity calculated using a perimeter, and based on a circularity change amount that is a change amount of the circularity between contour images of a captured image including an object other than the two subjects. 20. The image processing method according to claim 17, further comprising: calculating a time until no object other than the subject is captured in a captured image taken by the camera. In the calculating step, the circularity change amount for each operator is stored in the circularity change amount storage unit every time the operator gives a photographing instruction, and the number of times the circularity change amount is stored in the circularity change amount storage unit is calculated. For an operator that has reached a certain number of times, the average value of the circularity change amount of the operator so far is calculated, and the average circularity change amount obtained by the calculation is stored in the circularity change amount storage means. When calculating the time until an object other than the subject is not captured in the captured image captured by the imaging means for the operator, the average circularity change amount of the operator stored in the circularity change amount storage means is calculated. Using to calculate,
The image processing method according to claim 20.