JP2013192101A - Image processor and original reading system equipped with the same - Google Patents

Abstract

PROBLEM TO BE SOLVED: To easily and securely complement a missing part of an outline even if a part of the outline cannot be acquired by existence of a finger which covers a part of the outline during photographing of an image of an original which is read by a document camera and the like.SOLUTION: An image processor (PC) 3 includes: an edge extraction section 32 which extracts an edge pixel in a photographed image; a corner point/throat point detection section 33 which detects right and left corner points on at least one side in a vertical direction of an original and detects a throat point on the one side; a segment recognition section 34 for recognizing candidates of a plurality of segments which can constitute an outline at least on one side of the original on the basis of the edge pixel; an invalid segment removing section 35 for removing an invalid segment from the candidates of the segments on the basis of an angle to the segment connecting the corner point and the throat point; and a segment interpolation section 36 for interpolating the segment to the outline constituted of the segments from which the invalid segment is removed.

Description

  The present invention relates to an image processing apparatus that processes an image obtained by reading a document such as a book, and a document reading system including the image processing apparatus.

  2. Description of the Related Art Document cameras (book scanners) that can take a picture of a page and read an image of the page while the page of the book is naturally opened are widely used (see Patent Document 1). When such a document camera is used, since the images of the pages can be read one after another while turning the pages, the work of digitizing the book can be performed efficiently.

  When reading a bound document such as a book with this type of document camera, if the document is left open naturally, the scanned image may be misaligned or distorted due to the page being lifted. It is necessary to eliminate the warp of the page by holding down the document. However, when the user presses the document with fingers or the like, the fingers and the like are also photographed, so that the acquired captured image becomes unsightly.

  Therefore, based on the density distribution of the image corresponding to a predetermined portion of the document (for example, near the left and right edges of the document with a high probability of placing a finger), it is determined whether or not the finger exists in the predetermined portion. In some cases, a technique is known in which the image is erased by replacing the area with white data (see Patent Document 2).

JP 2001-103240 A JP-A-6-78133

  By the way, information on edges (hereinafter referred to as outlines) that define the outline of a bound document is important for recognition of a document area in a captured image and various image processing (for example, planarization processing). However, when the outer edge of the document is covered with the user's finger or the like holding the document as described above, it may be difficult to recognize the outline.

  However, in the conventional technique described in Patent Document 2, since the area is simply replaced with white data when a finger is present, the appearance of the image is improved, but the outline covered by the finger is not necessarily provided. There was a problem that it could not be recognized. Further, in this conventional technique, since the finger is a detection target, there are cases where it is not possible to deal with other objects (such as a tool for holding a book) covering the outline.

  The present invention has been devised in view of such problems of the prior art, and even when a part of the outline cannot be obtained due to the presence of a finger or the like covering a part of the outline at the time of shooting, It is a main object of the present invention to provide an image processing apparatus capable of easily and surely complementing a missing portion of the outline and a document reading system provided with the image processing apparatus.

  An image processing apparatus according to the present invention includes a photographic image acquisition unit that acquires photographic images obtained by photographing left and right paper surfaces of a bound document, an edge extraction unit that extracts a plurality of edge pixels in the photographic image, and upper and lower sides of the document. Based on the edge pixel, a corner point detection unit that detects left and right corner points on at least one side of the direction, a throat point detection unit that detects a throat point that is an end point of a binding center line on at least one side of the document, and A line segment recognition unit for recognizing a plurality of line segment candidates that can form an outline on at least one side of the document, and a line connecting the corner point and the throat point with respect to at least one of the left and right sheets. An invalid line segment eliminating unit that eliminates an invalid line segment from the line segment candidates based on an angle with respect to a segment, and a missing part in an outline formed by the line segment candidate from which the invalid line segment is excluded Characterized in that it comprises a line segment interpolation unit for performing line interpolation on.

  As described above, according to the present invention, even when a part of the outline cannot be acquired due to the presence of a finger or the like that covers a part of the outline at the time of shooting, the missing portion of the outline can be easily and reliably complemented. It has an excellent effect of becoming.

1 is an overall configuration diagram showing a document reading system 1 according to an embodiment. 1 is a block diagram showing a schematic configuration of the document camera 2 and the PC 3 in FIG. FIG. 1 is a flowchart showing a main part of a document reading procedure by the document reading system 1 in FIG. Schematic diagram illustrating an example of a captured image acquired by image input (ST101) in FIG. Flow chart showing details of corner point detection (ST103) in FIG. Explanatory drawing which shows an example of the process of edge search (ST201) in FIG. Explanatory drawing which shows an example of the process of the corner point estimation (ST203) in FIG. Explanatory drawing which shows an example of the process of validity determination (ST204) of the corner point in FIG. Schematic diagram showing an example of the result of line segment recognition (ST105) in FIG. Schematic diagram showing an example of an edge trace processing process in line segment recognition (ST105) in FIG. FIG. 3 is a flowchart showing details of the invalid line segment elimination (ST106) in FIG. Explanatory drawing which shows the outline | summary of the process of ST301 and ST303 in FIG. Explanatory drawing which shows the outline | summary of the process of ST305 in FIG. Schematic diagram showing an example of the result of invalid line segment elimination (ST106) in FIG. Schematic diagram showing an example of the result of line segment interpolation (ST107) in FIG.

  An image processing apparatus according to a first aspect of the present invention, which has been made to solve the above-described problems, includes a captured image acquisition unit that acquires captured images obtained by capturing left and right sheets of a bound document, and a plurality of edge pixels in the captured image. An edge extracting unit for extracting the edge, a corner point detecting unit for detecting left and right corner points on at least one side in the vertical direction of the document, and a throat point that is an end point of the binding center line on at least one side of the document. A throat point detection unit, a line segment recognition unit for recognizing a plurality of line segment candidates that can form an outline on at least one side of the document based on the edge pixels, and at least one of the left and right paper surfaces , An invalid line segment eliminating unit that eliminates invalid line segments from the line segment candidates based on an angle with respect to a line segment connecting the corner point and the throat point, and the line from which the invalid line segment is excluded A configuration and a line segment interpolation unit for performing line interpolation on missing part in the candidate contour line composed of.

  According to this, even if a part of the outline cannot be acquired due to the presence of a finger or the like that covers a part of the outline on at least one side in the vertical direction of the document at the time of shooting, the invalid line recognized from the finger or the like Therefore, it is possible to easily and reliably complement the missing part of the outline.

  In a second aspect based on the first aspect, the line segment recognition unit traces the edge pixel between the corner point and the throat point, thereby determining the plurality of line segment candidates. The configuration is to recognize.

  According to this, it is possible to more appropriately recognize outline line segment candidates, and it is possible to more accurately complement missing parts of the outline line.

  According to a third aspect of the present invention, in the first or second aspect, the invalid line segment exclusion unit is a virtual straight line that passes through the end points of the line segment candidates and the end points of other line segments adjacent thereto. If the shortest distance is greater than or equal to a predetermined threshold, the line segment candidate is excluded as an invalid line segment.

  According to this, it is possible to more effectively eliminate invalid line segments recognized from fingers and the like based on the distance to other line segments, and it is possible to more accurately complement missing portions of the outline. .

  An image processing apparatus according to a fourth aspect of the present invention is a captured image acquisition unit that acquires a captured image obtained by capturing a paper surface of an original, an edge extraction unit that extracts a plurality of edge pixels in the captured image, and left and right sides of the original. A line segment that recognizes a plurality of line segment candidates that can form an outline on at least one side of the document based on the edge pixel and a corner point detection unit that detects upper and lower corner points on at least one side of the direction. A recognition unit, an invalid line segment elimination unit that eliminates invalid line segments from the line segment candidates based on an angle with respect to a line segment that connects the upper and lower corner points, and a plurality of line segments from which the invalid line segment is eliminated And a line segment interpolating unit that performs line segment interpolation on the missing part in the outline line configured as described above.

  According to this, even when a finger or the like that covers a part of the outline on at least one side in the left-right direction of the document exists at the time of shooting, even if a part of the outline cannot be obtained, an invalid line recognized from the finger or the like Therefore, it is possible to easily and reliably complement the missing part of the outline.

  According to a fifth aspect of the present invention, there is provided a document reading system including the image processing apparatus according to any one of the first to fourth aspects, and an image input apparatus including a camera unit that generates the captured image.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is an overall configuration diagram showing a document reading system 1 according to the first embodiment. The document reading system 1 reads a paper image of a book (original) B to acquire image data of the paper surface, and a document camera (image input device) 2 that captures the paper surface and converts it into a video signal. The document camera 2 and the PC 3 connected to be communicable with each other.

  The document camera 2 includes a camera unit 4 having a photographing function and a stand unit 5 that holds the camera unit 4. The camera unit 4 includes an image sensor made up of a CCD, a CMOS, etc., and an illumination light source (both not shown) made up of an LED, a fluorescent lamp, or the like. The stand portion 5 includes a substantially V-shaped leg 7 placed on a placement surface 6 such as a desk top surface, and an arm 8 that is supported by the leg 7 and that can be extended and contracted obliquely upward. The arm 8 holds the camera unit 4 so as to be rotatable by a hinge unit 8a, whereby the optical axis direction (shooting direction) of the camera unit 4 can be adjusted.

  The PC 3 functions as an input / output device for the user to set various operating conditions of the document camera 2 and for the user to check a captured image captured by the document camera 2, and to perform image processing and recording of the captured image. It also functions as an image processing apparatus that performs the above. The apparatus used together with the document camera 2 in the document reading system 1 is not limited to a PC (Personal Computer), and any information processing apparatus capable of realizing the same function can be used. It is also possible to add at least a part of the functions of the PC 3 to the document camera 2 or to integrally configure the PC 3 and the document camera 2.

  Further, the document camera 2 and the PC 3 do not need to be directly connected, and may be connected via a network (not shown), for example. In this case, the document camera 2 generates a trigger signal for imaging (specifically, a switch or button for instructing image capture, for example), and performs push-type data transmission to a remote PC 3 (for example, a server). Will do.

  In photographing by the document camera 2, the user places the book B in an open state on the placement surface 6 just below the camera unit 4 (in the optical axis direction), so that two pages of the book B and the periphery of the book B are placed. A captured image (moving image or still image) including a part of the mounting surface 6 is obtained. The captured image is appropriately transmitted to the PC 3, where necessary image processing is performed, and then stored in a predetermined recording medium and displayed to the user. The document read by the document reading system 1 is not limited to a book, but may be any information transmission medium including characters, drawings, photographs, and other similar information. Further, the information transmission medium may be a transmission original such as a positive film or a negative film. In this case, for example, light is emitted from the back side by a light source unit in which a light source is arranged on the side of a transparent light guide plate, and transmitted light is transmitted. May be configured to take an image.

  FIG. 2 is a block diagram showing a schematic configuration of the document camera 2 and the PC 3 in FIG.

  The document camera 2 includes an imaging processing unit 11 having a camera unit 4, an operation instruction unit 12 that causes the imaging processing unit 11 to perform a required operation based on an operation condition set by a user, and a PC 3. And an external interface 13 conforming to the USB standard or the like.

  The PC 3 includes an external interface 21 compliant with the USB standard for connection with the document camera 2, an image data input unit (captured image acquisition unit) 22 to which photographed image data from the document camera 2 is input, and photographing. An image processing unit 23 that performs image processing necessary for recording and displaying an image, an operation system control unit 25 that transmits operating conditions set by a user in an input operation unit 24 such as a keyboard to the document camera 2, and an image A display data generation unit 27 that generates data for displaying the processed captured image on a display unit 26 such as an LCD or a projector, and a data storage unit 28 that stores the captured image data after the image processing; Yes. The image processing unit 23 includes an outline detection unit 31 that detects the outline of the book B during image processing of captured image data. The captured image data processed by the outer shape detection unit 31 is read from the image data storage unit 37, and information on various parameters used for processing in the outer shape detection unit 31 described later is read from the parameter storage unit 38.

  The processing functions of the image data input unit 22, the image processing unit 23, the operation system control unit 25, the display data generation unit 27, and the like in the PC 3 can be realized by software processing in which a program such as an image processing application is executed by the CPU. Or it is good also as a structure provided with the hardware which interprets PC3 as an image processing apparatus and performs a specific process at high speed. The image data storage unit 37 and the parameter storage unit 38 are composed of general-purpose memories.

  In the PC 3, the image data input unit 22 stores the captured image data transmitted from the document camera 2 in the image data storage unit 37 of the image processing unit 23, and outputs the captured image data to the outer shape detection unit 31 as necessary. To do. The outline data detected by the outline detector 31 is information necessary for well-known image processing (for example, planarization) executed by the image processor 23. The captured image data that has undergone the predetermined image processing is stored in the data storage unit 28 and is also sent to the display data generation unit 27 and displayed on the display unit 26.

  The outer shape detection unit 31 extracts four edge pixels from a captured image and obtains a binarized edge image, and the four corners of the book B that has been bound (the back portion is bound). A corner point / throat point detection unit 33 for detecting upper and lower end points (hereinafter referred to as throat points) of the throat portion that forms the boundary line (binding center line) between the left and right pages of the point B and the book B, and the outline of the book B A line segment recognition unit 34 that recognizes a plurality of line segments (straight line components, etc.) connecting edge pixels in the edge image as candidates of a plurality of line segments that can constitute the image, and invalid from the recognized line segment candidates An invalid line segment eliminating unit 35 that eliminates a line segment, and a line segment interpolating unit 36 that performs line segment interpolation on a missing portion in an outline formed by candidate line segments from which an invalid line segment is excluded. ing.

  In the PC 3, the user can appropriately input operating conditions such as the resolution of the image captured by the document camera 2, the frame rate, the shutter speed, and the amount of light emitted from the illumination light source by operating the input operation unit 24. it can. This operating condition is transmitted as a control signal from the operation system control unit 25 to the document camera 2, and the document camera 2 captures the imaging processing unit 11 in accordance with a processing command sent from the operation instruction unit 12 based on the control signal from the PC 3. Performs the shooting operation.

  FIG. 3 is a flowchart showing a main part of a document reading procedure by the document reading system 1 in FIG. 1, and FIG. 4 is a schematic diagram showing an example of a photographed image acquired by image input (ST101) in FIG. .

  First, the user sets the book B in an opened state at the reading position (capturing position of the document camera 2), activates the document camera 2, and activates a required application on the PC 3. Thereby, image input, that is, imaging with the document camera 2 is started, and the captured image data is sequentially transmitted to the PC 3 (ST101).

  Here, the photographed image is defined by a rectangular photographing area 40 of the document camera 2 as shown in FIGS. 4A and 4B, for example, and the book B image and its background are included in the photographing area 40. And an image of the mounting surface 6. The left and right paper surfaces (spreads) 41L and 41R partitioned by the central throat portion 42 are not illustrated here, but a text area in which characters and symbols are displayed, a graphic frame in which drawings and photographs are displayed, and the like Is included.

  When scanning a bookbinding document such as book B, as shown in FIG. 4A, if the book B is left open naturally, the scanned image may be misaligned or distorted. Can occur. Therefore, as shown in FIG. 4B, the user may press the outer peripheral edge portions of the paper surfaces 41L and 41R with the fingers F1 and F2 to eliminate the page warpage. Below, the process of PC3 at the time of acquiring the captured image containing fingers F1 and F2 as shown to FIG. 4 (B) is demonstrated. Here, the fingers are shown as means for preventing the page from rising by pressing the book B. However, the present invention is not limited to this, and a holding jig or the like provided in the document camera 2 may be used.

  Referring again to FIG. 3, in PC 3, after the captured image data received from document camera 2 is stored in image data storage unit 37 by image data input unit 22, edge extraction unit 32 causes the brightness of the captured image to rapidly increase. An edge image is acquired by extracting a changing part as a binarized edge pixel (ST102). This edge extraction process can be performed using a known method such as the Canny method.

  Next, as will be described in detail later, the corner point / throat point detection unit 33 detects the four corner points of the book B having a substantially rectangular shape (ST103). The corner points detected here are usually the four corners of the cover that is slightly larger in size than the paper surfaces 41L and 41R.

  Further, the corner point / throat point detection unit 33 detects the two throat points of book B based on the detected corner point information (ST104). In throat detection, for example, by detecting multiple vertical lines in the edge image using Hough transform etc., compare the coordinates of the middle position of the upper left and right corner points with the coordinates of the middle position of the lower left and right corner points. A straight line detected from the throat portion can be selected, and the positions of the upper and lower ends of this straight line (intersections with the upper and lower sides of the outline) can be set as throat points (see symbols D1 and D2 in FIG. 8). . In this case, by preparing a plurality of image patterns assuming the shape (connection relationship) between the throat point candidate and the peripheral edge pixels in advance, and comparing the straight line and the peripheral pixels with those pixel patterns, Point detection is easy.

  The detection of corner points and throat points in ST103 and ST104 is not limited to the method described above, and can be performed using other known methods.

  Next, as will be described in detail later, the line segment recognition unit 34 performs line segment recognition (detection of straight line components and the like) from the detected plurality of edge pixels (ST105). In this line segment recognition process, a predetermined start point and end point are set, and by performing edge trace processing on the outer peripheral area of book B, a plurality of line segment candidates that can form the outline of book B are recognized. To do. Data such as the coordinate values of the end points related to the respective line segments recognized here are stored in the parameter storage unit 38.

  Thereafter, as described in detail later, the invalid line segment removing unit 35 extracts an invalid line segment inappropriate for constructing the outline from the plurality of line segments recognized in ST105, and the extracted invalid line unit is extracted. It is excluded from the line segment candidates (ST106). As a result, effective line segments appropriate for forming the outline are selected from the line segment candidates recognized by the line segment recognition unit 34.

  Finally, the line segment interpolation unit 36 has a missing part (area where the end points of the line segments are not connected) in the outline formed by the line segment candidates from which the invalid line segment is excluded. Then, line segment interpolation is performed on this missing portion (ST107). In this line interpolation process, the outline can be complemented by connecting the end points with a straight line by linear interpolation, but this is not limiting, and for example, high-order spline interpolation is performed to connect the end points with a curve. But you can.

  FIG. 5 is a flowchart showing details of corner point detection (ST103) in FIG. 3, and FIGS. 6, 7 and 8 respectively show edge search (ST201), corner point estimation (ST203) and corners in FIG. It is explanatory drawing which shows an example of a process of point validity determination (ST204).

  As shown in FIG. 5, first, the corner point / throat point detection unit 33 searches for edge pixels on the diagonal line from the four sides of the imaging area 40 (ST201). More specifically, as shown in FIG. 6, in a rectangular imaging area 40, a virtual inclined at a predetermined angle (± 45 degrees in this case) with respect to coordinate axes (X axis and Y axis) along two sides orthogonal to each other. While the straight lines L1 to L4 are translated from the start points of the four vertices 45a to 45d in steps toward the center of the captured image from the start points to the center of the captured image, the virtual straight lines L1 to L4 Intersection points (edge pixels) between L4 and the contour component of book B are collected. The movement of the virtual straight lines L1 to L4 is carried out until a predetermined number of intersections capable of estimating the corner points later are detected.

  In general, the user is in the vicinity (the situation shown in FIG. 4B) of the center of the lower side of book B (that is, the portion where the lower side of book B intersects with the “throat”), the left and right sides of both spread pages, and the upper and lower sides of book B. Often presses the center. By detecting edge pixels diagonally from the four sides of the imaging area 40 in this way, it is possible to effectively eliminate a finger or the like placed by the user to eliminate the warp of the book B at the time of corner point detection. .

  Here, for the virtual straight line, the virtual straight lines L1 to L4 are largely moved at the first movement interval step1 until the first intersection from the start point is detected, and when the intersection is detected, the virtual straight line is detected from the position immediately before it. The straight line may be moved finely at the second movement interval step2.

  When the intersection of the contour component and the virtual straight line is collected while the virtual straight line inclined in this manner is translated stepwise at a predetermined movement interval, when the virtual straight line reaches the contour component near the corner point of book B, Since the row of intersections appears to be divided in two directions in a state of being substantially close to a right angle, the corner point of book B can be accurately detected from the appearance of the intersection. However, as an actual problem, the shape of the outer corner point of a book or the like is often dull, and in this embodiment, the position of the outer corner point is obtained by estimation as follows.

  First, the corner / throat point detection unit 33 detects a straight line for estimating a corner point from the edge pixel searched in ST201 (ST202). For example, as shown in FIG. 7, in the lower left corner region of book B, contour points Px1 and Px2 that exist approximately along the X-axis direction and contour points Py1 and Py2 that exist approximately along the Y-axis direction are In ST201, the corner point / throat point detection unit 33 can detect the straight line Lx passing through the contour points Px1 and Px2 and the straight line Ly passing through the contour points Py1 and Py2, respectively. Then, the corner point / throat point detection unit 33 sets the intersection point of the straight line Lx and the straight line Ly as the corner point C (ST203).

  Here, when there are more contour points, a plurality of straight lines are extracted based on two of them (that is, a plurality of intersections that are corner point candidates are calculated). Can be selected based on the angle θa and the positional relationship between the two points. Further, when two sides of the contour line are closed in the contour component (that is, edge pixels near the corner points are continuous), the virtual straight lines L1 to L4 are translated from the start point and detected first. The intersection point can be selected as the corner point C. That is, when a virtual straight line is in contact with a corner point, there is ideally one intersection with the contour component, so that the corner point is detected when contacted. In addition, if the virtual straight line is moved diagonally at a step 1 movement interval and a plurality of intersections are detected, the direction is opposite to the previous movement direction (that is, the direction of the vertex 45a in the case of the virtual straight line L1). It may be moved finely (for example, in step 2), and the corner point may be detected based on the fact that the intersection is 1 or 0. Then, by repeating these operations a plurality of times while reducing the step width, theoretically the detection accuracy of corner points is improved. Furthermore, when a noise component exists in the image of the mounting surface 6, it is good to remove beforehand.

  After setting the four corner points of book B, the corner point / throat detection unit 33 determines the validity of the corner points set based on the positional relationship between these corner points (ST204). For example, as shown in FIG. 8, the effectiveness of the corner points is based on the positional relationship between the four corner points C1 to C4 (such as a distance between the corner points and a comparison of angles formed by line segments connecting the corner points). It is determined by whether or not the corner points C1 to C4 are arranged in a substantially rectangular shape.

  Then, when all the corner points C1 to C4 are valid (ST205; YES), the corner point / throat point detection unit 33 ends the corner point detection process. On the other hand, when it is determined that any corner point is not valid (ST205; NO), the corner point / throat point detection unit 33 corrects the coordinates of the corner point determined to be invalid based on the other corner points ( ST206). For example, as shown in FIG. 8, when an erroneous corner point C3e is set by detecting the corner of the graphic frame on the paper, the positional relationship with the other three corner points C1, C2, C4 is determined. The corner point C3 can be set at an appropriate position.

  9 is a schematic diagram showing an example of the result of line segment recognition (ST105) in FIG. 3, and FIG. 10 is a schematic diagram showing an example of an edge trace processing process in line segment recognition (ST105) in FIG. is there.

  For example, in the edge trace processing for the lower side (outline) of the left page 41L, the throat point D1 and the corner point C1 are set as the start point and the end point, respectively, and the edge pixel is searched from the throat point D1 toward the corner point C1. To do. Here, a plurality of pixel patterns are prepared in advance based on line segment characteristics (straight lines, curves, branches, etc.), and when a group of searched edge pixels are compared with those pixel patterns, the line segments and Recognize its end points.

  In the edge trace process, as shown in FIG. 10A, when there is an obstacle (here, finger F2) covering the outline of book B, the line is bent at a relatively large angle. In this case, a line segment branches along the outer contour of the finger F2.

  Further, as shown in FIG. 10B, when an edge pixel cannot be searched in a partial region, the line segment is completely folded back to form an angle close to 0 degrees (or 360 degrees). Thus, when an edge cannot be searched due to a lack of an obstacle or an edge pixel, the edge search range is expanded (the next start point positioned in the end point direction is selected) and the search is continued. Finally, when the search for edge pixels reaches the end point (here, corner point C1) or the outer edge of the imaging area 40, the line segment recognition process ends.

  Note that the line segment recognition process is not limited to the above-described method as long as at least a plurality of line segment candidates that can form the outline of book B can be recognized, and other well-known methods may be used. Although only the line segment recognition of the lower side (outline) of the left page is shown here, the same process can be performed for the lower side of the right page.

  FIG. 11 is a flowchart showing details of the invalid line segment elimination (ST106) in FIG. 3, FIG. 12 is an explanatory diagram showing an outline of the processing of ST301 and ST303 in FIG. 11, and FIG. 13 is in FIG. FIG. 14 is an explanatory diagram showing an outline of the process of ST305, and FIG. 14 is a schematic diagram showing an example of the result of the invalid line segment elimination (ST106) in FIG.

  As shown in FIG. 11, the invalid line segment removal unit 35 acquires the coordinate information of both end points of each line segment recognized by the line segment recognition unit 34 (ST301), and the both end points are preset. It is determined whether it is within the effective area (ST302). If the line segment is not within the effective area (NO), the line segment is removed as an invalid line segment (ST307).

  Here, for example, as shown in FIG. 12, the effective region R is a parallelogram region extending in a strip shape from the start point (throat point D1) of the edge trace to the end point (corner point C1) in line segment recognition. be able to. Here, one set of opposite sides of the parallelogram is parallel to the reference line Ls connecting the start point and end point of the edge trace, and the other set of opposite sides is set to pass through the start point and end point of the edge trace, respectively. Yes.

  When the line segment is within the effective region (ST302; YES), the invalid line segment exclusion unit 35 further forms an angle θb formed by a straight line connecting both end points of the line segment L0 to be determined and the reference line Ls (FIG. 12). Reference) is calculated (ST303), and it is determined whether or not it is equal to or less than a predetermined threshold value (for example, 20 degrees) (ST304). When the angle θb exceeds the threshold value (NO), the no relevant line segment is removed as an invalid line segment (ST307).

  When the angle θb between the line segment and the reference line Ls is equal to or smaller than the threshold (ST304; YES), the invalid line segment exclusion unit 35 calculates the distance between the end point of the line segment L0 to be determined and another line segment adjacent thereto. It is calculated (ST305), and it is determined whether this distance is equal to or less than a predetermined threshold value (ST306).

  Here, for example, as shown in FIG. 13, the invalid line segment exclusion unit 35 sets the shortest point between the end point Pa of the determination target line segment and the virtual straight line Li passing through the end points Pb and Pc of the other adjacent line segments. A distance d is calculated, and this shortest distance d is compared with a threshold value. In this case, the distance ΔX in the X-axis direction between the end point Pa and the end point Pb can be calculated, and the threshold value can be changed according to the value of the distance ΔX. For example, the threshold value may be increased linearly as the distance ΔX increases. Other adjacent line segments can be selected based on the size of ΔX (the line segment that minimizes ΔX is selected). Similarly, in the case of the left and right sides in the outer contour of the book B, the threshold value can be changed according to the distance ΔY in the Y-axis direction.

  When all the line segments are finally determined (ST308), the invalid line segment removal process is completed, and an outline composed of effective line segments as shown in FIG. 14 is obtained. Note that the processing of ST305 and ST306 can be omitted. Here, in the formed outline, there is a missing portion La in which the continuity of each line segment is cut off due to shielding by an obstacle (here, a finger) or missing edge pixels.

  FIG. 15 is a schematic diagram showing an example of the result of the line segment interpolation (ST107) in FIG.

  In the document reading system 1, as described above, invalid line segments are excluded from the line segment candidates constituting the outline based on the angle with respect to the line segment connecting the corner point (C1 or C2) and the throat point (D1). After that, as shown in FIG. 15, with respect to the missing portion La (see FIG. 14) in the outline formed by the remaining effective line segments, the line segments already generated by the edge trace and the missing portion La A line segment Lc is generated by combining open ends of two line segments adjacent to both sides. That is, interpolation is performed by inserting a line segment. Alternatively, an open end of a line segment that already exists is extended on an extension line of the line segment, and is connected to an open end of a line segment that is opposed to the missing portion La. Thereby, even when a part of the outline cannot be acquired due to the presence of a finger covering a part of the outline (lower side) on the lower side of the document at the time of photographing, the missing part La of the outline is easily and reliably complemented. can do.

  Although the present invention has been described based on specific embodiments, these embodiments are merely examples, and the present invention is not limited to these embodiments. In the above embodiment, only the processing of the lower side (outline) of the document has been described, but the same can be applied to the upper side of the document. Furthermore, the same processing can be performed for the left and right sides of the document (outline of the Y-axis direction). In this case, the coordinates of the throat point are not necessary, and edge tracing is performed between the upper and lower corner points. Further, the corner point used as the start point or end point of the edge trace in line segment recognition may be a page corner point. Note that not all of the components of the image processing apparatus according to the present invention and the document reading system including the image processing apparatus according to the present invention shown in the above-described embodiments are necessarily selected as long as they do not depart from the scope of the present invention. It is possible.

  The image processing apparatus according to the present invention and the document reading system provided with the image processing apparatus have a missing outline even when a part of the outline cannot be acquired due to the presence of a finger or the like covering a part of the outline during shooting. The present invention is useful as an image processing apparatus that processes an image obtained by reading a document such as a book, a document reading system including the same, and the like, which can easily and reliably complement a portion.

1 Document Reading System 2 Document Camera (Image Input Device)
3 PC (image processing device)
22 Image data input unit (captured image acquisition unit)
23 Image processing device 30 Outline detection unit 31 Edge extraction unit 33 Corner point / throat point detection unit 34 Line segment recognition unit 35 Invalid line segment exclusion unit 36 Line segment interpolation unit 41L Left paper surface 41R Right paper surface C, C1 to C4 Corner points D1 , D2 throat

Claims (5)

A captured image acquisition unit that acquires captured images of left and right sheets of a bound document;
An edge extraction unit that extracts a plurality of edge pixels in the captured image;
A corner point detector for detecting left and right corner points on at least one side of the document in the vertical direction;
A throat point detection unit for detecting a throat point that is an end point of a binding center line on at least one side of the document;
A line segment recognition unit that recognizes a plurality of line segment candidates that can form an outline on at least one side of the document based on the edge pixels;
With respect to at least one of the left and right paper surfaces, an invalid line segment eliminating unit that eliminates invalid line segments from the line segment candidates based on an angle with respect to a line segment connecting the corner point and the throat point;
An image processing apparatus comprising: a line segment interpolation unit that performs line segment interpolation on a missing portion in an outline formed by the line segment candidates from which the invalid line segment is excluded.   The image processing according to claim 1, wherein the line segment recognition unit recognizes the plurality of line segment candidates by tracing the edge pixel between the corner point and the throat point. apparatus.   The invalid line segment exclusion unit calculates a shortest distance between the end point of the line segment candidate and a virtual straight line passing through both end points of another line segment adjacent thereto, and the shortest distance is equal to or greater than a predetermined threshold value. 3. The image processing apparatus according to claim 1, wherein the line segment candidates are excluded as invalid line segments. A captured image acquisition unit for acquiring a captured image of a paper surface of the document;
An edge extraction unit that extracts a plurality of edge pixels in the captured image;
A corner point detector for detecting upper and lower corner points on at least one side in the left-right direction of the document;
A line segment recognition unit that recognizes a plurality of line segment candidates that can form an outline on at least one side of the document based on the edge pixels;
An invalid line segment exclusion unit that eliminates invalid line segments from the line segment candidates based on an angle with respect to the line segment connecting the upper and lower corner points;
An image processing apparatus, comprising: a line segment interpolation unit that performs line segment interpolation on a missing portion in an outline formed of a plurality of line segments from which the invalid line segment is excluded.   An original reading system comprising: the image processing apparatus according to claim 1; and an image input apparatus including a camera unit that generates the captured image.

Images