JP2013225819A - Image processing device and manuscript reading system having the same - Google Patents

Abstract

PROBLEM TO BE SOLVED: To prevent the occurrence of an unnatural region caused by missing imaged image data at four corners of a rectangular image display area, when displaying an imaged image with tilt correction on a display etc., and to enable an efficient use of imaged image data.SOLUTION: A manuscript reading system comprises: an image rotation unit for tilt correcting an imaged image; a cutout area setting unit for setting a rectangular cutout area positioned in an image area 40 of a pre-corrected imaged image before tilt correction, and positioned in an image area 41 of a reference correction imaged image obtained by rotating the imaged image by a predetermined reference rotation angle; from the imaged image tilt corrected by the image rotation unit, an image cutout unit for cutting out an image in a region corresponding to a cutout area 42; and a display unit for displaying the imaged image on a rectangular image display area, on the basis of display data generated from the image cut out by the image cutout unit.

Description

  The present invention relates to an image processing apparatus that processes an image obtained by reading a document such as a book, and a document reading system including the image processing apparatus.

  2. Description of the Related Art Document cameras (book scanners) that can take a picture of a page and read an image of the page while the page of the book is naturally opened are widely used (see Patent Document 1). When such a document camera is used, the images on the page can be read one after another while turning the page, so that the work of digitizing the book can be performed efficiently. Further, in the document reading system using the document camera, even when the document is set in an inclined state with respect to a normal reading position (that is, a reference direction such as horizontal or vertical), an image of the read document ( The inclination of the image can be automatically corrected by estimating the inclination angle based on the document binding portion and the position of the document edge.

  By the way, when the captured image is rotated and its inclination is corrected, areas that do not overlap with the image area of the captured image before the inclination correction are generated at the four corners of the rectangular image area. Therefore, when a captured image after tilt correction is displayed on a display or the like, regions where image data is lost are generated at the four corners of a rectangular image display area set based on the image area before tilt correction.

  Therefore, for example, the four corners of the screen (image display area) caused by the rotation of the image (image data loss) are masked using a specific pattern and synthesized with another pattern image (specified as the image data loss region). In other words, there is a technique for removing unnaturalness of an image due to a screen cut (see Patent Document 2).

JP 2001-103240 A JP 11-328408 A

  However, in the prior art described in Patent Document 2, it is necessary to generate data for specific display in a missing area of image data in accordance with a specific pattern that masks the cutting of the four corners of the image display area. Therefore, there is a problem that the image display processing becomes complicated. Further, even if the four corners of the image display area are masked with a specific pattern, the problem that the image display area becomes an unnatural octagon has not been solved.

  In this case, by using only image data of a predetermined area (size that can be accommodated in the image display area even if the image is rotated by tilt correction) in the center of the captured image, the image data at the four corners in the image display area is used. It is also conceivable to avoid an unnatural display caused by a defect. However, since the shape and size of the missing area of the image data (in other words, the area not affected by the rotation of the image in the image display area) changes according to the rotation angle of the image (the angle to be corrected). As a result, the displayed image size and imaging region change from moment to moment, and there arises a problem that it is difficult to stably perform image display and the like.

  Furthermore, in the above prior art, it is assumed that the captured image is rotated to a relatively large angle (for example, ± 180 °), and a circular or elliptical shape is used as the specific pattern. Therefore, the captured image having a rectangular image area is used. There was also a problem that it was difficult to efficiently use the data.

  The present invention has been devised in view of such problems of the prior art, and even when displaying a photographic image with tilt correction on a display or the like, it is photographed at the four corners of a rectangular image display area. An object of the present invention is to provide an image processing apparatus capable of preventing the occurrence of an unnatural area due to image data loss and making it possible to efficiently use captured image data, and a document reading system including the image processing apparatus.

  An image processing apparatus according to the present invention includes an image rotation unit that performs tilt correction of a captured image by rotating a rectangular captured image obtained by capturing a paper surface of a document around the centroid, and shooting before correction before the tilt correction. A cut-out area setting unit that sets a rectangular cut-out area that is located within the image area of the image and that is located within the image area of the reference-corrected shot image that is obtained by rotating the shot image at a predetermined reference rotation angle; An image cutout unit that cuts out an image of an area corresponding to the cutout area from a shot image that has been tilt-corrected by the unit, and a shot image in a rectangular image display area based on display data generated from the image cut out by the image cutout unit And a display unit for displaying the.

  As described above, according to the present invention, even when a captured image with tilt correction is displayed on a display or the like, the occurrence of unnatural areas due to the loss of captured image data at the four corners of the rectangular image display area In addition, it is possible to prevent the image data and to effectively use the captured image data.

1 is an overall configuration diagram showing a document reading system 1 according to the present invention. The top view which shows the state which set the book B in the regular reading position in the original reading system 1 shown in FIG. 1 is a block diagram showing a schematic configuration of the document camera 2 and the PC 3 in FIG. FIG. 1 is a flowchart showing a main part of an image display procedure by the document reading system 1 shown in FIG. Explanatory drawing which shows an example of the input image in the image display procedure shown in FIG. Explanatory drawing which shows the outline | summary of inclination correction | amendment (ST103) in FIG. Explanatory drawing which shows the outline | summary of image extraction (ST104) in FIG. Explanatory drawing which shows the outline | summary of the image display (ST105) in FIG. FIG. 4 is a flowchart showing details of the cut-out area setting (ST101) in FIG. Explanatory drawing which shows the outline | summary of extraction area setting (ST101) in FIG. Flow chart showing details of tilt correction (ST103) in FIG. Flow chart showing details of line segment information statistical processing (ST305) in FIG. FIG. 4 is a flowchart showing details of image cutting (ST104) in FIG.

  A first invention made to solve the above-described problem is an image rotation unit that corrects the inclination of a photographed image by rotating a rectangular photographed image obtained by photographing the paper surface of the document around the centroid. A cutout area for setting a rectangular cutout area that is located in the image area of the pre-correction photographed image before tilt correction and that is located in the image area of the reference correction photographed image obtained by rotating the photographed image by a predetermined reference rotation angle A rectangular shape based on a setting unit, an image cutout unit that cuts out an image of an area corresponding to the cutout area from a captured image that has been tilt-corrected by the image rotation unit, and display data generated from the image cut out by the image cutout unit And a display unit for displaying a photographed image in the image display area.

  According to this, a cut-out area that is located in both the image area of the pre-correction shot image and the image area of the reference correction shot image is set, and an image corresponding to the rectangular cut-out area is cut out from the tilt-corrected shot image. Since the captured image is displayed on the basis of the display data generated in this way, even if the captured image subjected to tilt correction is displayed on a display or the like, there is a problem caused by the loss of the captured image data at the four corners of the rectangular image display area. The generation of natural areas can be prevented and the captured image data can be used efficiently.

  In a second aspect based on the first aspect, the cut-out area has a shape obtained by reducing the image area of the pre-correction captured image with the same aspect ratio.

  According to this, it becomes easy to set a rectangular cut-out area, and it is possible to display an image without giving a sense of incongruity to the user by an image cut out with the same aspect ratio as that of a standard image area.

  According to a third aspect of the present invention, in the first or second aspect of the present invention, in the cutout area, two vertices forming the opposite vertices are positioned on a side of the image area of the reference corrected photographed image.

  According to this, it becomes easy to set a rectangular cut-out area, and it is possible to more efficiently use captured image data in display data.

  According to a fourth aspect of the present invention, in any one of the first to third aspects, the image correction unit further includes a tilt correction angle setting unit that sets a tilt correction angle of the captured image, and the image cutout unit includes the tilt correction. When the angle is larger than the reference rotation angle, the image is not cut out.

  According to this, when rotation larger than the reference rotation angle is performed in the inclination correction of the captured image, it is possible to prevent the occurrence of unnatural areas at the four corners of the clipped image.

  According to a fifth aspect, in any one of the first to fourth aspects, the display unit includes an image display area corresponding to the cut-out area.

  This eliminates the need for processing such as displaying the entire captured image (the clipped image) in a large size in the image display area and combining another image with a portion where the captured image does not exist.

  According to a sixth aspect of the present invention, there is provided a document reading system including the image processing apparatus according to any one of the first to fifth aspects of the present invention and an image input apparatus having a camera unit that generates the photographed image.

  According to a seventh aspect, in the sixth aspect, the camera unit can change a photographing range.

  According to this, regardless of the size of the document, the user adjusts the shooting range of the camera unit on the basis of the shot image displayed in the image display area of the display unit, so that the document area that needs to be shot is within the cut-out area. Therefore, it is possible to take a picture without being aware of the image clipping.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is an overall configuration diagram showing a document reading system 1 according to the present invention, and FIG. 2 is a plan view showing a state where a book B is set at a regular reading position in the document reading system 1. The document reading system 1 reads a paper image of a book (original) B to acquire image data of the paper surface, and a document camera (image input device) 2 that shoots the paper surface and converts it into a video signal. The document camera 2 and the PC 3 connected to be communicable with each other.

  The document camera 2 includes a camera unit 4 having a photographing function and a stand unit 5 that holds the camera unit 4. The camera unit 4 includes an image sensor made up of a CCD, a CMOS, etc., and an illumination light source (both not shown) made up of an LED, a fluorescent lamp, or the like. The stand unit 5 includes a leg 7 that is mounted on a mounting surface 6 such as a desk surface and that is open in a substantially V shape, and an arm 8 that is supported by the leg 7. On the upper surface of the leg 7, a pair of guide members 9 for defining the photographing position of the book B are projected. The arm 8 is provided so as to be capable of extending and contracting obliquely upward from the leg 7, while holding the camera unit 4 so as to be rotatable by a hinge unit 8 a, whereby the angle of view and the optical axis direction of the camera unit 4 (the shooting direction). ) Is adjustable. The camera unit 4 also has a known zoom function that continuously changes the angle of view and magnification.

  The PC 3 functions as an input / output device for the user to set various operating conditions of the document camera 2 and for the user to check a captured image captured by the document camera 2, and to perform image processing and recording of the captured image. It also functions as an image processing apparatus that performs the above.

  The apparatus used together with the document camera 2 in the document reading system 1 is not limited to a PC (Personal Computer), and any information processing apparatus capable of realizing the same function can be used. It is also possible to add a part of the functions of the PC 3 to the document camera 2 or to configure the PC 3 and the document camera 2 integrally. Further, the document camera 2 and the PC 3 do not need to be directly connected, and may be connected via a network (not shown), for example. In such a configuration, the document camera 2 generates a trigger signal for imaging (specifically, for example, a switch or button for instructing image capture), and pushes the remote PC 3 (for example, a server). Data transmission.

  In photographing with the document camera 2, the user places the book B in an open state on the placement surface 6 immediately below (in the optical axis direction) of the camera unit 4, so that two pages of the book B and the periphery of the book B are placed. A captured image (moving image or still image) including a part of the mounting surface 6 is obtained. At this time, as shown in FIG. 2, the user sets the book B at the reading position with high accuracy by adjusting the position and inclination of the book B by contacting the upper edge Ba of the book B with the guide member 9 extending in the left-right direction. Is possible. The captured image is appropriately transmitted to the PC 3, where necessary image processing is performed, and then stored in a predetermined recording medium and displayed on the display for the user.

  The document read by the document reading system 1 is not limited to a book, but may be any information transmission medium including characters, drawings, photographs, and similar information. Further, the information transmission medium may be a transmission original such as a positive film or a negative film. In this case, for example, light is emitted from the back side by a light source unit in which a light source is arranged on the side of a transparent light guide plate, and transmitted light is transmitted. May be configured to take an image.

  FIG. 3 is a block diagram showing a schematic configuration of the document camera 2 and the PC 3 in FIG.

  The document camera 2 includes an imaging processing unit 11 having a camera unit 4, an operation instruction unit 12 that causes the imaging processing unit 11 to perform a required operation based on an operation condition set by a user, and a PC 3. And an external interface 13 conforming to the USB standard or the like.

  The PC 3 includes an external interface 21 compliant with the USB standard for connection with the document camera 2, an image data input unit 22 to which photographed image data from the document camera 2 is input, and recording and display of photographed images. An image processing unit 23 that performs necessary image processing, an operation system control unit 25 that transmits operation conditions set by the user to the document camera 2 in an input operation unit 24 such as a keyboard, and a captured image after image processing. It has a display data generation unit 27 that generates data to be displayed on a display unit 26 such as an LCD or a projector, and a data storage unit 28 that stores captured image data after image processing. The image processing unit 23 includes an image tilt correction unit 30 that performs tilt correction of a captured image. Captured image data processed by the image tilt correction unit 30 is read from the image data storage unit 38, and various parameters used for a reference image rotation angle and statistical processing of the captured image in the image tilt correction unit 30 described later. Is read from the parameter storage unit 39. The parameter storage unit 39 stores information related to the image sensor mounted on the camera unit 4 in advance (for example, the number of vertical and horizontal effective output pixels (pixels)).

  The processing functions of the image data input unit 22, the image processing unit 23, the operation system control unit 25, the display data generation unit 27, and the like in the PC 3 can be realized by software processing in which a program such as an image processing application is executed by the CPU. Of course, the PC 3 may be interpreted as an image processing apparatus, and may be configured to include hardware that executes specific processing at high speed. The image data storage unit 38 and the parameter storage unit 39 are composed of general-purpose memories.

  In the PC 3, the image data input unit 22 sequentially stores the captured image data transmitted from the document camera 2 in the image data storage unit 38 of the image processing unit 23, and the captured image data is stored in the image inclination correction unit 30 as necessary. Are output sequentially. Then, the image tilt correction unit 30 estimates the tilt angle of the captured image during image processing of the captured image data, and corrects the tilt of the captured image based on the estimated tilt angle. The captured image data whose inclination is corrected is stored in the data storage unit 28 and is also sent to the display data generation unit 27 and displayed on the display unit 26.

  As will be described in detail later, the image inclination correction unit 30 includes a cut-out area setting unit 31 that sets a rectangular cut-out area that is cut out from the shot image and displayed on the display unit 26, and performs grayscale conversion of the shot image. A grayscale conversion unit 32 that performs binarization, an edge extraction unit 33 that extracts a plurality of edge pixels (performs edge detection) from the grayscale-converted captured image, and the edge based on the extracted plurality of edge pixels While estimating the inclination angle of a picked-up image by performing a statistical process about the inclination angle of the extracted some line segment, the line segment extraction part 34 which extracts the several line segment (straight line component) which ties a pixel, An inclination correction angle setting unit 35 that sets an inclination correction angle of the captured image based on the inclination angle, and an image rotation unit 36 that rotates the captured image based on the inclination correction angle. , And an image cut-out unit 37 cuts out the image of the region corresponding to the cut-out area of the supra from photographed image tilt correction by the image rotation unit 36.

  In the PC 3, the user can appropriately input operating conditions such as the resolution of the image captured by the document camera 2, the frame rate, the shutter speed, and the amount of light emitted from the illumination light source by operating the input operation unit 24. it can. This operating condition is transmitted as a control signal from the operation system control unit 25 to the document camera 2, and the document camera 2 captures the imaging processing unit 11 in accordance with a processing command sent from the operation instruction unit 12 based on the control signal from the PC 3. Performs the shooting operation.

  4 is a flowchart showing the main part of the image display procedure by the document reading system 1 shown in FIG. 1, and FIG. 5 is an explanatory diagram showing an example of the input image in the image display procedure shown in FIG. 6, 7 and 8 are explanatory diagrams showing the outline of the inclination correction (ST103), image cutout (ST104) and image display (ST105) in FIG. 4, respectively.

  A user who uses the document reading system 1 first sets the book B in an opened state at a reading position (position where the document camera 2 can be photographed), activates the document camera 2, and activates a required application on the PC 3. . Thereby, image input, that is, imaging with the document camera 2 is started, and the captured image data is sequentially transmitted to the PC 3. At this time, in the PC 3, as will be described in detail later, a rectangular cut area 42 (see FIG. 7) that becomes a region cut out from the captured image and displayed on the display unit 26 is set by the cut area setting unit 31 ( ST101).

  Here, for example, as shown in FIG. 5, the image area 40 of the photographed image has a rectangular outline based on the photographing area of the document camera 2, and in this image area 40, the image of the book B and its background And an image of the mounting surface 6 to be. The page of book B (here, a spread page) includes a text area 43 in which characters and symbols are displayed and a graphic area 44 in which drawings and photographs are displayed.

  FIG. 5 shows an example of a photographed image that needs to be tilt-corrected. If the horizontal (horizontal) direction of the image area 40 is the X-axis direction and the vertical (vertical) direction is the Y-axis direction, the image of the book B The virtual horizontal line (document reference direction) H1 is inclined by an angle θ with respect to a correction reference line H0 parallel to the X axis, which is a target direction of inclination correction described in detail later. Although illustration is omitted here for convenience of explanation, a virtual vertical line related to the image of the book B can also be set as the reference direction of the document in the same manner as the horizontal line H1, and in this case, the direction parallel to the Y axis Is the target direction for tilt correction.

  Referring to FIG. 4 again, next, when the input of the photographed image from the document camera 2 is detected in the image data input unit 22 of the PC 3 (ST102: YES), the inclination correction angle setting is set as described in detail later. The unit 35 estimates the tilt angle of the input captured image and sets the tilt correction angle. Further, based on the tilt correction angle, the image rotation unit 36 rotates the rectangular captured image to correct the tilt of the captured image. (ST103).

  Here, assuming that the image area 40 of the photographed image before tilt correction is an image display area (that is, an image display area on a display or the like), the photographed image is tilt-corrected around the centroid G as shown in FIG. When the image is rotated clockwise by an angle θ, an area that does not overlap with the image area 40 is generated in the image area 41 after tilt correction indicated by a broken line in FIG. 6 (that is, image data is stored at the four corners of the image display area). A missing region R occurs).

  Therefore, as shown in FIG. 7, the image cutout unit 37 cuts out an image of the area corresponding to the cutout area 42 set in ST101 from the captured image after the inclination correction (ST104), and the image data in the image display area is missing. The region R (see FIG. 6) is removed.

  Thereafter, the display unit 26 displays a captured image (cut image) in a rectangular image display area based on the display data generated by the display data generation unit 27 from the image cut out by the image cutout unit 37 (ST105).

  As described above, when the cutout area 42 is set so as to remove the missing area R of the image data generated by the rotation of the image data, an unused area of the image data (imaging data) is formed in the peripheral portion of the image area 40 of the photographed image. (In FIG. 7, the portion of the captured image excluding the cut-out area 42 from the image area 40 is a non-use area). In this case, if the area occupied by the image data output from the image sensor (that is, the image area 40 of the pre-correction captured image) is displayed on the display unit 26 as it is, not only the cutout area 42 but also the unused area is covered. It is displayed and the display becomes unnatural. That is, although it does not appear as an image defect at the four corners as in the region R described above, the entire periphery of the image is displayed in a missing state.

  Thus, for example, as shown in FIG. 8, a rectangular image display area 52 corresponding to the cut-out area 42 is provided on the screen 51 of the display unit 26. In this image display area 52, the captured image (entire image) is displayed in ST104. Only images cut out from (Image) are displayed. This eliminates the need for processing such as displaying the entire captured image (the clipped image) in a large size in the image display area 52 and combining another image with a portion where the captured image does not exist. Here, an operation setting area 53 including a plurality of operation setting buttons that can be operated by the user with a mouse or the like (not shown) is provided around the image display area 52, but the operation setting area 53 is not provided. Alternatively, the entire screen 51 may be used as the image display area 52.

  As will be described later, the size of the cutout area 42 is uniquely determined by a reference rotation angle θs (maximum inclination angle of the document, which will be described later in detail) set by the user, so that the image display area is set according to the reference rotation angle θs. 52 may be determined. For example, if the reference rotation angle θs is increased, the cut-out area 42 is reduced, and the image display area 52 may be reduced accordingly. On the other hand, when the size of the image display area 52 is constant, the display magnification may be changed according to the reference rotation angle θs. That is, since the image size is small when the reference rotation angle θs is large, the display magnification may be relatively increased.

  In the PC 3, the series of processing ST <b> 102 to ST <b> 105 is repeatedly executed, whereby a series of images (video) whose inclination is corrected is displayed on the display unit 26. When it is finally determined that image input is not performed for a predetermined time (ST102: NO), all operations are completed.

  Since the cutout area 42 is smaller than the image area 40 of the photographed image as described above, the document may not completely fit in the image display area 52 in FIG. The angle of view (shooting range) is changed while viewing the screen 51 by adjusting the position (height) of the camera unit 4 and using the zoom function, and the entire document (or a part of the document that needs to be shot) is changed. The operation of fitting in the image display area 52 is naturally performed. As a result, it is possible for the user to execute shooting without being conscious of image clipping.

  Now, regarding the provision of the cutout area 42, the number of pixels actually displayed is reduced with respect to the number of effective output pixels of the image sensor due to the occurrence of the above-described unused area. However, since recent image sensors are increasing in pixel count (for example, products with 5 million pixel image sensors are known as document cameras), the presence of unused areas immediately affects image quality. Not give.

  9 and 10 are a flowchart and an explanatory diagram showing details of the cut-out area setting (ST101) in FIG.

  As shown in FIG. 9, first, the cut-out area setting unit 31 accesses the parameter storage unit 39 (see FIG. 3), and acquires the size information of the input image (ST201). Here, as the image size information, the image area 40 of the photographed image before tilt correction shown in FIG. 10 (that is, the image area of the previous photographed image that does not require tilt correction displayed in the entire image display area as in the past). The number of pixels Lix in the X-axis direction and the number of pixels Liy in the Y-axis direction corresponding to the length in the X-axis direction and the Y-axis direction, respectively.

  Next, the cutout area setting unit 31 acquires the value of the reference rotation angle θs that is the rotation angle of the captured image for setting the cutout area 42 (ST202). The default value of the reference rotation angle θs is also stored in the parameter storage unit 39, but the user can set a predetermined reference rotation angle θs separately via the input operation unit 24. As the reference rotation angle θs, a maximum value (for example, ± 20 °) of an inclination of a document (such as a book) assumed in normal shooting is set.

  Here, the user using the document reading system 1 installs the document camera 2 at a position facing the user, and keeps the upper edge Ba of the book B as shown in FIG. It is considered natural to place the book in the shooting range by abutting against the guide member 9 extending in the left-right direction. Then, since the inclination angle of the captured image is usually within a certain range, it is not necessary to consider a relatively large image inclination (for example, ± 45 ° or more).

  Subsequently, the cutout area setting unit 31 calculates the size of the cutout area 42 (ST203). This cut-out area 42 is located at least in the image area 40 of the photographed image (pre-correction photographed image) before tilt correction, and is an image of the photographed image (reference-corrected photographed image) obtained by rotating the photographed image at the reference rotation angle θs. It is set to be located in the area 41. Thereby, even when the captured image subjected to tilt correction is displayed on a display or the like, it is possible to prevent the occurrence of unnatural areas due to the loss of the captured image data at the four corners of the rectangular image display area 52, and It is possible to efficiently use rectangular captured image data.

  Here, in particular, as shown in FIG. 10, the cutout area 42 is set so that two vertices P <b> 1 and P <b> 3 forming the opposite vertices are positioned on the side of the image area 41 of the reference correction photographed image (however, It is assumed that the tilt correction is executed by rotating the image area 40 of the pre-correction captured image clockwise, and when the tilt correction is performed in the counterclockwise direction, the vertices P2 and P4 are the image areas of the reference corrected captured image. 41). Thereby, the rectangular cut-out area 42 can be easily set, and the cut-out area 42 can be maximized to use the captured image data more efficiently. In addition, the cut-out area 42 is set to have a rectangular shape that is reduced around the centroid G with the same aspect ratio as the image area 40 of the pre-correction captured image. As a result, it is possible to display an image without giving a sense of incongruity to the user by an image cut out with the same aspect ratio as that of the standard image area 40.

  Further, as can be understood from FIG. 10, the cutout region 42 becomes larger as the reference rotation angle θs is smaller. That is, as θs is smaller, the side position of the image area 41 of the reference corrected photographed image that intersects the diagonal line of the image area 40 of the pre-correction photographed image moves outward, and the interval between P1 and P3 increases. Therefore, when the non-use area of the above-described image sensor is desired to be reduced (for example, when the original is read with reduced resolution degradation), the user may set the reference rotation angle θs to be small. However, since the maximum angle at which the tilt correction is performed at this time becomes small, it is necessary to set the document on the document camera so as not to tilt as much as possible. Conversely, when it is necessary to limit the amount of data of the read image or when high resolution is not required, the user may set θs large. When θs is increased, the cut-out area 42 is reduced, but this is set irrespective of the arrangement state of the original (that is, whether the inclination correction is necessary and the rotation angle). That is, the size of the cutout area 42 is reduced even when the document does not require tilt correction. If the document does not fit in the cut-out area 42 that has become smaller, the user raises the camera position of the document camera 2 so that the entire image is displayed, for example, so the resolution when reading the same document is lowered. As described above, the user can substantially adjust the reading resolution of the document by setting the reference rotation angle θs.

  In the above description, tilt correction (image rotation processing) is performed with the centroid G as the axis. However, in the implementation of software that performs rotation processing, the origin of the X and Y coordinates (for example, FIG. 10). Rotation processing may be performed centering on the left corner of the image area 40 of the pre-correction photographed image, and finally coordinate conversion may be performed so that the center of the rotation area matches the centroid G. That is, using the centroid G as an axis in the rotation process itself is not an essential concept.

  Lx and Ly, which are ½ widths of the length in the X-axis direction and the Y-axis direction of the cut-out area 42 shown in FIG. 10, can be calculated by the following (Expression 1) and (Expression 2).

  Then, the cutout area setting unit 31 calculates the coordinates of the vertices P1 to P4 of the cutout area 42 from the coordinates of the respective vertices of the known image area 40 and the values of Lix, Liy, Lx, and Ly (ST204).

  FIG. 11 is a flowchart showing details of the inclination correction (ST103) in FIG. In the PC 3, after the captured image data acquired from the document camera 2 is stored in the image data storage unit 38 by the image data input unit 22 (ST 301), the grayscale conversion unit 32 converts the captured image from an RGB color image to a monochrome image. (ST302). This gray scale conversion process can be performed using a known method such as an intermediate value method. If the document camera 2 is configured to directly output the signal after YC separation, the Y (luminance) signal may be used as it is.

  Subsequently, the edge extraction unit 33 extracts, as edge pixels, a portion where the luminance changes abruptly in the grey-scale converted captured image (ST303). This edge pixel extraction process can be performed using a known method such as the Canny method. Furthermore, the line segment extraction unit 34 extracts a line segment as a straight line component from the acquired plurality of edge pixels (ST304). This line segment extraction process can be performed using a known method such as probabilistic Hough transform. Probabilistic Hough transform is used when detecting a line segment having an end point from an image, and the coordinate values of the start point and end point of the detected line segment can be acquired. Data of each line segment such as the coordinate value of the end point extracted here is stored in the parameter storage unit 39. Note that the linear component extracted from the photographed image is not limited to the line segment shown here, and any component that reflects at least the degree of inclination of the book B may be used.

  Next, the tilt correction angle setting unit 35 performs a statistical process (hereinafter referred to as a line segment information statistical process) on information regarding the tilt angles of the plurality of line segments detected by the line segment extracting unit 34, thereby taking a captured image. Is estimated (ST305). In this line segment information statistical process, histogram information relating to the inclination angle of the line segment is generated.

  Thereafter, the image rotation unit 36 corrects the inclination of the photographed image by rotating the photographed image based on the estimated inclination angle, whereby the paper surface of the book B at the regular reading position (that is, without the inclination). A captured image equivalent to the case where is read is generated (ST306). By this image rotation processing, for example, as shown in FIG. 6, the reference direction (horizontal / vertical direction) of the document and the target direction (X-axis / Y-axis direction) substantially coincide with each other, and the inclination of the photographed image is eliminated. .

  FIG. 12 is a flowchart showing details of the line segment information statistical processing (ST305) in FIG. First, the inclination correction angle setting unit 35 acquires information on all line segments (hereinafter referred to as “line segment information”) about the captured image that is the target of image processing from the parameter storage unit 39 (ST401). This line segment information is generated by the stochastic Hough transform or the like in ST304 described above, and includes, for example, coordinate data of both end points in each of a plurality of line segments.

  Next, in order to determine the mutual intersection state of each line segment, one line segment is extracted as an inspection target line segment (line segment to be determined) from all the line segments in the line segment information (ST402), and further Then, one line segment other than the inspection target line segment is extracted as a comparison target line segment from all the line segments in the line segment information (ST404). It should be noted that at least at the start of the line segment information statistical process, there are inspection target line segments and comparison target line segments that have not been compared. Therefore, it is determined No in both ST403 and ST405 for determining the presence or absence of the target line segment. Is done.

  Next, it is determined whether or not the inspection target line segment extracted in ST402 intersects with the comparison target line segment extracted in ST404. If it intersects (ST406; Yes), the inspection target line segment is extracted in ST402. The information is excluded from the candidates (ST407), and the information on the comparison target line segment to be extracted in ST404 is reset (ST408). Thereby, the inspection target line segment is excluded from the target of statistical processing. Thereafter, the process returns to ST402 again, and the next inspection target line segment for which the intersection state has not yet been determined is extracted in the same manner as described above. In addition, the intersection state of each line segment can be determined using a known method based on the coordinates of both end points of the line segment.

  On the other hand, if the inspection target line segment and the comparison target line segment do not intersect each other (ST406; No), the comparison target line segment is excluded from the extraction candidates of ST404 (ST409). After that, the process returns to ST404 again, and the next comparison target line segment for which the intersection state has not yet been determined is extracted in the same manner as described above.

  Finally, when there is no comparison target line segment that intersects the predetermined inspection target line segment (ST405; Yes), the inclination correction angle setting unit 35 sets the inclination angle θ based on the coordinate data of the inspection target line segment. The calculation result is added (ST410), and the calculation result is added to the histogram information (ie, extracted as a target for statistical processing) (ST411). That is, the frequency corresponding to θ is counted up with the horizontal axis as the tilt angle θ and the vertical axis as the frequency.

  As a result of this processing, when the determination of the intersection state of all the inspection target line segments is finished (ST403; Yes), the inclination correction angle setting unit 35 generates a histogram based on the histogram information added in ST411. The maximum frequency in the inclination angle distribution of the histogram is searched (ST412). Then, the tilt correction angle setting unit 35 estimates the retrieved maximum frequency value (for example, the intermediate value of the class interval) as the tilt angle of the captured image (that is, the tilt correction angle) (ST413).

  In this way, by eliminating the intersection line segment from the target of statistical processing, an effective line segment (line segment along the reference direction of the document) appropriate for estimating the tilt angle of the captured image is captured as the processing target. It is possible to accurately estimate the inclination angle of the image. Here, the tilt angle of the captured image is estimated based on the linear component (line segment) extracted from the captured image. However, the present invention is not limited to this, and the tilt angle of the captured image may be estimated by other known methods. . In some cases, a configuration including various sensors for detecting the inclination of the document itself is also possible.

  FIG. 13 is a flowchart showing details of the image cutout (ST104) in FIG. First, the image cutout unit 37 acquires the captured image whose inclination has been corrected in ST103 (ST501), and then secures a memory area in the data storage unit 28 for storing the output image (cutout image) (ST502). .

  Next, the image cutout unit 37 acquires the coordinates of the vertices P1 to P4 (see FIG. 10) of the rectangular cutout area 42 calculated in ST101 (ST503). Therefore, the image cutout unit 37 sequentially extracts all the pixels in the Y-axis direction and the X-axis direction starting from a predetermined pixel (for example, the vertex P1) within the cutout area 42 of the photographic image whose inclination has been corrected. (ST504, ST505) Based on the coordinates of the extracted pixel, it is determined whether or not the pixel is in the cutout area 42 (ST506). Note that ST509 and ST508 are loop end terminations corresponding to ST504 and ST505, respectively.

  When the pixel extracted in ST506 is in the cut-out area 42 (YES), the pixel value information of the pixel is stored in the memory area set in ST502 (ST507). On the other hand, when the extracted pixel is not in the cutout area 42 (NO), the pixel value of the pixel is excluded from the display data as unnecessary information.

  In this way, pixel values are acquired for all the pixels in the cutout area 42. As a result, the display data generation unit 27 can generate appropriate display data from the cut-out image data stored in the data storage unit 28.

  Although not shown, the image cutout unit 37 compares the tilt angle of the captured image estimated by the tilt correction angle setting unit 35 with the reference rotation angle when cutting out the image in the cutout area 42, and takes a picture. When the image inclination correction angle exceeds the reference rotation angle, a series of image cutout processes can be stopped. In other words, when a rotation larger than the reference rotation angle is performed in the tilt correction of the captured image, the cutout area 42 set based on the reference rotation angle θs does not fit in the captured image after the tilt correction. In this case, by stopping the image cutting process, it is possible to prevent unnatural areas from occurring at the four corners of the cut image. As another method, when the tilt correction angle of the photographed image exceeds the reference rotation angle θs, the cropping area setting unit 31 sets the crop size of ST203 to the size of the image area 40 of the photographed image before tilt correction. Configuration is also possible. Further, when the tilt correction angle of the photographed image exceeds the reference rotation angle θs, the image of the cutout area 42 corresponding to the reference rotation angle θs may be cut out after being rotated by the reference rotation angle θs. . As a result, a certain degree of tilt angle correction can be performed, and unnatural areas can be prevented from occurring at the four corners of the image.

  Although the present invention has been described based on specific embodiments, these embodiments are merely examples, and the present invention is not limited to these embodiments. Note that not all of the components of the image processing apparatus according to the present invention and the document reading system including the image processing apparatus according to the present invention shown in the above-described embodiments are necessarily selected as long as they do not depart from the scope of the present invention. It is possible.

  The image processing apparatus according to the present invention and the document reading system provided with the image processing apparatus, even when displaying a photographic image subjected to tilt correction on a display or the like, lack of photographic image data at the four corners of the rectangular image display area. An image processing apparatus that can prevent an unnatural area caused by the occurrence of the image and that can efficiently use captured image data and processes an image obtained by reading a document such as a book, a document reading system including the image processing apparatus, and the like Useful as.

1 Document Reading System 2 Document Camera (Image Input Device)
3 PC (image processing device)
26 Display unit 27 Display data generation unit 28 Data storage unit 30 Image tilt correction unit 31 Cutout area setting unit 35 Tilt correction angle setting unit 36 Image rotation unit 37 Image cutout unit 40 Image area 41 of the pre-correction shot image 41 Reference correction shot image Image area 42 Cutout area 52 Image display area G Centric center θs Reference rotation angle

Claims (7)

An image rotation unit that corrects the inclination of the photographed image by rotating a rectangular photographed image obtained by photographing the paper surface of the document around the centroid;
A cutout that sets a rectangular cutout area that is located within the image area of the pre-correction photographed image before the tilt correction and that is located within the image area of the reference correction photographed image obtained by rotating the photographed image by a predetermined reference rotation angle. An area setting section;
An image cutout unit that cuts out an image of an area corresponding to the cutout area from the captured image that has been tilt-corrected by the image rotation unit;
An image processing apparatus comprising: a display unit configured to display a captured image in a rectangular image display area based on display data generated from an image cut out by the image cutout unit.   The image processing apparatus according to claim 1, wherein the cut-out area has a shape obtained by reducing the image area of the pre-correction captured image with the same aspect ratio.   3. The image processing apparatus according to claim 1, wherein in the cut-out area, two vertices forming a pair of vertices are positioned on a side of the image area of the reference correction photographed image. A tilt correction angle setting unit for setting a tilt correction angle of the captured image;
The image processing apparatus according to claim 1, wherein the image cutout unit does not cut out the image when the tilt correction angle is larger than the reference rotation angle.   The image processing apparatus according to claim 1, wherein the display unit includes an image display area corresponding to the cut-out area.   6. A document reading system comprising: the image processing apparatus according to claim 1; and an image input apparatus having a camera unit that generates the captured image.   The document reading system according to claim 6, wherein the camera unit is capable of changing a shooting range.

Images