JP4952380B2 - Imaging apparatus and computer program therefor - Google Patents

Description

  The present invention relates to a photographing apparatus for photographing a document placed on a document table together with the document table and projecting a document image on a screen using a projector, and a computer program therefor.

  2. Description of the Related Art Conventionally, there has been known a photographing apparatus that photographs a document placed on a document table together with the document table and projects a document image on a screen using a projector. In such a photographing apparatus, when a document is placed obliquely on a document table, the document image is corrected so that an image of the document viewed from the front (hereinafter referred to as a corrected image) is obtained. Some have a function (see Patent Document 1). Specifically, this photographing apparatus first extracts the contour of the document image on the assumption that the document shape is rectangular and the information on the document shape can be acquired, and the contour of the document image and a predefined correction image The parameters for the affine transformation are calculated from the correspondence between the shapes of the two.

Next, the imaging device calculates a parameter for affine inverse transformation from the parameters for affine transformation, and affine inversely transforms the coordinates of each pixel constituting the correction image using the parameters for affine inverse transformation. To which position of the original image corresponds to each pixel that constitutes. Then, the photographing apparatus interpolates the value of each pixel constituting the corrected image from the value of each pixel constituting the document image based on the positional relationship between the pixel of the corrected image and the pixel of the document image, and the pixel subjected to the interpolation calculation The value is displayed and output as image data of a corrected image.
JP 2006-115334 A

  The coordinate value (integer value) of each pixel constituting the corrected image becomes a coordinate value (non-integer value) between each pixel constituting the original image because the original image is inclined (FIG. 4A). (See (b)). For this reason, the conventional photographing apparatus performs the interpolation calculation by the floating point calculation, and it is difficult to perform the interpolation calculation by an inexpensive processor that does not have a dedicated device for performing the floating point calculation.

  In order to solve such a problem, a method of performing an interpolation operation by a fixed-point operation can be considered. However, when performing an interpolation operation by fixed-point arithmetic, it is necessary to increase the number of digits to the lower side (bit shift) by an amount corresponding to the resolution for interpolation. Interpolation may fail due to exceeding the number of digits of the calculator.

  Further, when various processes are performed in order to compensate for the failure of the interpolation calculation, the processing speed decreases. Furthermore, when the number of digits of the bit shift is reduced in order to avoid the failure of the interpolation calculation, the interpolation accuracy (interpolation resolution) is lowered and the image quality of the corrected image is impaired.

  The present invention has been made in order to solve the above-described problems, and an object of the present invention is to obtain a maximum interpolation resolution within a range that does not reduce the processing speed when performing an interpolation operation by a fixed-point operation. It is to provide an apparatus and a computer program thereof.

A photographing apparatus according to a first aspect of the present invention acquires a document image corresponding to the document from a photographing unit that photographs a document, and a photographed image photographed by the photographing unit, and affine transforms the shape of the document image It is corrected to generate a corrected image by the image processing unit for interpolation calculation from the values of each pixel value of each pixel constituting the corrected image by inverse transformation of the affine transformation constituting the original image, the image processing unit and an output unit for outputting the corrected image according to interpolation calculation pixel value by the image processing unit includes a predetermined number of digits divider, when performing the interpolation operation, the digit of the divider The value obtained by subtracting the number of digits obtained from the maximum coordinate value of the corrected image when performing the affine inverse transformation from the number is used as the number of shift digits, and the number of digits of the fixed-point part of the affine inverse transformation by the amount of the shift digits Shift Determines the resolution of the interpolation calculation accuracy is improved enough value of the shift digits is large.

The computer program of the photographing apparatus according to the first aspect of the present invention includes a process for photographing a document, a process for obtaining a document image corresponding to the document from the photographed photographed image, and an affine transformation of the shape of the document image The process of generating a corrected image by correcting by the amount of shift digits calculated by subtracting the number of digits determined by the maximum coordinate value of the corrected image at the time of affine inverse transformation from the number of digits of the divider wherein by shifting the digits of the fixed-point part of the affine inverse transformation, a process of determining the resolution of the interpolation computation accuracy enough value of the shift digits is large is improved, by the affine inverse transformation at the determined resolution a process of interpolation calculation from the values of each pixel constituting the value an original image of the pixels constituting the corrected image, and processing for outputting the corrected image according to interpolation calculation pixel value computing To be executed by the data.

  According to the imaging device and the computer program thereof according to the present invention, when performing interpolation calculation by fixed-point calculation, the difference between the number of digits of the divider and the number of digits required for the inverse conversion of the conversion, or the output unit Since the interpolation resolution is determined according to the output image size, the maximum interpolation resolution can be obtained within a range in which the processing speed is not reduced.

  Hereinafter, the configuration of the photographing apparatus according to an embodiment of the present invention will be described in detail.

[Configuration of the imaging device]
First, the configuration of a photographing apparatus according to an embodiment of the present invention will be described with reference to FIGS. 1 to 3.

  As shown in FIG. 1, a photographing apparatus according to an embodiment of the present invention includes a document camera 1 and a projector 3 electrically connected to the document camera 1 through a video image cable 2 as main components. Prepare. The document camera 1 is a camera system for photographing an object to be photographed, and includes a camera unit 4, a column 5 to which the camera unit 4 is attached, and a document table 7 on which a document 6 is placed. The projector 3 converts the RGB signal supplied from the camera unit 4 into projection light, and irradiates the screen 8 with the projection light, whereby an image (output image) to be photographed by the document camera 1 is displayed on the screen 8. To form an image.

  As shown in FIG. 2, the camera unit 4 includes an image data generation unit 11 and a data processing unit 12. The image data generation unit 11 includes an optical lens device 13 and an image sensor 14 and captures image data to be photographed. The optical lens device 13 includes a lens that collects light for photographing the document 6, and includes peripheral circuits for adjusting camera setting parameters such as focus, exposure, and white balance. The image sensor 14 is constituted by a CCD or the like, and digitizes an image formed by the optical lens device 13 condensing light and takes it as image data.

  The image data generation unit 11 is configured to be able to capture a high resolution image and a low resolution image. The low-resolution image shooting is, for example, shooting with an image resolution of about XGA (1024 × 768 dots). According to the low-resolution image shooting, although the resolution is low, moving image shooting and image reading can be performed at a speed of 30 fps (frame / second). On the other hand, high-resolution image capturing is image capturing with a maximum number of pixels that can be captured, for example, 4 million pixels (2304 × 1728). According to high-resolution image capturing, although reading of image data is delayed, an image with a higher resolution can be obtained as compared with low-resolution image capturing.

  The data processing unit 12 includes a memory 15, a video output device 16, an image processing device 17, an operation unit 18, a program code storage device 19, and a CPU 20, and outputs image data acquired from the image data generation unit 11 to the projector 3. Image processing is performed. As shown in FIG. 3, the memory 15 includes sensor image storage areas 21a and 21b, a processed image storage area 22, a display image storage area 23, a work data storage area 24, and a threshold storage area 25. Image data, various flag values, threshold values, etc. are stored.

  The sensor image storage areas 21a and 21b are areas for temporarily storing the image data captured by the image sensor 14 alternately every time the image data is captured. The image sensor 14 alternately stores image data in the sensor image storage areas 21a and 21b, and the image processing device 17 and the CPU 20 alternately read the temporarily stored image data from the sensor image storage areas 21a and 21b. The processed image storage area 22 is an area for the image processing apparatus 17 to write image data necessary for processing. The work data storage area 24 is an area for storing coordinate data and various flags. The threshold storage area 25 is an area for storing a threshold used for various determinations.

  The video output device 16 generates an RGB signal based on the image data stored in the display image storage area 23 of the memory 15 and outputs the generated RGB signal to the projector 3. The image processing device 17 is controlled by the CPU 20 to perform image processing on the image data temporarily stored in the sensor image storage areas 21 a and 21 b of the memory 15. Although not shown, the operation unit 18 includes a power switch, an enlargement key, a reduction key, a left movement key, a right movement key, an up movement key, a down movement key, a correction valid switch, and the like, and acquires user operation information. The program code storage device 19 is configured by a storage device such as a ROM, and stores a computer program executed by the CPU 20. The CPU 20 includes a H-number divider and controls each unit according to a computer program stored in the program code storage device 19.

[Correction process]
In the photographing apparatus having such a configuration, when the document 6 is placed obliquely on the document table 7, specifically, when the correction valid switch of the operation unit 18 is set to be effective, the following correction is performed. By executing the processing, image processing is performed on the document image so that the output image is an image obtained when the document 6 is viewed from the front (hereinafter referred to as a corrected image). Hereinafter, with reference to the flowchart shown in FIG. 5, the operation of the photographing apparatus when executing the correction process will be described.

  The flowchart shown in FIG. 5 starts at the timing when the user instructs to start photographing the original image by operating the operation unit 18, and the correction process proceeds to step S1. The operation of each unit described below is realized by the CPU 20 reading a computer program stored in the program code storage device 19 and controlling each unit according to the computer program.

  In the process of step S <b> 1, the CPU 20 detects the projector 3 connected to the camera unit 4 via the video image cable 2. Thereby, the process of step S1 is completed and a correction process progresses to the process of step S2.

  In the process of step S2, the CPU 20 detects the size (for example, VGA, XGA, UXGA, etc.) of the output image set in the projector 3 detected by the process of step S1. Thereby, the process of step S2 is completed and the correction process proceeds to the process of step S3.

  In step S3, the CPU 20 controls the optical lens device 13 and the image sensor 14 to photograph the image of the document 6 placed on the document table 7 together with the document table 6, and the captured image data. Are stored in the sensor image storage area 21a or the sensor image storage area 21b. As shown in FIG. 6, the photographed image R obtained by photographing the image of the document 6 placed on the document table 7 includes a document image region R1 corresponding to the document 6 and the surface of the document table 7. Is included (an area other than the document image area R1). Thereby, the process of step S3 is completed and a correction process progresses to the process of step S4.

  In the process of step S4, the image processing device 17 reads the captured image data stored in the sensor image storage area 21a or the sensor image storage area 21b into the processed image storage area 22. Then, the image processing device 17 uses a known edge detection technique to extract an edge image E corresponding to the contour of the document image region R1 as shown in FIG. 7, and the extracted edge image E data is used as work data. Store in the storage area 24. The edge detection technique is already known at the time of filing of the present invention, and thus detailed description thereof is omitted. For example, an edge detection filter called a Roberts filter may be used. Thereby, the process of step S4 is completed and the correction process proceeds to the process of step S5.

In the process of step S5, the CPU 20 uses the correspondence relationship between the data of the edge image E stored in the work data storage area 24 and the shape of the output image (corrected image) of the projector 3 as shown in the following formula 1. Such an affine transformation parameter (hereinafter referred to as affine transformation) for affine transformation of an original image (XYZ coordinate system, coordinate values (x, y, z)) into a corrected image (UVW coordinate system, coordinate values (u, v, 1)). (Denoted as a coefficient) A is calculated, and the data of the calculated affine coefficient A is stored in the work data storage area 24. In this embodiment, the coordinate value of the corrected image is normalized by the w coordinate value. Since the calculation method of the affine coefficient A is a well-known technique at the time of the filing of the present invention, detailed description thereof will be omitted. Proceeds to the process.

  In the process of step S <b> 6, the CPU 20 calculates the aspect ratio (y / x) of the document image using the edge image E data stored in the work data storage area 24. Then, the CPU 20 refers to the aspect ratio of the document image and the size of the output image detected by the process in step S2, and the maximum coordinate value (um, vm) of the corrected image obtained by affine transformation of the document image (FIG. 8). ). Thereby, the process of step S6 is completed and a correction process progresses to the process of step S7.

In the process of step S7, the CPU 20 reads the data of the affine coefficient A from the work data storage area 24, and uses the read affine coefficient A as a correction image as shown in the following equations 2 and 3 as a document image. An affine inverse transformation parameter A- ¹ for affine inverse transformation is calculated. Then, the CPU 20 calculates the number of digits (Nx, Ny) of the maximum coordinate value (um, vm) when the affine inverse transformation represented by the following formula 4 is performed using the calculated affine inverse transformation parameter A- ^1. calculate. Thereby, the process of step S7 is completed, and the correction process proceeds to the process of step S8.

In the process of step S8, the CPU 20 substitutes the number of digits (Nx, Ny) calculated in the process of step S7 into the following formula 5 to thereby shift the number of shift digits P of the fixed-point part when performing the affine inverse transformation. As described above, a difference value between the maximum value Max [Nx, Ny] of the number of digits H of the divider and the number of digits (Nx, Ny) calculated by the process of step S7 is calculated. Thereby, the process of step S8 is completed, and the correction process proceeds to the process of step S9.

In the process of step S 9, CPU 20, as shown in Equation 6 below, shifts the digits of the fixed-point portion of the shift digits P only affine inverse transform calculated by the processing in step S8 (see Equation 4) . This shift digit number P corresponds to the interpolation resolution, and the larger the value, the higher the accuracy of the interpolation process and the higher the quality of the corrected image. Thereby, the process of step S9 is completed and the correction process proceeds to the process of step S10.

  In the process of step S10, the CPU 20 calculates the position of each pixel constituting the corrected image corresponding to the position of the original image using Equation 6 and configures the original image with the value of each pixel constituting the corrected image. Interpolation is performed from the value of each pixel. Then, the CPU 20 stores the value of each pixel of the corrected image subjected to the interpolation calculation in the display image storage area 23. Thereby, the process of step S10 is completed, and the correction process proceeds to the process of step S11.

  In the process of step S <b> 11, the video output device 16 generates an RGB signal according to the pixel value of the corrected image stored in the display image storage area 23, and outputs the generated RGB signal to the projector 3. The projector 3 converts the RGB signals into projection light and irradiates the projection light on the screen 8 to form a corrected image on the screen 8. Thereby, the process of step S11 is completed and a correction process progresses to the process of step S12.

  In the process of step S <b> 12, the CPU 20 determines whether or not the user has instructed the end of photographing of the document image by operating the operation unit 18. As a result of the determination, when the user instructs the end of photographing of the document image, the CPU 20 ends a series of correction processes. On the other hand, when the user has not instructed the end of photographing of the document image, the CPU 20 returns the correction process to the process of step S3. The CPU 20 controls each process so that the processes in steps S3 to S12 are repeatedly executed at a constant frame rate.

  As is clear from the above description, in the imaging apparatus according to an embodiment of the present invention, when performing the interpolation calculation, the CPU 20 performs the affine inverse conversion with the number of digits H of the divider. The difference value P of the number of digits (Nx, Ny) of the maximum coordinate value (um, vm) is calculated, and the number of digits in the fixed-point part of the affine inverse transformation is shifted by the calculated difference value P. That is, in the photographing apparatus according to the embodiment of the present invention, the CPU 20 shifts the number of digits of the fixed-point part of the affine inverse transformation within a range not exceeding the number of digits H of the divider.

  According to such a configuration of the photographing apparatus, since the number of digits in the affine inverse transformation does not exceed the number of digits of the divider, the interpolation calculation can be prevented from failing. In addition, since it is not necessary to perform various processes to compensate for the failure of the interpolation calculation, it is possible to suppress a decrease in processing speed. In addition, the maximum interpolation resolution can be obtained within a range that does not reduce the processing speed. It is possible to prevent the interpolation resolution from deteriorating and the image quality of the corrected image from being impaired.

  As mentioned above, although the embodiment to which the invention made by the present inventor is applied has been described, the present invention is not limited by the description and the drawings that form part of the disclosure of the present invention according to this embodiment. As described above, it is a matter of course that all other embodiments, examples, operation techniques, and the like made by those skilled in the art based on the above embodiments are included in the scope of the present invention.

  The present invention can be applied to a photographing apparatus that photographs a document placed on a document table with a camera and projects the photographed document image on a screen using a projector.

It is a schematic diagram which shows the structure of the imaging device which becomes one Embodiment of this invention. It is a block diagram which shows the internal structure of the document camera shown in FIG. FIG. 2 is a block diagram showing an internal configuration of a memory shown in FIG. 1. FIG. 6 is a schematic diagram illustrating a positional relationship between a pixel of a corrected image and a pixel of an original image when affine inverse transformation is performed on the corrected image. It is a flowchart figure which shows the flow of the correction process used as one Embodiment of this invention. It is a schematic diagram which shows an example of the image image | photographed with a document camera. It is a schematic diagram which shows an example of the edge image extracted from the picked-up image shown in FIG. FIG. 5 is a schematic diagram illustrating a correspondence relationship between a document image and a corrected image for explaining affine inverse transformation.

Explanation of symbols

1: Document Camera 2: Video Video Cable 3: Projector 4: Camera Unit 5: Column 6: Document 7: Document Table 8: Screen 11: Image Data Generation Unit 12: Data Processing Unit 13: Optical Lens Device 14: Image Sensor 15 : Memory 16: Video output device 17: Image processing device 18: Operation unit 19: Program code storage device 20: CPU
21a, 21b: Sensor image storage area 22: Processed image storage area 23: Display image storage area 24: Work data storage area 25: Threshold storage area

Claims (4)

A shooting section for shooting a document;
A document image corresponding to the document is acquired from a photographed image photographed by the photographing unit, a corrected image is generated by correcting the shape of the document image by affine transformation, and a corrected image is formed by inverse transformation of the affine transformation. an image processing unit for interpolating calculating the value of each pixel from the value of each pixel constituting the original image,
An output unit that outputs a corrected image according to the pixel value interpolated by the image processing unit,
Wherein the image processing unit includes a predetermined number of digits divider, when performing the interpolation operation, determined by the maximum coordinate value of the correction image when the number of digits of the divider was carried out the affine inverse transformation the value obtained by subtracting the number of digits to the shift number of digits is, the shifted digits of minute shifts the digits of the fixed-point part of the affine inverse transformation, the shift digits of value interpolation accuracy is improved enough large An imaging apparatus characterized by determining a resolution of calculation.   The photographing apparatus according to claim 1, wherein the maximum coordinate value of the corrected image is calculated based on an output image size of the output unit. The process of shooting the document,
Processing for obtaining a document image corresponding to the document from the photographed image;
Processing to generate a corrected image by correcting the shape of the original image by affine transformation;
The number of digits of the fixed-point part of the affine inverse transformation is calculated by subtracting the number of digits calculated from the maximum coordinate value of the corrected image at the time of affine inverse transformation from the number of digits of the divider. shifted, the process of determining the resolution of the interpolation computation to improve the accuracy enough value of the shift digits is large,
A process of interpolating the value of each pixel constituting the corrected image by the affine inverse transform with the determined resolution from the value of each pixel constituting the document image;
Imaging device computer program, characterized in that to execute a process of outputting the corrected image according to interpolation calculation pixel values to the computer. The computer program of the projection apparatus according to claim 3 , wherein the computer program of the projection apparatus further causes the computer to execute a process of calculating a maximum coordinate value of the corrected image based on an output image size of the output unit. program.

Images