JP2016129417A - Image reading device, image reading method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To appropriately preserve photographed images for each page only by turning pages without performing a photographing instruction operation and adjusting photographing possible ranges for each page when a paper medium including a plurality of pages is to be photographed.SOLUTION: When a new photographed image is stored in an A buffer as a photographed image for this time whenever a paper medium is photographed, and when an operation to store the image which is stored as the photographed image for this time till then in a B buffer as a previous photographed image is performed as shown in figures 7(1) to (3), it is discriminated whether a system is in a page turning state or not by analyzing the photographed image for this time. If the system is in the page turning state (refer to figures 7(4) and (5)), the previous photographed image stored in the B buffer 3d is determined as a preservation object (refer to figure 7(6)).SELECTED DRAWING: Figure 7

Description

  The present invention relates to an image reading apparatus and a program including an imaging unit that captures information described in a paper medium (for example, a notebook or a book) composed of a plurality of pages.

  In general, an image reading apparatus that reads information written on a paper medium such as a notebook is scanned as image data by scanning the paper medium with a scanner, or read as a captured image by photographing with a digital camera. Like to do. Here, when reading with a scanner, the paper medium needs to be one sheet (sheet-like) instead of a booklet state (bookbinding state), and when reading with a digital camera, the paper medium is Although it may be in a state, it was necessary for the photographer to shoot while turning pages one page at a time.

  As a technique for shooting while turning pages in this way, conventionally, for example, after detecting that a page has been turned, the characters (number of pages) in the image taken by the imaging unit are recognized and saved. A technique is known in which a captured image is recorded on a condition that the image and the captured image are not the same number of pages (see Patent Document 1). In this technology, when the photographer is holding a book and flipping the page using gravity, the horizontal line in the image (the edge of the page) has been lowered and stabilized. The end of page turning is detected on the basis of whether or not.

JP 2012-65261 A

  However, in the technique of Patent Document 1 described above, if the stored image and the current photographed image have the same number of pages, the photographed image is not actually recorded. If you take a picture of a page while filling in a notebook with characters and charts, even if new letters and charts are entered, the number of pages of the saved image and the current shot image will be the same. Therefore, the current captured image is ignored without being recorded. This is not a problem limited to the case where the paper medium is a notebook, but the same problem occurs even when a page is shot while being handwritten in the book.

  The subject of the present invention is that when shooting a paper medium consisting of a plurality of pages, the photographed image is displayed for each page simply by turning the page without performing the shooting instruction operation for each page or adjusting the shooting range. It is to be able to save appropriately.

In order to solve the above-described problems, an image reading apparatus according to the present invention includes:
An image reading apparatus provided with an imaging unit that captures information described on a paper medium composed of a plurality of pages,
Each time a paper medium is photographed by the imaging means, the new photographed image is stored as the current photographed image, and the image stored as the current photographed image until then is stored as the previous photographed image. When,
A turning determination means for determining whether or not it is a page turning state by analyzing the current captured image stored in the temporary storage means;
A determination unit that determines a previous captured image stored in the temporary storage unit as a storage target when it is determined by the page determination unit that the page is being turned;
An image reading apparatus comprising:

In order to solve the above-described problems, the program of the present invention
Against the computer,
Each time the paper medium is photographed by the imaging means for photographing information described on the paper medium composed of a plurality of pages, the new photographed image is temporarily stored as the current photographed image, and as the current photographed image until then. In a state where the stored image is temporarily stored as the previous captured image, a function of determining whether or not the page is turned by analyzing the temporarily stored current captured image;
A function for determining the temporarily stored previous captured image as a storage target when it is determined that the page is turned;
It is a program for realizing.

  According to the present invention, when a paper medium consisting of a plurality of pages is photographed, the photographed image is displayed for each page simply by turning the page without performing a photographing instruction operation for each page or adjusting the photographing range. Therefore, it is possible to store them appropriately, and it is rich in convenience and certainty.

The block diagram which showed the basic component of the tablet terminal device provided with the imaging function (digital camera) applied as an image reader. (1) is a state where the tablet terminal device 11 is protected by the main body cover 12, (2) is a state where the tablet terminal device 11 is erected using the main body cover 12, and (3) is erected. The figure which showed the state which looked at the tablet terminal device 11 from the front direction. The relationship between the angle of view of the imaging unit 6 and the paper medium 13 when a paper medium (notebook) 13 to be read is placed at the foot of the tablet terminal device 11 in an upright state and the paper medium is photographed will be described. Figure to do. (1)-(3) is a figure for demonstrating the mark M attached to the predetermined position of the right-and-left page in the state which opened the paper medium 13. FIG. 7 is a flowchart that starts execution when an image reading function (note reading function) is designated by the image reading key 4b. 6 is a flowchart following the operation of FIG. (1)-(10) is the figure which illustrated the contents of A buffer 3c and B buffer 3d which change according to passage of time. The figure for demonstrating the characteristic of 2nd Embodiment. 6 is a flowchart showing an operation following the flow of FIG. 5 in the second embodiment.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
(Embodiment 1)
First, a first embodiment of the present invention will be described with reference to FIGS.
This embodiment exemplifies a case where the present invention is applied to a tablet terminal device having an imaging function (digital camera) as an image reading device, and FIG. 1 shows basic components of the tablet terminal device with a camera. It is a block diagram.
The tablet terminal device, for example, is a portable information terminal device of A5 size as a whole, and will be described in detail later in addition to basic functions such as a touch input function and a wireless communication function. It is equipped with an image reading function (note reading function) that reads the information (characters, charts, etc.) written in the notebook by taking a picture of the printed paper medium (the notebook installed in a double-page spread) It is the structure which makes the part 1 a core.

  The control unit 1 operates by supplying power from the power source unit (secondary battery) 2 and controls the overall operation of the tablet terminal device according to various programs in the storage unit 3. Are provided with a CPU (Central Processing Unit) and a memory (not shown). The storage unit 3 includes, for example, a ROM, a flash memory, and the like, and stores programs and various applications for realizing the present embodiment in accordance with operation procedures shown in FIGS. 5 and 6 to be described later. A program memory 3a, a work memory 3b for temporarily storing various information (for example, flags and the like) necessary for the operation of the tablet terminal device, and an A buffer 3c and a B buffer 3d described later. Have. The storage unit 3 may include a removable portable memory (recording medium) such as an SD card or an IC card, and is connected to a network via a communication function (not shown). The state may include a storage area on a predetermined server device side.

  The operation unit 4 includes, as a push button type key, a power key 4a for turning on / off the power, an image reading key 4b for starting and ending an image reading function (note reading function), and the like. Then, processing corresponding to the input operation signal output from the operation unit 4 in response to the operation key is performed. The touch display unit 5 has a configuration in which a touch panel 5b is stacked on a display panel 5a. The display panel 5a is a high-definition liquid crystal display or an organic EL display having screens with different aspect ratios (for example, horizontal 4: vertical 3). Electro Luminescence) display. The touch panel 5b constitutes a touch screen that detects a position touched by a photographer's finger and inputs coordinate data, and employs, for example, a capacitance method or a resistance film method. Other methods may be used.

  Although the imaging unit 6 constitutes an image reading function (note reading function) and is not shown in the drawing, the subject image from the optical lens is imaged on an imaging element (CCD, CMOS, etc.), so that the subject is high-definition. The camera unit is capable of photographing, and has a photographing lens, an image sensor, various sensors, an analog processing unit, and a digital processing unit. The imaging unit 6 is capable of continuous shooting (high-speed shooting) with a frame rate of 15 frames per second (15 fps), for example, and the control unit 1 is newly shot during the operation of the image reading function (note reading function). The captured image for one frame is temporarily stored in the A buffer 3c as the current captured image, and the previously stored captured image in the A buffer 3c is temporarily transferred to the B buffer 3d as the previous captured image. I try to remember it.

  A wireless LAN (Local Area Network) communication unit 7 is a wireless communication module capable of high-speed and large-capacity communication, and can be connected to the Internet via a nearby wireless LAN router (not shown). The acceleration sensor 8 is, for example, a three-axis type acceleration sensor, which is an acceleration component in three axis directions (X, Y, Z directions) orthogonal to each other, that is, a voltage value proportional to the magnitude of acceleration applied to the three axes. Is an attitude sensor that detects the attitude (tilt and orientation) of the housing (apparatus body) according to the detected value. During operation of the image reading function (note reading function), the control unit 1 refers to the detection result by the acceleration sensor 8 and determines whether the casing (device main body) is in a posture suitable for the image reading function (note reading function). Is determined.

  2A is a diagram showing a state in which the tablet terminal device 11 is protected by the main body cover 12, and FIG. 2B is a state in which the tablet terminal device 11 is stood up obliquely using the main body cover 12. FIG. FIG. That is, the entire casing of the tablet terminal device 11 is a thin rectangular parallelepiped, and FIG. 2 (2) shows that the casing is in a landscape orientation, and its short side direction is slightly laid down (tilt backward). ) It is the side view which showed the state which stood up and looked at the standing state from the one side direction. The main body cover 12 is configured to be bendable at a predetermined position (for example, two locations). That is, as shown in FIG. 2 (2), the main body cover 12 can be bent at two places, the apex portion and the back center portion, and the tablet terminal in a state where it is raised by being bent at those portions. The apparatus 11 can be supported from behind.

  FIG. 2 (3) is a schematic perspective view of the tablet terminal device 11 in the landscape orientation as shown in FIG. 2 (2), with the short side direction slightly laid and viewed from the front direction. It is. As shown in FIG. 2 (3), in a state where the tablet terminal device 11 is in the landscape orientation and is erected, the touch display unit 5 is disposed in substantially the entire region on the front side of the casing. In addition, an imaging unit (digital camera) 6 is disposed at the center of the upper end of the front side of the housing. Although not shown, marks or lines for guiding the position of the paper medium may be printed on a part of the main body cover 12.

FIG. 3 shows a case in which a paper medium (notebook) 13 is arranged as a reading object at the foot (near the lower side) in a standing state in which the tablet terminal device 11 is oriented horizontally and its short side direction is slightly laid down. 4 is a diagram for explaining a relationship between an angle of view of the imaging unit 6 and a paper medium 13 when photographing the paper medium. FIG.
Although the angle of view of the imaging unit 6 depends on the size of the paper medium 13, an area corresponding to the size of the paper medium 13 is an imageable range, and the paper medium 13 is left and right at the foot of the tablet terminal device 11. When the pages are arranged in a state where the pages are opened, the left and right pages in the opened state are the shooting range. The imaging unit 6 can adjust the angle of view, exposure, white balance, and the like according to the type (size, paper quality, color, etc.) of the paper medium 13.

FIG. 4 is a diagram for explaining a mark M added to a predetermined position on the left and right pages in a state where the paper medium (notebook) 13 is opened.
A predetermined mark M is added to a predetermined position (the lower corner portion of the page) of each page constituting the paper medium 13. That is, in the state in which the paper medium 13 is opened, the mark M having the same content is added to the lower left corner of the left page and the lower right corner of the right page, but the left and right pages of the other spreads have different contents. A mark M is added. For example, a circular mark M is added to the left and right spread pages (n page, n + 1 page) shown in FIG. 4 (1), and the next spread left and right pages (n + 2 page, n) as shown in FIG. 4 (2). A diamond-shaped mark M is added to the (n + 3 page), and a cross-shaped mark M is added to the next spread left and right pages (page n + 4, page n + 5) as shown in FIG. 4 (3). .

  Similarly, on the left and right pages of the spread that follows, in the order of the circle, diamond, and cross mark M, that is, the spread pages that follow the circle mark are the diamond marks and the spread pages that follow the diamond mark. The circular mark, the diamond-shaped mark, and the cross-shaped mark M are cyclically added to the continuous spread pages, such as a cross mark, a circular mark on the next spread page of the cross mark, and so on. The means for adding the mark M to the paper medium 13 is not limited to printing the mark M in advance, and the user may attach a mark-attached seal to the paper medium 13 as necessary. In the figure, in order to clearly show the mark M, it is expressed larger than the paper medium 13, but in actuality, a mark having a small size is used so that the mark M does not stand out.

  When the mark M is added to each page of the paper medium 13 as described above, when the left and right pages are photographed so that the information of two pages that open the paper medium is within the photographing range of the imaging unit 6. The control unit 1 recognizes the mark M appended to each page opened by analyzing the current photographed image taken, and the page turning state based on the relevance of the recognized mark M It is determined whether or not it is during page turning. That is, if the mark M on the left and right pages is the same type and the same shape, it is determined that the page is not turned, but the type of the mark M on the left and right pages is different or the type is the same. However, when the shapes are different, it is determined that the page is being turned (during page turning).

  In other words, turning the page makes the mark on the page appear deformed or disappears, or the mark on the back page appears, but the way the mark looks changes by turning the page in this way, By using the appearance of a mark, the control unit 1 determines whether or not a page has been turned by detecting the state. For example, even if the mark is a circular mark, if the mark on the page is deformed or part of it is missing by turning the page, it is determined that the mark M of the same type is not the mark of the same shape. In addition, even when the mark M cannot be suddenly recognized or overlapped with a part of the mark on the rear page, it is determined that the marks do not have the same shape.

  In the first embodiment, it is determined whether the page is turned depending on whether the marks M on the left and right pages of the spread are the same type and the same shape. It may be determined based on only the type of M, that is, whether the page is turned according to whether the marks M on the left and right pages of the spread are of the same type. As a result of such determination, when the control unit 1 determines that the page is turned, the previous captured image temporarily stored in the B buffer 3d is determined as an image to be saved and stored. The data is recorded and saved on the SD card of section 3.

Next, the operation concept of the tablet terminal device 11 in the first embodiment will be described with reference to the flowcharts shown in FIGS. Here, each function described in these flowcharts is stored in the form of a readable program code, and operations according to the program code are sequentially executed. Further, it is possible to sequentially execute the operation according to the above-described program code transmitted via a transmission medium such as a network. The same applies to other embodiments described later. In addition to the recording medium, an operation peculiar to the present embodiment can be executed using a program / data supplied externally via a transmission medium.
5 and 6 are flowcharts showing an outline of the operation of the characteristic part of the present embodiment in the overall operation of the tablet terminal device 11. When the flow of FIG. 5 and FIG. Return to the main flow (not shown) of the overall operation.

5 and 6 are flowcharts that are started when an image reading function (note reading function) is designated by the image reading key 4b.
Hereinafter, the operation of the image reading function (note reading function) will be specifically described with reference to FIGS. FIGS. 7 (1) to 7 (10) are diagrams illustrating the contents of the A buffer 3c and the B buffer 3d that change with the passage of time, and the orientation of each image shows a case viewed from the user. In addition, after the user activates the image reading function (note reading function) by operating the image reading key 4b, the user sequentially turns the pages while adding the contents of the lesson to the notebook as the class progresses. I am doing so.

  First, when the image reading function is activated, the control unit 1 activates the acceleration sensor 8 to acquire the detection result (step S1 in FIG. 5), and the posture of the housing is a posture suitable for image reading. That is, it is determined whether or not the tablet terminal device 11 is in a landscape orientation and the short side direction is slightly laid down (a state suitable for image reading) (step S2). If it is not in a state suitable for image reading (NO in step S2), a message to that effect is sent to the touch display unit 5 to inform the user that the orientation of the housing should be changed to a state suitable for image reading. After displaying (step S3), the process returns to the above-described step S1, and the above-described operation is repeated until the posture is changed.

  When it is detected that the posture is suitable for image reading (YES in step S2), it is checked whether the mode is switched to the high-speed turning reading mode (high-speed reading mode) (step S4). Here, the high-speed turning reading mode refers to a case where so-called flipping is performed, that is, when a plurality of pages are turned continuously at a high speed (for example, one page / second), and the images are read at a high speed. If it is not the high-speed turning reading mode, it is determined that it is a normal reading mode other than high-speed turning. If it is the normal reading mode (NO in step S4), the shooting interval is set to be slow (for example, 50 ms) (step S6), but if it is the high-speed turning reading mode (YES in step S4), the shooting interval is set. It is set quickly (for example, 15 ms) (step S5). Then, it is checked whether or not an operation for instructing to start photographing has been performed (step S7), or whether or not an operation for instructing a change of the reading mode has been performed (step S8). Here, when an operation for instructing to change the reading mode is performed (YES in step S8), the process returns to the above-described step S4, and processing for resetting the photographing interval according to the changed reading mode is performed (step S5). , S6).

  Now, when an operation for instructing the start of photographing is performed (YES in step S7), photographing by the imaging unit 6 is started at each of the above set times (step S9). Then, the flow proceeds to the flow of FIG. 6, and after performing the initialization 0 process (step S10) for clearing the contents of the A buffer 3c and the B buffer 3d, an operation for instructing the end of the image reading function (note reading function) is performed. A check is made (step S11). If the reading end operation is not performed (NO in step S11), a captured image for one frame is acquired from the imaging unit 6 that captures images for each set time described above (step S12). The buffer 3c is temporarily stored as the current captured image (step S13).

  Then, by analyzing this photographed image, each mark M added to a predetermined position on the left and right pages is recognized (step S14), and each mark M on the left and right pages has the same type and the same shape. Whether it is a mark is checked (step S15). That is, it is checked whether the marks are of the same type (for example, a circle, a diamond shape, a cross shape, etc.), and whether the mark M of the same type is a mark of the same shape is checked. At the beginning of reading, since it is detected that the marks M on the left and right pages are of the same type and shape (YES in step S15), it is determined that the page is not turned and the next step S16 is performed. It is checked whether the captured image is temporarily stored in the B buffer 3d. However, immediately after the initialization process described above, the contents of the B buffer 3d are cleared (NO in step S16), so the next step S21 is performed. Move on. For example, as shown in FIG. 7A, the current captured image is stored in the A buffer 3c, but the captured image is not stored in the B buffer 3d, so this is detected in step S16. Then, the process proceeds to the next step S21. Here, the content of the B buffer 3d is updated by transferring the content of the A buffer 3c to the B buffer 3d. FIG. 7 (2) shows a state immediately after the captured image in the A buffer 3c is transferred into the B buffer 3d.

  Thereafter, the process returns to step S11 described above, and a captured image for one frame is acquired from the imaging unit 6 that captures images at the set time described above (step S12), and the acquired image is temporarily stored in the A buffer 3c as the current captured image. Store (step S13). FIG. 7 (3) shows the contents of the A buffer 3c and the B buffer 3d in this case. Here, on the condition that the marks M on the left and right pages in the photographed image of this time are the same type and the same shape (YES in step S15), and the image is stored in the B buffer 3d. (YES in step S16), the marks M of the image in the A buffer 3c (currently captured image) and the image in the B buffer 3d (previous captured image) are compared (step S17), and both marks M Are different (step S18). In the case of FIG. 7 (3), since both marks M are the same (NO in step S18), the content of the B buffer 3d is updated by transferring the content of the A buffer 3c to the B buffer 3d. (Step S21). Thereafter, the process returns to step S11 described above.

  Similarly, a captured image for one frame is acquired from the imaging unit 6 (step S12), and the acquired image is temporarily stored in the A buffer 3c as the current captured image (step S13). It is checked whether the marks M on the left and right pages are the same type and the same shape (steps S14 and S15). In this case, the types of the marks M on the left and right pages are different as shown in FIG. If (NO in step S18), it is determined that the page is turned. If it is determined that page turning is performed in this way, the process proceeds to the next step S22, and the contents of the A buffer 3c are cleared in order to ignore the image in the A buffer 3c (currently taken image). FIG. 7 (5) shows a case where the contents of the A buffer 3c are cleared.

  Thereafter, the process returns to step S11 described above, and similarly to the above, a captured image for one frame is acquired from the imaging unit 6 (step S12), and the acquired image is temporarily stored in the A buffer 3c as the current captured image ( Step S13). FIG. 7 (6) shows the contents of the A buffer 3c and the B buffer 3d in this case. In this case, as a result of comparing each mark M between the image in the A buffer 3c (currently captured image) and the image in the B buffer 3d (previous captured image), both marks M are different (YES in step S18). Then, the image (previous photographed image) in the B buffer 3d is determined as a storage target (step S19), and this determined image is recorded and stored in the SD card of the storage unit 3 (step S20). Then, the process proceeds to the next step S21, in which the content of the B buffer 3d is updated by transferring the content of the A buffer 3c to the B buffer 3d. FIG. 7 (7) shows the state in this case.

  7 (8), as in FIG. 7 (3), each mark M of the previous photographed image stored in the B buffer 3d is stored in each mark M of the current photographed image stored in the A buffer 3c. Is the same case. FIG. 7 (9) shows a case where it is determined that the page turning is performed with different types of the marks M on the left and right pages in the photographed image as in the case of FIG. 7 (4). is there. Thereafter, if the page turning is not detected as a result of the same operation as described above, the contents of the B buffer 3d are updated. However, when the page turning is detected, an image (previous photographed image) in the B buffer 3d is displayed. It is determined as a storage target, and this determined image is recorded and stored in the SD card of the storage unit 3.

  When an operation for instructing the end of the operation of the image reading function (note reading function) is performed after such an operation (YES in step S11), an image in the B buffer 3d (previous photographed image) Is determined as a storage target (step S23), and the determined image is recorded and stored in the SD card of the storage unit 3 (step S24). Thereafter, the flow of FIGS. 5 and 6 is exited. FIG. 7 (10) shows a case where an image reading end operation is performed.

  As described above, in the first embodiment, the control unit 1 stores the new captured image in the A buffer 3c as the current captured image every time the paper medium is captured by the imaging unit 6, and the A Whether or not it is a page turning state by analyzing the current captured image when an operation of storing the image stored as the current captured image in the buffer 3c as the previous captured image in the B buffer 3d is performed. If the page is in the page turning state, the previous photographed image stored in the B buffer 3d is determined as an object to be saved, so that when a paper medium consisting of a plurality of pages is photographed, each page Without having to perform shooting instruction operations or adjusting the shooting range, you can simply turn the page and save the shot image appropriately for each page. It becomes rich in gender and certainty.

  Further, when the paper medium 13 to be read is a notebook, the previous photographed image includes all the contents written in the notebook until immediately before turning the page. These images are not stored, and the captured image stored for each page is appropriate. In addition, because the saved image is the image immediately before turning the page, that is, to move the page writing operation away from the notebook, the fingertip turning the page is reflected in the image. Even if there is a case, the hand in the writing state holding the writing instrument does not appear in the image as it is.

  Since the control unit 1 recognizes a mark attached to each page of the paper medium 13 and determines whether the page is turned or not based on the recognition result of the mark, the page turning determination is performed. Is easy and accurate, and it is possible to prevent problems caused by wrong determination of page turning.

  When the control unit 1 recognizes a mark attached to each opened page by analyzing the captured image taken this time in a state where the information of the two pages opened in the paper medium 13 is contained. In addition, since the page turning state is determined based on the recognized relevance of each mark (whether they are the same or not), it is possible to accurately determine the page turning based on the relevance of the mark. it can.

  As the relationship between the marks attached to each open page, if the marks are the same (for example, the same type and the same shape), it is determined that the pages are not turned, and if they are different, the pages are turned. Therefore, it is possible to accurately determine the page turning depending on whether or not the marks are the same.

  When the control unit 1 determines that the page is turned, the current shot image stored in the A buffer 3c is deleted, so that the image (stored) during the page turning is saved. Can be reliably prevented from being transferred from the A buffer 3c to the B buffer 3d and stored, and only appropriate captured images taken immediately before turning the page can be stored. Become.

  Since the previous captured image in the B buffer 3d is determined as a storage target in accordance with the image reading end operation, it is possible to store up to the last page written in the notebook.

  When the posture of the apparatus main body detected by the acceleration sensor 8 is a predetermined posture, the photographing by the imaging unit 6 is performed. Therefore, the photographing is performed with the angle of view, the focus, the exposure, and the like determined according to the posture. This makes it unnecessary to adjust the angle of view, focus, exposure, etc.

  The imaging unit 6 approaches the lower side (foot) of the tablet terminal device 11 when the tablet terminal device 11 is in a landscape orientation with the short side direction slightly laid as the posture of the device body. Since the paper medium 13 arranged is photographed, the paper medium 13 can always be photographed from above, which is suitable for photographing while writing in a notebook.

  A so-called flipping high-speed turning reading mode and a normal reading mode other than high-speed turning can be arbitrarily selected, and the photographing timing of the image pickup unit 6 is controlled so as to obtain a photographing interval corresponding to the reading mode. Therefore, image reading according to the selection mode is possible, and convenience can be improved.

  In the first embodiment described above, the relevance of each mark M in the current photographed image is the same type and the same shape. However, as described above, the relevance of the mark M is the same type. You may make it discriminate | determine. Further, whether or not the same type of mark is in the page turning state may be determined based on whether or not the mark size and the mark arrangement position are the same. For example, the determination may be made based on a combination of the mark shape and the mark arrangement position.

  In the first embodiment described above, the type of the mark M is exemplified as a circle, a rhombus, and a cross. However, the type is not limited to this, and any type may be used. Further, the mark M is not limited to a figure or a symbol, and may be, for example, a character or a number, and of course, may be the number of pages (serial number) printed in a paper medium.

  In the first embodiment described above, the captured images for one frame are sequentially stored in the A buffer 3c and the B buffer 3d. However, the captured images for a plurality of consecutive frames may be temporarily stored. . In this case, for each of the A buffer 3c and the B buffer 3d, it is also possible to specify a captured image at an intermediate point from among a plurality of frames of captured images and compare the marks M.

(Second Embodiment)
A second embodiment of the present invention will be described below with reference to FIGS.
In the first embodiment described above, the mark M appended to the left and right pages with the paper medium 13 wide open is recognized, and it is determined whether the page is turned based on the recognition result of the mark M. However, in this second embodiment, the entry state of the current photographed image and the previous photographed image is recognized by recognizing the entry state in which characters, diagrams, etc. are written on the left and right pages with the paper medium 13 open. Are compared, and based on the comparison result (changed state of entry), it is determined whether or not the page is turned. Here, in both the embodiments, the same or the same names are denoted by the same reference numerals, the description thereof will be omitted, and the following description will focus on the features of the second embodiment. .

FIG. 8 is a diagram for explaining the features of the second embodiment.
In the second embodiment, when the left and right pages that open the paper medium 13 are photographed by the imaging unit 6, by analyzing the photographed image, as an entry state in which characters, diagrams, and the like are written in the notebook, for example, , The outline shape of the written part, that is, the size of the character group in the digit direction and the line direction, the outline shape according to the figure digit direction and the size in the line direction is recognized, but the entire page is recognized Instead of recognizing, only a part of the area that is highly likely to be written is recognized.

  In the example shown in the figure, when the upper left corner area (A area) and the upper left center area (B area) are set as areas with high possibility of writing in the left page, This is a case where the upper left corner area (C area) and the upper left center area (D area) are set as areas that are highly likely to be written. These areas are areas determined in advance on the manufacturer side or the like. Thus, in a state where the A to D areas are set as a plurality of areas on the left and right pages of the spread, the image captured this time is analyzed, and the entry state (for the entry portion) is written for each of the A to D areas. (Outline shape) is recognized and compared with the entry state for each of the areas A to D of the previously photographed image, it is determined for each area whether the entry states of the corresponding areas match. Then, the number of matching areas is counted, and it is determined whether or not page turning has been performed based on whether or not the number of areas is equal to or greater than a predetermined value (for example, three, that is, three of the four areas). .

FIG. 9 is a flowchart showing an operation following the flow of FIG. 5 and is a flowchart in place of FIG. 6 of the first embodiment. It should be noted that the description of the parts basically similar to those in FIG.
First, as in the first embodiment, initialization processing for clearing the A buffer 3c and the B buffer 3d is performed (step S100). If the reading end operation is not performed (NO in step S101), a captured image for one frame is acquired from the imaging unit 6 (step S102), and the acquired image is stored in the A buffer 3c as the current captured image. Temporary storage is performed (step S103).

  Next, after performing the process (step S104) of recognizing the entry state (outline shape of the entry portion) for each of the A region to D region in the left and right pages of the spread by analyzing the image taken this time, It is checked whether or not the captured image is temporarily stored in the B buffer 3d (step S105). Since the contents of the B buffer 3d are cleared immediately after the initialization process described above (NO in step S105), the process proceeds to the next step S110, and the captured image in the A buffer 3c is transferred to the B buffer 3d. Thus, the process of updating the contents of the B buffer 3d is performed. Then, in order to make the entry state for each area recognized by analyzing the current photographed image the entry state for each area with respect to the previous photographed image, the entry state corresponding to the A buffer is set as the entry state corresponding to the B buffer 3d. Processing for temporary storage is performed (step S111).

  Thereafter, the process returns to step S101 described above, and a captured image for one frame is acquired from the imaging unit 6 until a reading end instruction is issued (step S102), and the acquired image is temporarily stored in the A buffer 3c as the current captured image ( After step S103), the image captured this time is analyzed to perform processing for recognizing the entry state written for each of the areas A to D (step S104). Here, since the captured image is temporarily stored in the B buffer 3d (YES in step S105), the process proceeds to the next step S106, where the entry state corresponding to the A buffer 3c and the entry state corresponding to the B buffer 3d are stored in the area. Compare every time. Here, it is determined for each region whether the entry states of the corresponding regions match (substantially match, for example, 90% or more match), the number of matching regions is counted, and the number of regions is a predetermined value (for example, , “3”), it is determined whether or not page turning has been performed (step S107).

  As a result, if the number of matching areas is 3 or more, which is the predetermined number (YES in step S107), that is, if the contents of at least three of the four areas have not changed, page turning is not performed. Then, the process proceeds to the next step S110, where the captured image in the A buffer 3c is transferred to the B buffer 3d to update the contents of the B buffer 3d, and the entry state corresponding to the A buffer is changed to the B buffer 3d. A process of temporarily storing the corresponding entry state is performed (step S111).

  If the number of matching areas is less than the predetermined number of 3 (NO in step S107), if the contents of at least two of the four areas have changed, it is determined that page turning has been performed and step S108 is performed. Then, the image (previous photographed image) in the B buffer 3d is determined as a storage target, and this determined image is recorded and stored in the SD card of the storage unit 3 (step 109). Then, the captured image in the A buffer 3c is transferred to the B buffer 3d to update the contents of the B buffer 3d (step S110), and the entry state corresponding to the A buffer is set as the entry state corresponding to the B buffer 3d. Processing for temporary storage is performed (step S111). Then, it returns to step S101. Hereinafter, the above operation is repeated, but when an operation for instructing the end of the operation of the image reading function (note reading function) is performed (YES in step S101), the image in the B buffer 3d (previous photographed image) ) As a storage target (step S112), and the determined image is recorded and stored in the SD card of the storage unit 3 (step S113). Thereafter, the flow of FIG. 9 is exited.

  As described above, in the second embodiment, the control unit 1 compares the entry state between the current photographed image and the previous photographed image, and whether the page is turned based on the comparison result (change state of entry). Since it is determined whether or not the page is turned, it is possible to easily and reliably detect the page turning from the entry state (change state of entry) written in the notebook.

  When analyzing the current captured image and the previous captured image, the contents of multiple areas in each captured image are compared, and page turning is performed based on the number of areas in the multiple areas where the image has changed. Since it is determined whether or not it has been received, it is possible to recognize only a part of the area that is likely to be written in, instead of recognizing the entire left and right pages of the spread, and to improve the recognition efficiency Can do.

  In the second embodiment described above, the areas A to D determined in advance by the manufacturer are fixedly set as the plurality of areas in the opened left and right pages. The size of is not limited to this, and may be an area arbitrarily set by a user operation. Furthermore, where to set each area may be determined dynamically according to the writing state (writing amount) in the notebook.

  In each of the above-described embodiments, the case where the present invention is applied to the tablet terminal device 11 as an image reading device has been described. However, the present invention is not limited to this, and a personal computer with a photographing function, a PDA (personal portable information communication device), It may be a digital camera / music player, mobile phone, etc. Of course, it may be a digital camera itself.

  Further, the “apparatus” and “unit” shown in each of the above-described embodiments may be separated into a plurality of cases by function, and are not limited to a single case. In addition, each step described in the above-described flowchart is not limited to time-series processing, and a plurality of steps may be processed in parallel or separately.

The embodiment of the present invention has been described above. However, the present invention is not limited to this, and includes the invention described in the claims and the equivalent scope thereof.
Hereinafter, the invention described in the claims of the present application will be appended.
(Appendix)
(Claim 1)
The invention described in claim 1
An image reading apparatus provided with an imaging unit that captures information described on a paper medium composed of a plurality of pages,
Each time the paper medium to be read is photographed by the imaging means, the new photographed image is stored as the current photographed image, and the image previously stored as the current photographed image is stored as the previous photographed image. Temporary storage means,
A turning determination means for determining whether or not it is a page turning state by analyzing the current captured image stored in the temporary storage means;
A determination unit that determines a previous captured image stored in the temporary storage unit as a storage target when it is determined by the page determination unit that the page is being turned;
An image reading apparatus comprising:
(Claim 2)
The invention according to claim 2 is the image reading apparatus according to claim 1,
The page turning discriminating means recognizes a mark attached to each page of the paper medium, and discriminates whether or not a page turning state is based on the recognition result of the mark;
The image reading apparatus is characterized in that it is configured as described above.
(Claim 3)
The invention according to claim 3 is the image reading apparatus according to claim 2,
The page turning discriminating means is added to each page opened by analyzing the current shot image taken in a state where the information of two pages of the paper medium spread within the shooting range of the imaging means is included. When the recognized mark is recognized, it is determined whether or not the page is turned based on the relationship between the recognized marks.
An image reading apparatus characterized by being configured as described above.
(Claim 4)
According to a fourth aspect of the present invention, in the image reading apparatus according to the third aspect,
The page turning discriminating means discriminates that if the marks are the same as the remarks attached to each of the spread pages, it is not a page turning state, and if they are different, it is in a page turning state. Determine that there is,
The image reading apparatus is characterized in that it is configured as described above.
(Claim 5)
According to a fifth aspect of the present invention, in the image reading apparatus according to any one of the first to fourth aspects,
Erasing the current captured image stored in the temporary storage means when it is determined by the turning determination means that the page is turned.
The image reading apparatus is characterized in that it is configured as described above.
(Claim 6)
The invention according to claim 6 is the image reading apparatus according to any one of claims 1 to 5,
In response to an end instruction for instructing the imaging unit to end shooting of the paper medium, the determination unit determines the previous captured image stored in the temporary storage unit as a storage target.
The image reading apparatus is characterized in that it is configured as described above.
(Claim 7)
The invention according to claim 7 is the image reading apparatus according to claim 1,
The turning determination unit determines whether or not page turning has been performed from a change state of both images by analyzing the current captured image and the previous captured image.
The image reading apparatus is characterized in that it is configured as described above.
(Claim 8)
The invention according to claim 8 is the image reading apparatus according to claim 7,
The turning determination unit compares the contents of a plurality of regions in each captured image when analyzing the current captured image and the previous captured image, and the image of the plurality of regions changes. Determine whether the page was turned based on the number of areas,
The image reading apparatus is characterized in that it is configured as described above.
(Claim 9)
The invention according to claim 9 is the image reading apparatus according to any one of claims 1 to 8,
Detection means for detecting the attitude of the apparatus body;
A first shooting control unit that permits shooting by the imaging unit when the posture detected by the detection unit is a predetermined posture;
Further comprising
The image reading apparatus is characterized in that it is configured as described above.
(Claim 10)
According to a tenth aspect of the present invention, in the image reading apparatus according to the ninth aspect,
The image reading device is a mobile terminal device,
In the state where the mobile terminal device main body stands up and the imaging means is arranged on the upper side of the terminal device main body, the first photographing control means causes the mobile terminal device main body to stand up by the detection means. Permitting the photographing of the paper medium disposed close to the lower side of the apparatus body when it is detected that
The image reading apparatus is characterized in that it is configured as described above.
(Claim 11)
The invention according to claim 11 is the image reading apparatus according to any one of claims 1 to 10,
Mode specifying means for selecting and specifying an arbitrary image reading mode from among a plurality of image reading modes having different shooting intervals in accordance with the page turning speed for a paper medium consisting of a plurality of pages;
Second imaging control means for controlling the imaging timing by the imaging means so as to be an imaging interval according to the image reading mode specified by the mode specifying means;
Further comprising
The image reading apparatus is characterized in that it is configured as described above.
(Claim 12)
The invention according to claim 12
Against the computer,
Each time the paper medium is photographed by the image pickup means for photographing the information described on the paper medium with a paper medium consisting of a plurality of pages as a reading target, the new photographed image is temporarily stored as the current photographed image. In the state where the image previously stored as the current captured image is temporarily stored as the previous captured image, whether or not the page is turned by analyzing the temporarily stored current captured image A function to determine whether or not
A function for determining the temporarily stored previous captured image as a storage target when it is determined that the page is turned;
A program to realize

DESCRIPTION OF SYMBOLS 1 Control part 3 Memory | storage part 3a Program memory 3b Work memory 3c A buffer 3d B buffer 4 Operation part 4b Image reading key 5 Touch display part 6 Imaging part 8 Acceleration sensor 11 Tablet terminal device 13 Paper medium (notebook)
M mark

The present invention relates to an image reading apparatus, an image reading method, and a program.

The image reading apparatus according to the present invention in order to solve the problems described above,
Each time a new photographed image is obtained by photographing, it is determined based on the new photographed image whether the new photographed image is a photographed image in the middle of turning the page of the book from the turning position to the turning position. First discrimination means;
When it is determined by the first determination means that the new photographed image is not a photographed image in the middle of turning the page of the book from the turning original position to the turning destination position, the new photographed image is changed to the previous photographed image. Temporary storage means for temporarily storing as,
Prior to the temporary storage unit temporarily storing the new captured image as the previous captured image, the latest captured image stored by the temporary storage unit as the previous captured image so far and the temporary Second discriminating means for discriminating whether or not the latest photographed image and the new photographed image are photographed images of the same spread page based on the new photographed image to be temporarily stored by the storage means; ,
Determining means for determining the latest photographed image as a storage target when the second distinguishing means determines that the latest photographed image and the new photographed image are not photographed images of the same spread page ;
An image reading apparatus comprising:

In order to solve the above-described problems, an image reading method according to the present invention includes:
A book that is marked with the same mark between the two open pages and marked differently between the front and back of the page is opened so that the book in the opened state fits in the angle of view. Shooting steps to shoot from the side,
Each time a new photographed image is obtained by photographing the book in the photographing step, the new photographed image is turned from the original page to the destination position based on the new photographed image. A first determination step of determining whether or not the captured image is,
When it is determined in the first determination step that the new photographed image is not a photographed image in the middle of turning the page of the book from the turning original position to the turning destination position, the new photographed image is changed to the previous photographed image. A temporary storage step for temporarily storing as,
Prior to the temporary storage unit temporarily storing the new captured image as the previous captured image, the latest captured image stored by the temporary storage unit as the previous captured image so far and the temporary A second determination step for determining whether the latest photographed image and the new photographed image are photographed images of the same spread page based on the new photographed image to be temporarily stored by the storage means; ,
A determination step of determining the latest captured image as a storage target when it is determined in the second determination step that the latest captured image and the new captured image are not captured images of the same spread page;
An image reading method characterized by comprising:
In order to solve the above-described problem, a program according to the present invention is:
Computer
Each time a new photographed image is obtained by photographing, it is determined based on the new photographed image whether the new photographed image is a photographed image in the middle of turning the page of the book from the turning position to the turning position. First discrimination means;
When it is determined by the first determination means that the new photographed image is not a photographed image in the middle of turning the page of the book from the turning original position to the turning destination position, the new photographed image is changed to the previous photographed image. Temporary storage means for temporarily storing as,
Prior to the temporary storage unit temporarily storing the new captured image as the previous captured image, the latest captured image stored by the temporary storage unit as the previous captured image so far and the temporary Second discriminating means for discriminating whether or not the latest photographed image and the new photographed image are photographed images of the same spread page based on the new photographed image to be temporarily stored by the storage means;
Determining means for determining the latest photographed image as a storage target when the second distinguishing means determines that the latest photographed image and the new photographed image are not photographed images of the same spread page;
It is a program to make it function as.

Claims (12)

An image reading apparatus provided with an imaging unit that captures information described on a paper medium composed of a plurality of pages,
Each time the paper medium to be read is photographed by the imaging means, the new photographed image is stored as the current photographed image, and the image previously stored as the current photographed image is stored as the previous photographed image. Temporary storage means,
A turning determination means for determining whether or not it is a page turning state by analyzing the current captured image stored in the temporary storage means;
A determination unit that determines a previous captured image stored in the temporary storage unit as a storage target when it is determined by the page determination unit that the page is being turned;
An image reading apparatus comprising: The page turning discriminating means recognizes a mark attached to each page of the paper medium, and discriminates whether or not a page turning state is based on the recognition result of the mark;
The image reading apparatus according to claim 1, which is configured as described above. The page turning discriminating means is added to each page opened by analyzing the current shot image taken in a state where the information of two pages of the paper medium spread within the shooting range of the imaging means is included. When the recognized mark is recognized, it is determined whether or not the page is turned based on the relationship between the recognized marks.
The image reading apparatus according to claim 2, wherein the image reading apparatus is configured as described above. The page turning discriminating means discriminates that if the marks are the same as the remarks attached to each of the spread pages, it is not a page turning state, and if they are different, it is in a page turning state. Determine that there is,
The image reading apparatus according to claim 3, which is configured as described above. Erasing the current captured image stored in the temporary storage means when it is determined by the turning determination means that the page is turned.
The image reading apparatus according to claim 1, wherein the image reading apparatus is configured as described above. In response to an end instruction for instructing the imaging unit to end shooting of the paper medium, the determination unit determines the previous captured image stored in the temporary storage unit as a storage target.
6. The image reading apparatus according to claim 1, wherein the image reading apparatus is configured as described above. The turning determination unit determines whether or not page turning has been performed from a change state of both images by analyzing the current captured image and the previous captured image.
The image reading apparatus according to claim 1, wherein the image reading apparatus is configured as described above. The turning determination unit compares the contents of a plurality of regions in each captured image when analyzing the current captured image and the previous captured image, and the image of the plurality of regions changes. Determine whether the page was turned based on the number of areas,
The image reading apparatus according to claim 7, which is configured as described above. Detection means for detecting the attitude of the apparatus body;
A first shooting control unit that permits shooting by the imaging unit when the posture detected by the detection unit is a predetermined posture;
Further comprising
The image reading apparatus according to claim 1, wherein the image reading apparatus is configured as described above. The image reading device is a mobile terminal device,
In the state where the mobile terminal device main body stands up and the imaging means is arranged on the upper side of the terminal device main body, the first photographing control means causes the mobile terminal device main body to stand up by the detection means. Permitting the photographing of the paper medium disposed close to the lower side of the apparatus body when it is detected that
The image reading apparatus according to claim 9, which is configured as described above. Mode specifying means for selecting and specifying an arbitrary image reading mode from among a plurality of image reading modes having different shooting intervals in accordance with the page turning speed for a paper medium consisting of a plurality of pages;
Second imaging control means for controlling the imaging timing by the imaging means so as to be an imaging interval according to the image reading mode specified by the mode specifying means;
Further comprising
The image reading apparatus according to claim 1, wherein the image reading apparatus is configured as described above. Against the computer,
Each time the paper medium is photographed by the image pickup means for photographing the information described on the paper medium with a paper medium consisting of a plurality of pages as a reading target, the new photographed image is temporarily stored as the current photographed image. In the state where the image previously stored as the current captured image is temporarily stored as the previous captured image, whether or not the page is turned by analyzing the temporarily stored current captured image A function to determine whether or not
A function for determining the temporarily stored previous captured image as a storage target when it is determined that the page is turned;
A program to realize

Images