JP6399135B1 - Image input / output device and image input / output method - Google Patents

Abstract

When description information and / or image information is corrected by displaying description information described in a manuscript and image information obtained by imaging the manuscript on the same screen. However, we will share the understanding of the multiple viewers who are reading the manuscript to make the work common and efficient. A document placing table having a placement surface on which a document is placed, and description information on the document placed on the document placing table are imaged and image image data of the document is recorded. The image data D1 acquired by the imaging unit 2 is projected as projection data D2 and projected onto the imaging unit 2 that acquires D1 and the description information W on the document P placed on the placement surface 1A. Or the projection means 3 which projects the other projection data D3 in an overlapping manner. [Selection] Figure 1

Description

  The present invention relates to an image input / output device and an image input / output method for capturing and outputting description information on a document as image image data.

For example, at a consultation desk such as a financial institution, a bank clerk and a customer may talk face-to-face across a table.
At this time, on the table, a PC including a tablet terminal, a display device for displaying data from the PC, an input device for inputting an image of a document brought by a customer, a document signed on the spot, and the like are installed.
As the input device, a stand-type image input device for inputting an image at a distance from the document surface is used.

As an image input / output device that can be used in such a field, techniques disclosed in Patent Documents 1 and 2 are provided.
For example, Patent Document 1 discloses a technique relating to an image input / output device capable of imaging a subject existing on a reading surface and simultaneously forming an image on a projection surface.
In this image input / output device, an imaging device that captures an image of a document that is a subject, a light modulation device that generates an image, and a subject image is formed on the imaging device via an imaging optical path, and the projection is different from the imaging optical path. And a lens group that projects an image generated by the light modulation element onto the projection surface via the optical path.

  Further, in Patent Document 2, an imaging unit that captures an image of a subject on which description information is displayed and acquires imaging data related to the subject, a description information recognition unit that recognizes description information from the imaging data acquired by the imaging unit, A technique relating to an image input / output device including projection means for projecting related information related to description information recognized by the description information recognition means onto a subject is shown.

JP 2008-92234 A JP 2006-31506 A

However, the image input / output device disclosed in Patent Document 1 has a configuration in which a reading surface on which a subject is imaged by an imaging element and a projection surface on which an image is projected by a lens group are arranged adjacent to each other. Therefore, there are two display bodies, that is, an original document that is a subject and a projection surface on which an image is projected.
For this reason, the bank clerk and the customer need to compare these two display bodies depending on the situation, and there is a problem that the work efficiency is deteriorated.

  Further, in the image input / output device disclosed in Patent Document 2, the related information related to the description information recognized by the description information recognition means is configured to be projected onto the subject. There were two pieces of information, related information, and confusion occurred depending on the situation.

  The present invention has been made in view of the above-described circumstances, and displays the description information described in the manuscript and the image information obtained by imaging the manuscript on the same screen. Alternatively, even when image information or both of these information corrections are made, an image input / output device capable of sharing work and improving efficiency by sharing the understanding of a plurality of viewers who are viewing the manuscript And an image input / output method.

In order to solve the above problems, the present invention proposes the following means.
In the image input / output device according to the first aspect of the present invention, a document placing table having a placing surface on which a document is placed, and description information on the document placed on the placing surface of the document placing table are imaged. And photographing means for obtaining image image data of the original, and projecting the image image data obtained by the photographing means as projection data in a superimposed manner toward the description information on the original placed on the placement surface. And / or projection means for projecting other projection data on top of each other.

  In the image input / output method according to the second aspect of the present invention, the description information on the document placed on the document placement table is imaged to obtain image image data of the document, and then the image image data is used as projection data. And projecting the projection data or other projection data on the description information on the document on the document table.

  According to the present invention, it is possible to share the understanding of a plurality of viewers browsing the manuscript and to make the work common and efficient.

(A) is a schematic configuration diagram of an image input / output apparatus according to the present invention, and (B) is a diagram showing a state in which an image is projected on description information on a document. 1 is a schematic configuration diagram of an image input / output apparatus according to a first embodiment of the present invention. It is a figure which shows the control flow of a pointing object recognition part. (A) is a state in which the pointing object is separated from the shadow portion, and (B) is a diagram illustrating a state in which the pointing object and the shadow portion are in contact with each other. It is a schematic block diagram of the image input output device which concerns on 2nd Embodiment of this invention. It is sectional drawing of a sensor apparatus. It is a schematic block diagram of the image input output device which concerns on 3rd Embodiment of this invention. It is explanatory drawing which shows the state which the original document shifted | deviated. It is a schematic block diagram of the image input output device which concerns on 4th Embodiment of this invention. It is a figure which shows the control flow of an image processing unit.

The present invention will be described with reference to FIG.
FIG. 1A is a schematic configuration diagram of an image input / output apparatus 100 according to the minimum configuration of the present invention. Reference numeral 1 denotes a document placement table having a placement surface 1A on which a document P is placed.
Above the document table 1, a photographing unit 2 that captures the description information W on the document P placed on the document table 1 and obtains image image data D 1 of the document P is installed.

In addition, a projection unit 3 that projects the image image data D1 acquired by the imaging unit 2 as projection data D2 onto the mounting surface 1A is provided above the document table 1 and in the vicinity of the imaging unit 2. Yes.
Further, between the photographing unit 2 and the projection unit 3, a control unit 4 is installed that causes the projection unit 3 to project the projection data D 2 on the description information W on the document P.
In addition to projecting image image data D1 acquired by the photographing means 2 from the projection means 3 onto the placement surface 1A as projection data D2, other projection data D3 is directed toward the placement surface 1A. Can be projected.

Then, according to the image input / output apparatus 100 configured as described above, after the description information W on the document P placed on the document table 1 is imaged and the image image data D1 of the document P is acquired, The image data D1 is projected as projection data D2 toward the placement surface 1A. At that time, as shown in FIG. 1B, the image image M indicated by the projection data D2 and / or other projection data D3 is projected onto the description information W on the document P.
Further, such an operation of acquiring the image image data D1 of the document P is performed at regular intervals. Thus, a feedback process is performed to update the image image data D1 and the projection data D2 to the latest one.

In the image input / output apparatus 100 of the present invention, the description information W written on the document P on the placement surface 1A of the document placement table 1 and the image image obtained by imaging the document P are on the same plane. Even if the description information W and / or the image image M are corrected, the correction result is immediately reflected on the screen by the feedback processing described above.
For example, even when the description information W on the document P is directly corrected with a writing instrument such as a ballpoint pen, or when an image image projected on the document P is corrected by a touch sensor or PC operation, the correction results are the same. By being immediately reflected on the screen, it is possible to make the work common and efficient by sharing the understanding of multiple viewers.

In the image input / output apparatus 100 according to the present invention, “after the description information W on the document P placed on the document table 11 is imaged and the image image data D1 of the document P is acquired, the image image Data D1 is projected onto the document P as projection data D2, and the projection data D2 and / or other projection data D3 are projected onto the description information W on the document P on the document table 11 so as to be projected. "Output method" is adopted.
When the projection data D2 and D3 are projected onto the description information W on the document P of the document table 11, the projection data D2 and D3 are made to match the description information W and are different from the description information W. The projection pattern may be projected to various locations, and various projection patterns are employed.

(First embodiment)
A first embodiment of the present invention will be described with reference to FIGS.
FIG. 2 is a configuration diagram of the image input / output apparatus 101 according to the first embodiment of the present invention. Reference numeral 11 denotes a document placement table having a placement surface 11A on which a document P is placed.
Above the document table 11 is an image input unit that captures the description information W on the document P placed on the placement surface 11A of the document table 11 and obtains image image data D1 of the document P. The photographing means 12 is installed.

The photographing means 12 is of a camera type capable of inputting an image indirectly with a certain distance from the document P placed on the placement surface 11 A of the document placement table 11. A lens that forms an image of reflected light corresponding to the placed document P and an image sensor that is a photoelectric conversion element such as a CCD or a CMOS (not shown) are provided on the imaging surface.
Then, an electrical signal is sent from the image sensor to the signal processing unit, subjected to digital processing, and a document image is acquired as digital data of the image image data D1.

Also, above the document table 11 and in the vicinity of the photographing means 12, the image data D1 acquired by the photographing means 12 is used as projection data D2 and / or other projection data D3 is placed on the placement surface 11A. Projection means 13 for projecting toward the projector is installed.
This projection means 13 is comprised by the projector part which consists of three color light-emitting bodies, such as LED, a microphone mirror semiconductor, an optical lens, and a curved mirror.
In the projector unit, the component arrangement and the projection direction of the apparatus are set so as to project onto the projection plane including the same document table 11 as that of the image input apparatus.

  In this projection means 13, red, blue, green, and three color LEDs are turned on, an image signal is input to the micromirror LSI from a PC or the like, and the micromirror is opened and closed in units of pixels according to the image. To adjust ON / OFF of the light from the LED and the amount of light. The light beam whose light amount has been adjusted corresponding to the image signal is refracted by the lens, reflected by the mirror, and travels to the same projection surface as the placement surface 11A.

Further, an image processing unit 14 having a plurality of functions relating to image data, for example, an image positioning unit 14A and a pointing object recognition unit 14B, is installed between the photographing unit 12 and the projection unit 13.
In the image positioning unit 14A, the projection unit 13 projects the image image M indicated by the projection data D2 or other projection data D3 on the description information W on the document P, specifically, The position data of the description information W on the document P is obtained from the image image data D1 obtained by the photographing means 12, and based on the position data, the projection data D2 or other projection data D3 is sent to the projection means 13. The description information W on P is projected on top of it.

Further, as shown in FIGS. 3 and 4, the pointing object recognition unit 14B recognizes the tip of the pointing object 20 pointing to the description information W on the document P based on the image image data D1 of the photographing unit 12, and The data input location is specified based on the recognition result. Specifically, the pointing object recognition unit 14B recognizes the tip position of the pointing object 20 by detecting the pointing object 20 and the outline of its shadow based on the image image data D1 of the photographing unit 12.
Note that as the pointing object 20, means that can specify a part of an image projected by the projector, such as a stick having a tapered tip, a pen tip, and a fingertip of an operator or a consultant, is used.

The processing contents of the pointing object recognition unit 14B will be described with reference to FIGS.
In the pointing object recognition unit 14B, as shown in step (S) of the flowchart of FIG. 3, after acquiring the image image data D1 by the photographing means 12 (step 1), the pointing object recognition step (step 1) for recognizing the pointing object 20 ( Step 2) and a shadow recognition step (Step 3) for detecting the shadow portion 21 caused by the pointing object 20 blocking the projection light of the projection data D2 or other projection data D3 are executed.
Thereafter, in the comparison process (step 4), the position of the pointing object 20 obtained by executing steps 2 and 3 is compared with the position of the shadow portion 21, and the determination process (step 5), it is determined whether or not the pointing object 20 has come into contact with the document S as shown in FIGS.
4A shows a state in which the pointing object 20 and the shadow portion 21 are separated (a separation distance is indicated by a symbol a), and FIG. 4B shows a state in which the pointing object 20 and the shadow portion 21 are in contact with each other. It is shown.

Further, as a scene where the processing in FIG. 3 is used, it is assumed that an operator and a counselor are seated with a table on which the image input / output device 101 is placed.
In order to perform a process equivalent to clicking the button image projected from the projection unit 13 on the document placing table 11 with a mouse, a touch pen, a finger or the like, the following operation is performed.
That is, the image input / output device 101 executes a feedback process in which image capturing by the image capturing unit 12 is performed simultaneously with projection by the projecting unit 13, and the contour of the pointing object 20 and the instruction are performed during the feedback process. The outline of the shadow portion 21 of the pointing object 20 due to the object 20 blocking the projection light is captured by the photographing unit 12.
When both contours come into contact with each other, it is determined that the pointing object 20 has touched the document table 11 and processing equivalent to that the button is clicked by the pointing object 20 is performed. .
At this time, whether or not both contours are in contact with each other is determined based on whether or not the contrast becomes sharp after a sharp change in brightness between the shadow contour and the non-shadow portion occurs. May be.

According to the image input / output device 101 configured as described above, after the description information W on the document P placed on the document table 11 is imaged and the image image data D1 of the document P is acquired, , the image picture data D1 as the projection data D2, or projected toward the surface 11A mounting the other projection data D3. At this time, the projection data D2 or other projection data D3 is projected onto the description information W on the document P.

Thus, in the image input / output apparatus 101, the description information W described on the document P on the placement surface 11A of the document placement table 11 and the image image M obtained by imaging the document P are on the same surface. Even if the description information W and / or the image image M, or both of the information corrections are made, it is possible to share the work with the understanding of the plurality of viewers who are viewing the original P. , Efficiency can be achieved.
For example, even when the description information W on the document P is directly corrected with a writing instrument such as a ballpoint pen, or when an image image projected on the document P is corrected by a touch sensor or PC operation, the correction results are the same. By being reflected on the screen, it is possible to share the understanding of a plurality of viewers and to make the work common and efficient.

In the image input / output apparatus 101 according to the present embodiment, when two or more people are consulting with each other across the table, the document placement table 11 is placed on the table so that the display on the same table is displayed. Both people can check the screen.
Further, since the document P confirmed by both sides can be input as an image, it is effective in sharing information. In addition, since it is possible to talk from the front without turning the line of sight sideways compared to the example displayed on the display or the like, there is an effect that the negotiation and consultation can be processed smoothly.

  Furthermore, in the image input / output apparatus 101 of the present embodiment, image input is performed by pressing a button or the like projected from the photographing unit 12 on the document table 11 with a pen tip or a finger as the pointing object 20. The same function as when the device is read and the process is clicked with a mouse or a touch pen can be obtained. For this reason, there is no need to provide a dedicated touch sensor, which is effective in reducing the number of parts and simplifying the apparatus.

(Second Embodiment)
A second embodiment of the present invention will be described with reference to FIGS.
The image input / output device 102 shown in the second embodiment differs from the image input / output device 101 shown in the first embodiment in the configuration of the document table 11.
In other words, the placement surface 11A of the document placement table 11 shown in the second embodiment is provided with a sensor device 30 that irregularly reflects the projection light projected from the projection unit 13 and detects the contact of the tip of the pointing object 20. It has been.
As shown in FIG. 6, the sensor device 30 includes a base member 31 located in the lower layer, a touch sensor 32 located on the base member 31, and a surface sheet 33 located on the touch sensor 32. The surface of the top sheet 33 is processed to diffusely reflect the projection light.

As a specific surface sheet 33, a surface material that finely and diffusely reflects light is desired on the side on which the projection light is incident, and a white paper material, a white plastics material, or a plastic material is white on a low cost basis. A mud-like paint is used.
In addition, as a sensing unit of the touch sensor 32, for example, a detection sheet using a capacitance detection method or a resistance film detection method is used.
In the detection sheet of the touch sensor 32, a change in capacitance is detected by capacitance, a change in resistance is detected by resistance film detection, and a detection signal indicating these changes is supplied to the contact detection processing unit 34.
In the touch detection processing unit 34, it is possible to detect which portion of the touch sensor 32 is pointed to by the touch of the pointing object 20 in the two-dimensional plane. At this time, in the case of the electrostatic capacitance detection method, the pointing object 20 having moisture such as a finger or the pointing object 20 that changes the electrostatic capacity needs to be used. In the case of the resistance film detection method, it is necessary to apply pressure directly to the indicated portion.

In the image input / output device 102 of the second embodiment configured as described above, the description information W described on the document P on the placement surface 11A of the document placement table 11, The image M obtained by imaging the document P can be displayed on the same plane, and even when the description information W and / or the image image M is corrected, the document P is browsed. By sharing the understanding of multiple viewers, it is possible to make work common and efficient.
Further, since the image input / output device 102 is provided with the sensor device 30 that directly detects the contact of the tip of the pointing object 20 on the placement surface 11A of the document placement table 11, a projection image such as a button appearing on the projection image. When is touched, a function equivalent to clicking the button with a mouse or the like can be obtained. As a result, when the document placement table 11 is provided on the table facing the display screen, the click operation can be performed while confirming both, so that there is an effect that information sharing can be easily performed.

(Third embodiment)
A third embodiment of the present invention will be described with reference to FIGS.
The image input / output device 103 shown in the third embodiment differs in configuration from the image input / output devices 101 and 102 shown in the other embodiments because of the control contents of the image processing unit 14 that controls the projection means 13. It is the details about.
That is, in the image processing unit 14, as shown in FIGS. 7A and 7B, the document P on the document table 11 is moved (parallel movement and rotation movement as indicated by an arrow (A)). Then, based on the amount of movement, the projection unit 13 performs a process of moving the projection data D2 or other projection data D3 so as to overlap the description information W on the document P.

Specifically, the image processing unit 14 performs a process of moving the image image data D1 obtained by the image capturing by the image capturing unit 12 in parallel and rotating according to the amount of movement of the actual document P on the document table 11. Do. In order to translate / rotate the projection image, the position information of the description information W described in the original document P serving as a reference is acquired, and the two-dimensional data of the digital image is translated / rotated.
In addition, as processing for detecting the movement of the document P, several to several tens of images are acquired per second, and the position coordinates of the feature amount of the document P are obtained from the image each time. When there is a change in the coordinates of the position recorded, the position change data is given as the reference position change of the photographing means 12. As a result, the projected image from the photographing unit 12 is an image that moves in response to a change in the position of the document P.

For example, let us consider a case where the name and date of birth of a person who came to the consultation are entered in the place where the ruled frame is projected.
The ruled line frame of the table provided with the name and date of birth fields is projected on the document table 11 by the photographing means 12 as a projected image. Then, ask the counselor to actually fill in the inside of the ruled line frame.
For example, as shown in FIG. 8, even when the paper that is the written original P is shifted as indicated by the original P ′, the image input / output device 103 shown in the third embodiment is used. Since the deviation of the entered document P is detected and the projected image is moved correspondingly, there is no problem in actual entry.

In the image input / output device 103 of the third embodiment configured as described above, description information described in the document P on the placement surface 11A of the document placement table 11 is the same as in the first and second embodiments. W and the image image M obtained by imaging the document P can be displayed on the same plane, and the document P can be displayed even when the description information W and / or the image image M is corrected. By sharing the understanding of the plurality of viewers who are browsing, it is possible to make the work common and efficient.
In the image input / output device 103, even if the document P is moved, the projected image also moves together with the document P. Therefore, for example, the name and date of birth are projected on the ruled line frame, and the person who came to consult the document P When the contents of the ruled line frame are filled in, no problem occurs even if the original P is displaced. That is, in the case of the image input / output device 103 shown in the third embodiment, the deviation of the document P being entered is detected, and the projected image also moves in accordance with the deviation, so that the actual entry and the projected image overlap. , There is an effect that the alignment between the entry and the ruled line frame is always kept.

(Fourth embodiment)
A fourth embodiment of the present invention will be described with reference to FIGS.
The image input / output device 104 shown in the fourth embodiment differs in configuration from the image input / output devices 101 to 103 shown in the other embodiments in the control contents of the image processing unit 14 that controls the projection means 13. It is.
As shown in FIG. 9, the image input / output device 104 shown in the fourth embodiment includes an operation unit 30 that is instructed by an operator, and an operation from the operation unit 30 is performed by the image processing unit 14. Based on the instruction, the image projection of the projection means 13 is adjusted.
Specifically, in the image processing unit 14, based on an operation instruction from the operation unit 30, when an image input operation is performed, the image projection is stopped, or uniform light indicated by reference numeral 40 is irradiated as illumination. To switch to.

As control contents of the image processing unit 14, as shown in the flowchart of FIG. 10, first, it is determined whether or not the mode for simply reading the document P on the document placement unit 11 is selected by the operation unit 30. (Step 11).
If the mode for simple reading is selected YES in step 11, the size of the original P is detected by reading the original P on the original placing portion 11 by the photographing means 12 (step 12). Then, the light beam 40 from the projection means 13 is irradiated as illumination on the original P (step 13), and in this state, the original P on the original placement portion 11 is read again by the photographing means 12 (step 14). ).
In step 13, the illumination by the projection unit 13 is stopped when the simple reading mode is selected in the operation unit 30 and the illumination function is stopped.
Thereafter, it is determined whether or not the reading operation of the document P on the document placement unit 11 by the photographing unit 12 has been completed (step 15). If YES, this flow is terminated. If NO, the previous step 11 is determined. Return to.

On the other hand, when the normal mode is selected in step 11, the photographing unit 12 reads the document P on the document placement unit 11 (step 16), and the projection unit 13 performs the photographing unit 12. A process of projecting the acquired image image data D1 onto the document P on the document placement unit 11 as projection data D2 is performed (step 17).
Thereby, as described above, the description information W described in the document P on the document table 11 and the image image M obtained by imaging the document P are displayed on the same plane.

  In the image input / output device 104 of the fourth embodiment configured as described above, the description information W written on the document P on the document table 11 and the document P are the same as in the first to third embodiments. The image image M obtained by imaging the image P can be displayed on the same plane, and even when the description information W and / or the image image M are corrected, a plurality of documents P are viewed. By sharing the understanding of human viewers, work can be made common and efficient.

In the image input / output device 104 of the fourth embodiment, the device may be installed in a dark place or an unstable place, and even if the brightness is insufficient or there is a shadow, An image can be acquired using the light beam 40 from the imaging means 12, and there is an effect that image input can be performed stably.
In addition, the image input / output device 104 of the fourth embodiment has the effect that convenience can be improved by making the device itself portable so that images can be obtained with confidence wherever it is held. is there.

  As mentioned above, although embodiment of this invention was explained in full detail with reference to drawings, the concrete structure is not restricted to this embodiment, The design change etc. of the range which does not deviate from the summary of this invention are included.

  The present invention relates to an image input / output device and an image input / output method for capturing and outputting description information on a document as image image data.

DESCRIPTION OF SYMBOLS 1 Document placement part 1A Placement surface 2 Imaging means 3 Projection means 4 Control means 11 Document placement part 11A Placement surface 12 Imaging means 13 Projection means 14 Image processing unit 14A Image positioning part 14B Pointed object recognition part 20 Pointed object 21 Shadow part 30 Sensor device 32 Touch sensor 100 Image input / output device 101 Image input / output device 102 Image input / output device 103 Image input / output device 104 Image input / output device P Original M Image image W Description information

Claims (5)

An image input / output device disposed obliquely above a document placement table having a placement surface on which a document is placed,
Photographing means for capturing description information on a document placed on a placement surface of the document placement table and acquiring image image data of the document;
Projection means for projecting and / or projecting other projection data by superimposing the image data acquired by the photographing means as projection data toward the description information on the document placed on the placement surface. When,
Control means for causing the projection means to project the projection data on the description information on the document based on position data of the description information on the document,
The control means recognizes the tip of the pointing object indicating the description information on the document based on the image image data of the photographing means, specifies the data input location based on the recognition result,
By detecting the contour of the pointing object and the contour of the shadow of the pointing object based on the image image data of the photographing means, the tip position of the pointing object is recognized, and the document or projection image on the document mounting table be one that detects the presence or absence of a touch to,
An image input / output device that detects the presence or absence of a touch based on whether or not the contrast becomes sharp after a sharp change in the brightness of the shadow and non-shadow of the pointing object occurs . When the document on the document table is moved, the control unit moves the projection data so that the projection unit overlaps the description information on the document based on the movement amount. The image input / output device according to claim 1 , wherein: It has an operation unit where operation instructions are given by the operator,
The image input / output apparatus according to claim 1 , wherein the control unit adjusts projection from the projection unit based on an operation instruction from the operation unit. The control unit is configured to switch the projection unit to stop image projection or to irradiate uniform light as illumination based on an operation instruction from the operation unit during an image input operation. The image input / output device according to claim 3 . The image information of the original is acquired from an image of the original placed on the original placing table by imaging the description information on the original placed from obliquely above the original placing table, and then the image image data is used as projection data on the original. And projecting the projection data and / or other projection data on the description information on the document on the document table,
Based on the position data of the description information on the document, the projection unit is controlled to project the projection data on the description information on the document,
Recognizing the tip of the pointing object pointing to the description information on the document based on the image image data, specifying the data input location based on the recognition result,
By detecting the contour of the pointing object and the contour of the shadow of the pointing object based on the image image data, the tip position of the pointing object is recognized to touch the document or the projected image on the document placing table. detects the presence or absence of,
An image input / output method characterized by detecting the presence or absence of a touch based on whether or not the contrast becomes sharp after a sharp change in brightness of a shadow of the pointing object and a non-shadow .

Images