JP2008134793A - Electronic handwriting input device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To surely determine handwriting input by a simulated pen and the like for a handwriting input face projected on any of various projection bodies such as a desk, table, and white board. <P>SOLUTION: An electronic handwriting input device 10 is constituted such that a compound eye imaging section 14 and a projection section 11 are integrally composed and are freely changed in angle by a universal joint 34. Then, input for the handwriting input surface 36 by an indicated point 112 of an indicating means 110 such as the simulated pen is defined according to a distance between the indicated point 112 and the handwriting input surface 36. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  This invention captures a scene image including a handwriting input surface projected from a projection unit onto a projection object by a compound eye imaging unit, and uses a tip or the like as a pointing point (pointer) corresponding to a pen tip or the like. The present invention relates to an electronic handwriting input device for determining input confirmation on the handwriting input surface by instruction means.

  Conventionally, as shown in Patent Document 1, an image projection unit that projects an operation surface image, a display control unit that performs display control of the image projection unit, and a user operation position on the projected operation surface image are detected. An operation position detection unit; and a detection data processing unit that executes data processing corresponding to the detected user operation position, and the display control unit determines the operation surface according to the user operation position detected by the operation position detection unit. An electronic device that changes a part of an image has been proposed.

  In this electronic apparatus, a user operation position is detected by using a laser light source as the operation position detection means and receiving reflected light of the laser beam scanned in a fan shape with a light receiving sensor.

  Moreover, as shown in Patent Document 2, the imaging unit includes an imaging unit that captures an imaging region on the stage, and an image processing unit that detects an indicated position indicated in the imaging region from the captured image information. An input system for inputting an instruction portion by directly indicating an arbitrary position in an area with an instruction member is disclosed.

  In this input system, the stage is irradiated with light to have a light emitting unit that defines the range of the imaging region, and the range of the imaging region is visually recognized. Further, the input position is determined by projecting and displaying an image in the imaging region. Further, the menu image is irradiated on the stage, and the input is determined when a desired display position on the menu image is designated for a predetermined time. Alternatively, a dedicated pallet function having light emitting units arranged in a matrix on the stage is provided, and a switch for determination is provided. Alternatively, information is input by capturing an image written in the imaging region with a pen using ink.

  Furthermore, as shown in Patent Document 3, a distance image is generated by a stereo method from images input from a plurality of cameras, and the distance image and two predetermined parts (start point and end point) of the operator's body are used. The three-dimensional coordinates of the start point and the end point in the real space are calculated, the direction indicated by the operator's two parts is calculated, and the intersection point is obtained from the extension line of the direction and the information of the previously registered object. An interface device for detection is disclosed.

Japanese Patent Laying-Open No. 2005-242725 (FIG. 7) JP 2000-259338 A (FIG. 17) JP 2004-265222 A (FIG. 3)

  However, in the technique according to Patent Document 1, since a laser light source is used as the operation position detection unit, laser light may enter the eye, which is not preferable. In addition, this technology has a drawback that the position cannot be accurately detected when the projection surface is curved, for example, because the surface on which the laser beam is irradiated is limited to the installation surface of the electronic device.

  In addition, the technique according to Patent Document 2 has a disadvantage that a position indicating unit such as a pallet in which light emitting units are arranged in a matrix is required separately from the projection display unit, and the system is expensive.

  Furthermore, in the technique according to Patent Document 3, since the start point and end point in space are detected and the direction indicated by the operator is detected, a slight detection error at hand becomes a large error on the projection plane. The operation is unstable and lacks accuracy.

  The present invention has been made in consideration of the above-mentioned drawbacks, has few restrictions on the handwriting input surface, does not require a position indicating means such as a pallet, and the hand by the indicating means having the tip as an indicating point. It is an object of the present invention to provide an electronic handwriting input device that can reliably determine the confirmation of an input to a document input surface.

  An electronic handwriting input device according to the present invention includes a projection unit that projects a handwriting input surface onto a projection target, instruction means for performing handwriting input with a tip as an instruction point, and the projected handwriting input A compound eye imaging unit that captures a scene image including a surface, and a distance calculation that calculates a distance from the compound eye imaging unit for each pixel of the captured handwriting input surface to obtain a projected coordinate of the handwriting input surface And an instruction means recognition unit for recognizing the instruction point of the instruction means from the object scene image, and has the following features (1) to (4).

  (1) The compound-eye imaging unit is configured to freely change the angle integrally with the projection unit, and confirms the input to the handwriting input surface by the indication point of the indication unit and the position of the indication point and the A handwriting input confirming unit that confirms the distance according to the distance from the handwriting input surface is provided.

  According to the invention having the feature (1), since the compound eye imaging unit and the projection unit are configured to freely change the angle integrally, the projection surface of the handwriting input surface is a table, a desk, a wall, etc. The handwriting input surface is less restricted, and the input to the handwriting input surface by the pointing means of the pointing means is confirmed by the handwriting input confirming unit with the position of the pointing point. Since it is determined according to the distance from the handwriting input surface, it is possible to reliably determine the input without requiring a pallet or the like.

  (2) In the invention having the above feature (1), the projection unit further includes a first marker projection function unit that projects a first marker at a plurality of positions according to the handwriting input surface, and the distance calculation. The unit calculates each of the projected distances to the plurality of first markers, and calculates three-dimensional coordinates of the handwriting input surface based on the calculation result.

  According to the invention having the feature (2), the position coordinates of the handwriting input surface can be detected with higher accuracy.

  (3) In the invention having the above feature (1), the projection unit further includes a second marker projection function unit that projects a second marker at a plurality of positions in the handwriting input surface, and the distance calculation unit Calculates the distance to each of the plurality of projected second markers, and approximates the handwriting input surface to a three-dimensional curved surface based on the calculation result, thereby obtaining the three-dimensional coordinates of the handwriting input surface. It is characterized by calculating.

  According to the invention having the feature (3), the distance calculation unit calculates distances to the plurality of second markers projected on the handwriting input surface, and based on the calculation result, the handwriting input surface. Is approximated to a three-dimensional curved surface, so that the three-dimensional coordinates of the handwriting input surface are calculated. Therefore, the three-dimensional coordinates of the curved handwriting input surface can be calculated with high accuracy. As a result, electronic handwriting input can be performed even if the handwriting input surface is curved.

  (4) In the invention having the above feature (1), the handwriting input confirmation unit sets the distance between the handwriting input surface and the indication point to the handwriting input surface for confirming handwriting input. The nearest handwriting input confirmed distance range and the boundary area distance range of a certain distance range from the maximum distance of the handwriting input confirmed distance range are provided, and the position of the indicated point once enters the handwriting input confirmed distance range. After the determination, the determination of the handwriting input is canceled only when it is determined that the distance is longer than the maximum distance of the boundary area distance range.

  According to the invention having the feature (4), since the boundary region distance range is provided, the influence of fluctuations in input confirmation is reduced, and unintended noise mixing (unintentional handwriting confirmation) is prevented.

  (5) The invention having the feature (1) is characterized in that a plurality of the projection units are provided, and a stereoscopic image can be displayed on the handwriting input surface.

  According to the invention having the feature (5), when the user uses polarized stereoscopic glasses, handwriting or the like can be input on the handwriting input surface while viewing a stereoscopic image or the like.

  According to the present invention, since the compound eye imaging unit and the projection unit are integrally configured to freely change the angle, the restriction on the handwriting input surface to be projected is reduced, so that the restriction on the arrangement of the handwriting input device is moderated. And the handwriting input is determined according to the distance between the handwriting input surface and the pointing point of the pointing means, so the position inputting means such as a pallet is not required. It is possible to reliably determine the confirmation of the input to.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a system block diagram of an electronic handwriting input device 10 according to an embodiment of the present invention.

  An electronic handwriting input device 10 as a document device basically includes a projection unit (image projection unit) 11 such as a projector, and a compound eye imaging unit 14 such as a compound eye camera including a first camera 21 and a second camera 22. And an image processing unit 16 that drives and controls the projection unit 11 and the compound eye imaging unit 14.

  The image processing unit 16 is configured to be capable of mounting (detaching) a recording medium 40 that is a storage unit that stores data such as an image to be projected and a document. It should be noted that data such as images and documents to be projected can also be taken in via the communication interface 42 from an external device or the like.

  Further, the image processing unit 16 includes a CPU, a ROM, a RAM, and the like, and functions as various function implementing units (means) described later when the CPU executes a program stored in the ROM.

  FIG. 2 is an explanatory diagram showing the appearance and use state of the electronic handwriting input device 10. As shown in FIG. 2, the electronic handwriting input device 10 includes an image processing unit 16 that is arranged on a projection target 18 such as a table and also serves as a stand, and a universal joint 34a for the image processing unit 16. One end of the support rod 32 is engaged, and the projection unit 11 and the compound-eye imaging unit 14 are integrated with the base 13 attached to the other end of the support rod 32 via a universal joint 34b (see also FIG. 11 described later). It is fixed to.

  Therefore, the projection unit 11 and the compound eye imaging unit 14 are configured to freely change the angle integrally with the image processing unit 16 via the base 13, the universal joints 34 (34 a and 34 b), and the support rod 32. .

  The projection unit 11 projects the handwriting input surface 36 on the projection target 18. A sheet of paper or the like that covers the projected handwriting input surface 36 may be disposed on the projection surface of the projection target 18.

  The handwriting input surface 36 is opposed to instruction means 110 having a tip 112 as a point (tip point, pointer). The instruction unit 110 may be replaced with a pen that can actually write a character or graphic document on the projection target 18, a pen-like object that cannot be written (simulated pen), or a finger such as an index finger. it can.

  In FIG. 1, the first and second cameras 21 and 22 constituting the compound-eye imaging unit 14 output a lens system 44, an imaging device 46 such as a CCD, an A / D conversion unit 48, and image signals Sa and Sb, respectively. A signal processing unit 50, a lens system 44, and an automatic exposure / focus control unit (AE / AF control unit) 51 of the image sensor 46 are provided.

  Here, the first and second cameras 21 and 22 are arranged on the box-shaped base 13 at predetermined intervals and directions, and correction (calibration) as a compound eye camera is a reference chart in advance. It has been implemented using such as.

  The object scene image including the handwriting input surface 36 projected onto the projection target 18 by the projection unit 11 is captured by the compound eye imaging unit 14, and the corresponding image signals Sa and Sb are supplied to the image processing unit 16.

  The image processing unit 16 includes a block matching unit 53 that detects corresponding points on each image of the image signals Sa and Sb corresponding to the object scene image captured from the compound-eye imaging unit 14, and an actual corresponding point (handwriting input surface). A distance calculation unit 54 for obtaining a distance to the corresponding point 36), a determination processing unit 60 including an instruction unit recognition unit 62 for recognizing the instruction point 112 of the instruction unit 110 from the captured scene image, and the like. Is provided.

  For example, the block matching unit 52 sets a block (window) of an appropriate size on the image of one image signal Sa, and obtains an area corresponding to the block in the image of the other image signal Sa. An area having the same size as the block is set in the image. Then, a difference is obtained by subtracting the pixel values of the corresponding pixels (corresponding points) constituting the image in each block on both images, and the absolute value of the difference is further taken. Next, the sum in the block of the absolute value of the difference for each pixel, the so-called sum is obtained. In this way, the calculation for obtaining the sum of the absolute values of the differences between the pixel values in the block is sequentially performed by changing the block position on the other image, and the block of the other image in which the sum is minimized A so-called stereo matching process is performed to determine that the area corresponds to an image block.

  The distance calculation unit 54 calculates the distances (three-dimensional coordinates) from the image sensors 46 of the first and second cameras 21 and 22 for each pixel in the corresponding region in the imaged handwriting input surface 36. The distance data is obtained by determining the coordinates of each pixel when the handwriting input surface 36 constituting the projected handwriting input surface 36 is regarded as a two-dimensional surface. Store in the storage unit 56.

  The discrimination processing unit 60 recognizes a specific pattern such as the instruction point 112 of the instruction unit 110 from the image signals Sa and Sb, and the handwriting input surface 36 by the instruction point 112 of the instruction unit 110 in addition to the instruction unit recognition unit 62 described above. A handwriting input confirmation unit for determining confirmation of handwriting input to the handwriting according to a normal distance between the coordinate position of the pointing point 112 and the handwriting input surface 36 (normal distance b shown in FIG. 9 and the like described later) 64.

  Data and commands from the discrimination processing unit 60 are supplied to the system control unit 66. The system control unit 66 generally controls the electronic handwriting input device 10 including the discrimination processing unit 60, the compound eye imaging unit 14, the image processing unit 16, and the projection unit 11.

  The system control unit 66 includes a discrimination processing unit 60, a monitor 68 such as a liquid crystal display device, an image processing unit 70, a communication interface (communication I / F) 42, a media interface (media I / F) 72, a memory buffer 74 and the projection unit 11 are connected.

  The projection unit 11 includes, for example, LEDs 81, 82, 83 of a light source 85 composed of LEDs 81, 82, 83 arranged in an array of red, blue, and green driven through an LED driver 80. The combined light is arranged on each surface, reflected by the polarization beam splitter 88 through the lens 86, and selectively reflected on the pixel electrode of the LCOS (Liquid Crystal on Silicon) 90 which is a reflective liquid crystal device according to the polarization surface. The emitted light is projected onto the projection target 18 via the polarization beam splitter 88 and the projection lens 92.

  The LCOS driving unit 94 that drives the pixel electrodes of the LCOS 90 is supplied with image data from the image processing unit 70 and predetermined image data (pattern data) from the pattern generation unit 96 through the system control unit 66. . In this case, the ROM (or EEPROM) of the system control unit 66 stores image data Im1 of a first marker and image data Im2 of a second marker, which will be described later, and image data Im1, Im2 of these first and second markers. Is set in the pattern generator 96. When the image data Im1 and Im2 of the first or second marker are set in the pattern generation unit 96, the projection unit 11 operates as a first marker projection function unit or a second marker projection function unit, respectively. The image data Im1 and Im2 of the first and second markers can be stored in the recording medium 40.

  Then, as shown in FIG. 2, the video (image, pattern, etc.) projected from the projection unit 11 is projected onto the projection target 18 that is a plane such as a table or a desk, and the projected image, pattern, etc. The handwriting input surface 36 including the image is photographed by the compound eye imaging unit 14.

  A more detailed operation of the electronic handwriting input device 10 that is basically configured and operates as described above will be described with reference to the flowchart of FIG. Note that the program corresponding to this flowchart is basically executed by the image processing unit 16.

  When power is supplied to the electronic handwriting input device 10, the size is larger than the contour of the handwriting input surface (image projection surface) 36 from the projection unit 11 through the pattern generation unit 96 under the control of the system control unit 66. The rectangular trapezoidal distortion correction pattern 100 shown in FIG. 4 is irradiated (light is irradiated on the inner side of the rectangular shape, and it is recognized as a rectangular shape because light is not irradiated on the outer side).

  In general, since the projection planes of the projection unit 11 and the projection target 18 are often not facing each other, a rectangular image projected from the projection unit 11 is projected on the plane of the projection target 18 on the projection target 18. Then, it is projected as a square image such as a trapezoid.

  Next, in step S <b> 2, the system control unit 66 captures the projected square image with either one of the first camera 21 and the second camera 22 constituting the compound eye imaging unit 14, and the system control unit 66. Then, in order to make the captured square such as a trapezoid into a rectangle, adjustment is performed by supplying image data obtained by deforming the trapezoidal distortion correction pattern 100 to the pattern generator 96 or by driving the optical system of the projection unit 11. To do. After adjusting the trapezoidal distortion correction pattern 100 to be projected in a rectangular shape on the projection object 18 in this way, the light source is adjusted so that the luminance of the entire projected rectangular trapezoidal distortion correction pattern becomes uniform. The luminance is adjusted by driving and controlling each element electrode 85 or LCOS 90.

  Here, the size of the display image displayed on the projection target 18 can be enlarged within the range allowed by the optical system of the projection unit 11 and the light source luminance. Normally, A4 size is displayed, and if necessary, A3 size, landscape, multi-image display, etc. are also possible. In either case, the irradiation position of the trapezoidal distortion correction pattern 100 described above or a marker to be described later is changed according to the display image size and shape.

  Next, in step S3, whether or not the distortion of the shape included in the rectangular shape of the trapezoidal distortion correction pattern 100 projected and imaged after the adjustment of the trapezoidal distortion (after distortion correction) exceeds a predetermined threshold value. Determine.

  If it is within the threshold value, the projection surface of the projection target 18 is regarded as a flat surface, and in the next step S4, in order to calculate the coordinates for each pixel of the handwriting input surface 36 described later, a handwriting described later. As shown in FIG. 5A, outside the outline of the projection surface of the size of the input surface 36, a predetermined shape of eight or more points (minimum, the total of four points at the four corners of the rectangle and the midpoint of each side) The first marker 102, which is a planar marker having 8 points), is projected and imaged by the compound eye imaging unit 14.

  Next, in step S5, the distance calculation unit 54 performs block matching using the first marker 102 as a corresponding point, and after block matching, calculates the three-dimensional coordinates by triangulating the first marker 102, as shown in FIG. 5A. The rectangle is divided into rectangular lattices, the three-dimensional coordinates from the compound-eye imaging unit 14 of each pixel of the projection plane corresponding to the handwriting input surface 36 described later are calculated by interpolation, and the handwriting input surface is stored in the distance data storage unit 56. The coordinates (two-dimensional data) of each pixel when 36 is viewed in a plane are stored. In this way, the positional relationship between the compound eye imaging unit 14 and a handwriting input surface 36 described later is determined.

  The first marker 102 is periodically displayed during use, or is always projected on the outside of a handwriting input surface 36 to be described later, whereby the relative position between the electronic handwriting input device 10 and the projection target 18 is determined. The measurement accuracy can be maintained even when the projection position is changed in the middle corresponding to the change in the relationship.

  On the other hand, if it is determined in step S3 that the rectangular distortion of the trapezoidal distortion correction pattern 100 after distortion correction exceeds a predetermined threshold, in step S6, as shown in FIG. The second marker 104, which is a marker for use, is projected into a projection plane corresponding to a handwriting input surface 36 to be described later. In the second marker 104, markers are arranged at lattice points that divide the rectangle formed by the first marker 102.

  Next, in step S7, the distance calculation unit 54 calculates the distances to all the first and second markers 102 and 104 on the projection surface corresponding to the handwriting input surface 36, which will be described later, respectively. Based on this, the projection surface is approximated to a three-dimensional curved surface to calculate the three-dimensional coordinates of the projection surface corresponding to the handwriting input surface 36 described later, and the handwriting input surface 36 described later is added to the distance data storage unit 56. Stored as coordinates (two-dimensional data) of each pixel when viewed in a plane.

  Next, in step S <b> 8, the system control unit 66 reads a plurality of image files composed of documents and images corresponding to the handwriting input surface 36 from the recording medium 40 and develops them as display data on the memory buffer 74. The name of the image file corresponding to the display data is displayed on the monitor 68 and projected and displayed as the handwriting input surface 36 through the projection unit 11. Here, for example, when a predetermined name is selected through an external device such as a mouse (not shown), or selected as indicated later by the indication point 112 of the indication means 110, a display corresponding to this name is displayed in step S9. The image data (handwriting input surface 36) is expanded on the memory buffer 74, and an image corresponding to the display data (handwriting input surface 36) is displayed on the monitor 68, and the image processing unit 70 and the projection unit 11 are displayed. Is displayed as a handwriting input surface 36 on the projection surface.

  FIG. 7 shows an example of an image (projected image) 106 displayed as the handwriting input surface 36 corresponding to the display data. On the projected image 106, a menu button 108 is displayed in addition to an object (image) such as a building or a ruled line. In FIG. 7, a sentence “The office buildings are...” Or a thick wavy free curve is made by handwriting input to be described later.

  The menu button 108 displays menus of various modes (processing modes) such as “erase”, “save”, “display video selection”, “keyboard display”, and “image editing”. When the “image editing” mode is selected, specific processing modes for image editing such as “enlargement” mode, “reduction” mode, and “sharpness enhancement” mode are displayed.

  Next, in step S10, an image of the compound eye imaging unit 14 that images the handwriting input surface 36 is input to the instruction unit recognition unit 62, and the instruction unit recognition unit 62 has a specific shape / image pattern such as a bend or a fingertip. The instruction means 110 is identified. The instruction means recognition unit 62 functions as a so-called pattern recognition unit.

  Here, when the pointing means 110 is a pen tip, an acute vertex is discriminated from the shape of the basic fingertip or the nail when the fingertip is used, and the tip is set as a pointing point (pointer) 112. Here, when recognizing the instruction unit 110, the recognition accuracy (identification accuracy) of the instruction point 112 is reduced due to the photographing of the handwriting input surface 36 that is the projection surface on which the image is projected. In this case, the projection image 106 of the handwriting input surface 36 when the instruction unit 110 is not present is captured and stored in advance, and the difference between the input images when the instruction unit 110 is captured is detected. By doing so, the indication point 112 can be detected more reliably.

  When the designated point 112 is recognized in step S10, the normal distance b from the handwriting input surface 30 (the pixel constituting the designated point) 112 is calculated in step S11, and the normal distance b is It is determined whether or not the handwriting input fixed distance range shown in FIGS. 8 and 9 is within the maximum distance x1. The maximum distance x1 is set to x1 = 2 [mm], for example.

  If it is within the maximum distance x1 (b <x1), in step S12, a processing mode other than the “erase” or “save” mode (here, “display video selection”, “keyboard display”, “image editing” described above). It is determined whether or not the pointing point 112 exists on the menu button 108 in the (equal processing mode).

  If the determination in step S12 is negative, the processing mode in the menu is executed or the processing mode is changed in step S13. For example, when the “enlargement” mode of the “image editing” mode is selected, as shown in FIG. 10, the image is made an enlarged image by moving the pointing point 112 pointing to the image in an oblique direction.

  On the other hand, if the determination in step S12 is negative, it is determined in step S14 whether or not the pointing point 112 exists on the menu button 108 in the “erase” mode.

  If the determination in step S14 is affirmative, in step S15, the input of the “erase” mode is confirmed, and the display dot on the pixel corresponding to the designated point 112 is erased.

  On the other hand, if the determination in step S14 is negative, in step S16, the input of writing by the designated point 112 is confirmed, and a display dot is additionally written (written) on the pixel corresponding to the designated point 112. Stored in the memory buffer 74.

  Next, in step S17, it is determined whether or not the pointing point 112 exists on the menu button 108 in the “save” mode.

  If the determination in step S17 is affirmative, the display dot state is reflected in the original display data (image data) file in step S18, and the display data developed on the memory buffer 74 is updated. Update the file. In this case, the display data stored in the recording medium 40 can also be updated (changed).

  After the update process in step S18 or when the determination in step S17 is negative, in step S19, the normal distance b of the pointing point 112 is within the maximum distance x2 of the boundary region distance range shown in FIGS. Is determined. The maximum distance x2 is, for example, x2 = 4 [mm].

  When the normal distance b is within the maximum distance x2 (b <x2), the display dot determination process (update process) after step S14, that is, the document process is continued. On the other hand, if the maximum distance x2 is exceeded (b ≧ x2), it is determined that the indication point 112 of the indication means 38 is separated from the handwriting input surface 36 by the user's intention, the input is canceled, and the step The process returns to the instruction means identification process in S10.

  In practice, the display dot on the handwriting input surface 36 determined in step S16 is changed to a preset color (such as red).

  When the distance of the designated point 112 fluctuates in the vicinity of the maximum distance x1 of the handwriting input fixed distance range, noise appears on the display dot. Therefore, the maximum distance x2 of the boundary area distance range is provided, and the maximum distance x1 and the maximum distance x2 The boundary has a hysteresis characteristic as a boundary area distance range. After the input is confirmed, the input is continued if the indication point 112 is not separated from the maximum distance distance range x2 of the boundary area distance range.

  As described above, since the display data is changed at the place where the input is confirmed, the handwritten image is updated to the projected image 106 in which the handwritten image is reflected on the handwriting input surface 36. When saving the updated image (changed image), the instruction point 112 is brought to the menu button 108 in the “save” mode so that the change is reflected in the image file of the original document. The data is stored in the buffer 74 and the recording medium 40.

  When “keyboard display” is selected, the keyboard is displayed as an image in the handwriting input surface 36, and so-called keyboard input is possible by confirming the input of the key at the instruction point 112. However, in the case of keyboard input, if the above-mentioned hysteresis characteristic is provided, it is often determined that the key is continuously pressed. Therefore, this determination is not performed and the input is performed from the non-input state (b ≧ x2). Only the change to definite x1 is detected.

  In FIG. 2 described above, an example in which the handwriting input surface 36 is projected on the table as the projection object 18 is shown. However, as shown in FIGS. 11 and 12, the projection object 18a such as a wall or a whiteboard is projected on the table 18a. The handwriting input surface 36 can be projected and the electronic handwriting input device 10 can be used. In the examples of FIGS. 11 and 12, fingers are drawn as the instruction means 110. The indication point (pointer) 112 can be discriminated from the shape of the fingertip or the shape of the nail.

  As described above, according to the electronic handwriting input device 10 according to the above-described embodiment, the projection unit 11 that projects the handwriting input surface 36 onto the projection target 18 and the handwriting input with the tip as the indication point 112. Instruction means 110, a compound eye imaging unit 14 that captures a scene image including the projected handwriting input surface 36, and a compound eye imaging unit 14 for each pixel of the captured handwriting input surface 36. A distance calculating unit 54 for determining the coordinates of the projected handwriting input surface for determining a distance, and an instruction means recognition unit 62 for recognizing the instruction point 112 of the instruction means 110 from the object scene image.

  In this case, the compound eye imaging unit 14 is configured integrally with the projection unit 11 and can be freely changed in angle by the universal joint 34 (34a, 34b). Then, a handwriting input confirmation unit 64 that confirms the input to the handwriting input surface 36 by the instruction point 112 of the instruction means 110 according to the distance b between the position of the instruction point 112 and the handwriting input surface 36 is provided. Prepare.

  Thus, since the compound eye imaging unit 14 and the projection unit 11 are configured to freely change the angle integrally, the projection surface of the projection target 18 on which the handwriting input surface 36 is projected is not a table or a desk. It becomes possible to use a wall or the like, and the restriction on the handwriting input surface 36 is reduced. Further, the input to the handwriting input surface 36 by the instruction point 112 of the instruction means 110 is confirmed by the handwriting input confirmation unit 64 according to the distance b between the position of the instruction point 112 and the handwriting input surface 36. Therefore, it is possible to reliably determine the input without using a pallet or the like.

  Here, the projection unit 11 includes a projection function unit of a first marker that projects the first marker 102 at a plurality of positions according to the handwriting input surface 36, and the distance calculation unit 54 includes the plurality of projected first markers. The distance to 102 is calculated, the three-dimensional coordinates of the handwriting input surface 36 are calculated based on the calculation result, and the coordinates of each pixel of the handwriting input surface 36 are stored in the distance data storage unit 56. Therefore, the position coordinates of the handwriting input surface 36 can be detected with higher accuracy.

  The projection unit 11 further includes a second marker projection function unit that projects the second marker 104 at a plurality of positions in the handwriting input surface 36, and the distance calculation unit 54 includes the plurality of projected second markers 104. And calculating the three-dimensional coordinates of the handwriting input surface 36 by approximating the handwriting input surface 36 to a three-dimensional curved surface based on the calculation result. Are stored in the distance data storage unit 56, so that the three-dimensional coordinates of the curved handwriting input surface 36 can be calculated with high accuracy.

  Here, the handwriting input confirmation unit 64 sets the distance between the handwriting input surface 36 and the position of the pointing point 112, the handwriting input confirmation distance range closest to the handwriting input surface 36, and the handwriting input confirmation distance. After providing a boundary area distance range from the maximum distance x1 to a fixed distance range (x2-x1) and determining that the position of the designated point 112 once entered the handwriting input fixed distance range, the boundary area distance Only when it is determined that the distance is longer than the maximum distance x2 of the range, the confirmation of the input is canceled. In this way, by providing the boundary region distance range, the influence of fluctuation of input determination is reduced, and unintended noise is prevented from being mixed.

  FIG. 13 is a system block diagram of an electronic handwriting input device 210 according to another embodiment of the present invention.

  FIG. 14 is an explanatory diagram showing a usage state of the electronic handwriting input device 210 of the example of FIG.

  In addition, in the electronic handwriting input device 210 shown in FIGS. 13 and 14, the same or corresponding components as those in the electronic handwriting input device 10 shown in FIGS. Detailed description thereof is omitted.

  The electronic handwritten input device 210 can display a stereoscopic video image (stereoscopic image, projected image) 206 together with a two-dimensional video image (projected image) 106 such as a document on the handwritten input surface 36.

  In this case, two projection units 11 and 12, each of which is a so-called projector, are prepared. A right-rotating circularly polarizing filter 211 and a left-rotating circularly polarizing filter 212 for stereoscopic viewing are integrally attached to the front surfaces of the projection units 11 and 12, respectively.

  The electronic handwriting input device 210 includes projection units 11 and 12 that are mechanically attached via universal joints 34 (34 a and 34 b) and a base unit 13, and the base unit 13 is connected to the projection units 11 and 12. And a compound eye imaging unit 14 composed of a first camera 21 and a second camera 22 that are integrally attached (fixed) via the camera. Therefore, the projection units 11 and 12 and the compound eye imaging unit 14 are configured to freely change the angle integrally through the universal joints 34 (34a and 34b).

  In the electronic handwriting input device 210, two images having different polarization directions are projected simultaneously from the projection units 11 and 12. The user can see the stereoscopic image 206 of the parallax image by seeing this through glasses equipped with polarizing filters having different polarization directions on the left and right.

  The image 106 such as a document is displayed as a normal video by outputting the same video to both the projection units 11 and 12, and the specific image is projected as a video including left and right parallax to thereby display one projection plane. Thus, the normally projected image (planar image) 106 and the stereoscopic image 206 can be displayed simultaneously.

  In the example of FIGS. 13 and 14, for example, the indication point 112 of the indication means 110 is, for example, near the center of the stereoscopic image 206 that is a subject as a predetermined distance from the handwriting input surface 36 in the stereoscopic image 206. Is detected, it is possible to send a signal to the image processing unit 70 in accordance with the movement of the designated point 112 and to translate or rotate the stereoscopic image 206 using a preselected menu.

  The operation of the electronic handwriting input device 210 is performed except that the maximum distance x1 of the handwriting input fixed distance range and the maximum distance x2 of the boundary area distance range are set as described below in the flowchart shown in FIG. Is the same.

  That is, the handwriting input confirmation unit 64 (FIG. 1) calculates the parallax amount α by integrating the horizontal distances of the portions having similar patterns in the left and right images projected from the projection units 11 and 12. Assuming that k1 and k2 (k1 <k2) are predetermined fixed values (constants) and the parallax amount α is generally large, the stereoscopic image 206 approaching the user is displayed. The maximum distance x1 of the range is set as x1 = α × k1, and the maximum distance x2 of the boundary region distance range is set as x2 = α × k2.

  According to another embodiment shown in FIGS. 13 and 14, the projection units 11 and 12 having the circularly polarizing filters 211 and 212 having different rotations are provided, and the stereoscopic image 206 can be displayed on the handwriting input surface 36. It is said. In this case, the user can input handwriting or the like on the handwriting input surface 36 while viewing the stereoscopic image 206 or the like by using the polarized stereoscopic glasses.

  Note that the present invention is not limited to the above-described embodiment, and it is needless to say that various configurations can be adopted based on the contents described in this specification.

1 is a system block diagram of an electronic handwriting input device according to an embodiment of the present invention. It is explanatory drawing which shows the external appearance and use condition of an electronic handwritten input device. It is a flowchart with which operation | movement description of an electronic handwritten input device is provided. It is explanatory drawing of the keystone distortion correction pattern for distortion correction. FIG. 5A is an explanatory diagram of a planar marker, and FIG. 5B is an explanatory diagram of coordinate determination. It is explanatory drawing of the marker for curved surfaces. It is explanatory drawing which shows the example of the image displayed on the handwriting input surface. It is explanatory drawing of the input determination determination reference | standard with respect to the normal distance of an indication point. It is another explanatory drawing of the input decision discrimination standard with respect to the normal distance of an indication point. It is explanatory drawing of image editing (enlargement). It is explanatory drawing at the time of projecting a handwriting input surface on to-be-projected objects, such as a white board, and using an electronic handwriting input device. FIG. 12 is an explanatory diagram of an input confirmation criterion with respect to the normal distance of the indication point in the usage form of the example of FIG. 11. It is a system block diagram of the electronic handwritten input device which concerns on other embodiment of this invention. It is explanatory drawing which shows the use condition of the electronic handwritten input device of the example of FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10,210 ... Electronic handwriting input device 11,12 ... Projection part 14 ... Compound eye imaging part 18 ... Projection object 21 ... 1st camera 22 ... 2nd camera 34, 34a, 34b ... Universal joint 36 ... Handwriting input surface 54 ... Distance calculation unit 62 ... Instruction unit recognition unit 64 ... Handwriting input confirmation unit 102 ... First marker 104 ... Second marker 106 ... Projection image 110 ... Instruction unit 112 ... Indication point 206 ... Stereoscopic image

Claims (5)

A projection unit that projects a handwriting input surface onto a projection object, instruction means for performing handwriting input with the tip as an instruction point, and compound-eye imaging that captures an object scene image including the projected handwriting input surface A distance calculating unit that obtains a distance from the compound-eye imaging unit for each pixel of the imaged handwriting input surface that has been imaged, obtains a coordinate of the projected handwriting input surface, and the instruction means from the object scene image An electronic handwriting input device comprising an instruction means recognition unit for recognizing the indication point of
The compound eye imaging unit is configured to freely change the angle integrally with the projection unit,
A handwriting input confirming unit for confirming input to the handwriting input surface by the pointing point of the pointing means according to a distance between the position of the pointing point and the handwriting input surface. Electronic handwriting input device. The electronic handwritten input device according to claim 1,
The projection unit further includes a first marker projection function unit that projects a first marker at a plurality of positions according to the handwriting input surface,
The distance calculating unit calculates distances to the plurality of projected first markers, and calculates three-dimensional coordinates of the handwriting input surface based on a calculation result. Input device. The electronic handwritten input device according to claim 1,
The projection unit further includes a second marker projection function unit that projects a second marker at a plurality of positions in the handwriting input surface,
The distance calculation unit calculates the distances to the plurality of projected second markers, and approximates the handwriting input surface to a three-dimensional curved surface based on the calculation result, whereby the handwriting input surface An electronic handwriting input device that calculates three-dimensional coordinates. The electronic handwritten input device according to claim 1,
The handwriting input confirmation unit is configured to determine a distance between the handwriting input surface and the instruction point, a handwriting input confirmation distance range closest to the handwriting input surface to determine handwriting input, and the handwriting After providing a boundary area distance range of a certain distance range from the maximum distance of the input fixed distance range, and after determining that the position of the indicated point has entered the handwriting input fixed distance range, the boundary area distance range An electronic handwriting input device that cancels the confirmation of handwriting input only when it is determined that the distance is farther than the maximum distance. The electronic handwritten input device according to claim 1,
An electronic handwriting input device comprising a plurality of the projecting units and capable of displaying a stereoscopic image on the handwriting input surface.

Images