JPH08101739A - Storage pen unit - Google Patents

Abstract

PURPOSE: To develop the concepts of a man-machine interface and improve the availability by equipping a case, freely attached to and detached from a writing pen, with a means which picks up and stores the track of a character, etc., drawn with the writing pen. CONSTITUTION: The storage pen unit 1 is mounted on the writing pen 2 on the market with pen grip parts 7 formed at the tip parts of clip arms 6 and 6 fitted to the lower part of a columnar case 5 when writing operation is done by using the writing pen 2, and the strokes of characters, figures, etc., drawn on a writing surface 4 with the writing pen 2 are stored. For the purpose, a lens 9 and a CCD 8 for picking up an image of the strokes are provided in the storage pen unit 1, and image data detected by the CCD 8 are compressed in time series by a microcomputer 10 and stored in a memory 11. At this time, information on the generation time of the image data is also stored. Further, a gauge member 14 in the shape of a columnar body makes the distance between the lower end surface 13 of the case 5 and the writing surface 4 constant.

Description

Detailed Description of the Invention

[0001]

The present invention relates to a fountain pen, ballpoint pen,
The present invention relates to a storage pen unit that stores information about the loci of characters, figures, etc. written using commercially available writing pens such as pencils and writing brushes.

[0002]

2. Description of the Related Art Conventionally, input devices such as a keyboard and a mouse have been used to input information such as characters and figures into an electronic information processing apparatus represented by a computer. In addition, recently, people-friendly HMI (human m)
In the midst of being advocated as an "acade interface", a digitizer-type pen input means that can be easily input without any need for proficiency in input operation has been put into practical use as an input means for an information device such as an electronic notebook.

[0003]

However, in the pen input means using a digitizer or the like, a plastic pen that does not have a writing function for ordinary paper is used instead of a usual writing pen such as a fountain pen or a ballpoint pen. A special pen having a signal oscillator for light or ultrasonic waves at its tip is used. In addition, all of the digitizer type pen input means require a plate-shaped tablet for recognizing the position of the pen tip. Thus, conventionally,
There are various input means for computer processing such as keyboard, mouse, digitizer, etc., but all of them have special pen structure different from ordinary writing pen, or key input arranged for easy computer processing. It was a method, not an input method using a normal writing pen. Further, in the case of using this type of conventional input means, it is necessary to learn the key arrangement and to perform the key operation for converting the characters written on the tablet, and it is necessary to become familiar with the operation. However, it was not in line with the above HMI concept.

In order to realize the concept of HMI, the present applicant has already filed Japanese Patent Application No. 4-70559 and US Pat.
No. 215, 397, etc., a means for accommodating a writing core body such as a ballpoint pen tip in a case body, a means for detecting a locus of characters, figures, etc. described by the core body, and a means for storing the locus data. We have proposed a new writing device with a built-in.

The present invention has been made for the purpose of further developing the contents of the above proposals and providing an extremely useful input means capable of further developing the above-mentioned concept of HMI.

[0006]

A memory pen unit according to a first aspect of the present invention relates to an image pickup means for picking up a locus of a character, a figure, etc. drawn by a writing pen, and an image picked up by the image pickup means. The storage means for storing the information of (1) is provided in a case detachably attached to the writing pen.

A storage pen unit according to a second aspect of the present invention is such that the storage means according to the first aspect stores time information necessary for reproducing the stroke order in addition to the information about the image. is there.

A storage pen unit according to a third aspect of the present invention is provided with a focusing means for focusing an image formed on an image pickup device constituting the image pickup means according to the first aspect. .

According to a fourth aspect of the present invention, there is provided a memory pen unit according to the third aspect, in which the focusing means emits light directed in a predetermined direction from at least two points on the writing surface. It will be.

According to a fifth aspect of the present invention, in the memory pen unit, the focusing means according to the third aspect measures the distance between a predetermined position of the case and the writing surface, and, depending on the measured distance. And means for adjusting the automatic focal length.

A memory pen unit according to a sixth aspect of the present invention is such that the focusing means according to the third aspect comprises a gauge member for adjusting a distance between a predetermined position of the case and the writing surface.

A storage pen unit according to a seventh aspect of the present invention is such that the memory constituting the storage means according to the first aspect is detachably attached to a case.

According to a eighth aspect of the present invention, there is provided a storage pen unit, wherein a power source unit for driving the image pickup means and the storage means according to the first aspect is a solar cell integrally formed with the case. Is.

A storage pen unit according to a ninth aspect of the present invention is a rechargeable battery in which a power source section for driving the image pickup means and the storage means according to the first aspect is detachably configured from the case. is there.

[0015]

According to the first aspect of the present invention, since it is attached to the writing pen familiar to the writer and the written characters, figures, etc. are imaged and stored as data, it is convenient to carry and convenient to use. Further, since it is detachably configured, it is possible to easily change the normal writing operation that is not stored and the storage operation.

According to the second aspect, the storage means stores the time information necessary to reproduce the stroke order in addition to the information about the image, so that the stroke order drawn by the writer can be reproduced. Become.

According to the third aspect, since the focusing means is provided, it is possible to obtain a constant focused image on the image pickup device.

The focusing means according to claim 4 comprises a light irradiating means for irradiating the writing surface of the writing pen with a directional light from a predetermined portion of the case, and the main memory pen is located at a position where these two directional lights are aligned. Since it is preferable to mount the unit and the directional light is used, the degree of alignment can be increased.

Since the focusing means according to the fifth aspect uses a means for adjusting the automatic focal length, the main memory pen unit can be attached to any position of the writing pen within the range of the focusing.

Since the focusing means according to the sixth aspect has a simple structure composed of a gauge member for adjusting the focal position, the focusing is simple and the cost can be reduced.

According to a seventh aspect of the present invention, the memory forming the storage means is detachable from the case.
When reproducing the handwriting, only this memory may be removed and set in the reproducing device. Further, when the storage capacity becomes insufficient, it is only necessary to remove the memory and set a new memory, and it is not necessary to remove the main storage pen unit once focused and attached from the writing pen.

According to the eighth aspect, since the power source unit for driving the image pickup unit and the storage unit is a solar cell integrally formed with the case, it is necessary for driving by using illumination in a room or the like. Since power can be supplied, it is possible to prevent memorization failure due to battery exhaustion.

According to a ninth aspect of the present invention, the power source unit for driving the image pickup unit and the storage unit is a rechargeable battery detachably attached to the case. Therefore, for example, the contents of the memory are reproduced by the reproducing device. It is convenient because the batteries can be charged at the same time.

[0024]

Embodiments of the memory pen unit according to the present invention will be described below with reference to the drawings. FIG. 1 is a perspective view showing a first embodiment of a memory pen unit according to the present invention.
The memory pen unit 1 of this example is mounted on the side of the pen tip 3 that does not disturb the hand when writing with a writing pen 2 such as a commercially available fountain pen or ballpoint pen. It stores the handwriting such as characters and figures drawn on the surface 4. In this example, the case 5 is formed into a cylindrical body, and the writing pen 2 is gripped and mounted by the pen gripping portions 7 formed at the tips of the clip arms 6 and 6 attached to both sides of the lower portion of the case 5. .

FIG. 2 is a block diagram showing the internal structure of the memory pen unit 1. In FIG. 2, the storage pen unit 1 is provided with a CCD 8 for capturing the handwriting drawn on the writing surface 4 and storing it as image data.
A lens 9 for forming an image on D8 is arranged on the front side of the CCD 8. The image data detected by the CCD 8 is sequentially input to the microcomputer 10 in time series. Then, in the microcomputer 10, the image data sent according to the time series is data-compressed to the extent that it can be reproduced by a reproducing apparatus described later and stored in the memory 11. At this time, the generation time information of the sent image data is also stored. Reference numeral 12 is a power supply unit, and for example, a rechargeable secondary battery is used to drive a circuit in the storage pen unit 1. Of course, a solar cell that can be charged by indoor lighting can also be used. Further, in this example, the memory 11 and the power supply unit 12 are detachable from the case 5.

Here, in the memory pen unit 1 of this example,
Since it can be attached to any commercially available writing pen 2, the mounting position is not constant due to the overall structure of the writing pen 2, the size of the pen tip 3, etc., so the focus of the image data detected by the CCD 8 It is necessary to make adjustments.
For this reason, in FIG. 1, as the simplest focusing means, a gauge member 14 having a shape such as a pillar body so that the memory pen unit 1 can be mounted with a constant distance between the lower end surface 13 of the case 5 and the writing surface 4. Are prepared separately. In addition to this, a tape measure (not shown) having the above-mentioned distance scaled in the height direction may be stored in the case 5, and the tape measure may be pulled out to adjust the height.

Further, in addition to using the gauge member 14 described above, an automatic or semi-automatic focusing method as shown in FIG. 3 can be used. That is, in FIG. 3, a beam is directed to both ends of the lens 9 arranged in the lower part of the case 5 in the direction of the optical axis of the lens 9 (in this example, the direction from the center of the lens 9 to the vertically downward direction). Irradiating red L
The ED and its optical system are respectively arranged and focused so that the spot 15 of the light beam on the writing surface 4 is aligned with one point. The writing surface indicated by the broken line is when the focus is out of focus. In this case, the two spots 16, 1
You can see 7. In addition, although not shown, a lens 9 and a writing surface 4 are used by using a distance measuring sensor using triangulation or the like.
You can also measure the distance to and focus on.

Then, focusing is performed as described above, and the memory pen unit 1 is attached to the writing pen 2 so as to be at a predetermined height position from the writing surface 4 which is determined in advance, or
Alternatively, as shown in FIG. 3, an adjusting mechanism 18 for moving the lens 9 in the front-back direction so that the lens is in focus is provided like the focusing mechanism of the camera, and the memory pen unit 1 is used for the writing pen 2.
It is advisable to perform fine adjustment by means of the adjusting mechanism 18 and then focus the lens once it has been mounted.

FIGS. 4 to 10 show a second storage pen unit.
It is a perspective view showing an 8th example, respectively. In the memory pen units 19 to 25 shown in these second to eighth embodiments, unlike the lens 9 in the above-mentioned first embodiment, the optical axis of the optical system is perpendicular to the writing surface 4 from a line perpendicular to the writing surface 4, as shown in FIG. As shown in FIG. In particular, since the writing pen 2 has a different inclination angle due to individual differences among writers, the angle θ is not positioned in the vertical direction as in the first embodiment, and therefore the image detected by the CCD 8 is elliptical. Since the shape is distorted, there is a problem that the handwriting cannot be accurately reproduced.To solve such a problem, the lens built in the following examples has a tilt direction from the optical axis and a direction opposite to this tilt direction. It is preferable to use a telecentric lens having a characteristic that the image changes with equal magnification. Of course, the lens is not limited to the telecentric lens, and may be a wide-angle lens or the like.

The memory pen unit 19 of the second embodiment shown in FIG. 4 has the same basic structure as that of the first embodiment,
Since the case 19a is attached to the opposite side of the writing pen 2 in the direction of inclination, and the case 19a is attached to the side of the writing pen 2 so that the hand holding the writing pen 2 holds the case 19a.
Since it does not come into contact with a, writing operation is not hindered and writing is easy.

The memory pen unit 20 of the third embodiment shown in FIG. 5 is mounted in close contact with the writing pen 2 from the central portion to the lower portion thereof, and holds the lower end portion of the writing pen 2 such as a writing brush. It is convenient when writing down.

The memory pen unit 21 of the fourth embodiment shown in FIG. 6 is mounted on the upper part of the writing pen 2 and the part for holding the hand to be written is wide open. In this case, the lens 21a is used. Light passing through the lens 21a may be introduced into the case 21c by the optical fiber cable 21b.

The memory pen unit 22 of the fifth embodiment shown in FIG. 7 has a case 22a having a hollow cylindrical body structure and a writing pen 2.
It is externally inserted and fixed by the adjusting screw 22b.
In this case, if the area of the lower end surface of the case 22a is increased, the diameter of the entire case 22a becomes large and it becomes difficult to grip. Therefore, for example, a lens system and an optical fiber are used on the lower end peripheral surface of the case 22a to display an image on the CCD 8. Should be taken into.

The memory pen unit 23 of the sixth embodiment shown in FIG. 8 has a case 23a which constitutes a grip portion.
In this case, since the writing pen 2 can be held in the vertical direction, the pen tip 3 of the writing pen 2 does not greatly tilt in the oblique direction, and the portion of the pen tip 3 does not interfere and the imaging range can be widened. .

The memory pen unit 24 of the seventh embodiment shown in FIG. 9 is mounted on the fingertip so that the penpoint 3 of the writing pen 2 is imaged. It is convenient to have a focusing function.

The memory pen unit 25 of the eighth embodiment shown in FIG. 10 is a case where a wide-angle lens 25b capable of imaging the entire writing surface 4 is provided in the pen cap 25a. In this case, in each of the above-described embodiments, it is necessary to sequentially input the image of the CCD 8 and reproduce it in order to capture the partial area around the pen tip 3 of the writing pen 2, but in the case of this example. ,
Since the entire writing surface 4 can be imaged, it is also possible to take an image collectively after the writing work is completed, in addition to the case of sequentially inputting.
Of course, in the latter case, since the time information is not stored, the stroke order cannot be reproduced, but only the handwriting is reproduced.

FIG. 11 shows the memory pen units 1 and 1 described above.
9 is a block diagram showing a configuration example of a reproducing device 26 that removes the memory 11 storing the handwriting by 9 to 25 from the case body 5 and reproduces the handwriting in the writing order drawn with the writing pen 2.
In the reproducing device 26, the memory 11 and the power supply unit 12 removed from the storage pen unit are set, and the compressed image data (writing data) stored in the memory 11 is read out via the interface device 27, and the computer 2
Reproduction processing is performed by the reproduction software 29 in the computer 8. The interface device 27 also accommodates a charging device for the power supply unit 12.

Thus, the above memory pen units 1, 1
In 9 to 25, the storage procedure in the memory 11 is as follows. For example, it is assumed that the pen tip 3 of the writing pen 2 writes a character, for example, L of the alphabet on the paper surface,
The CCD 4 sequentially captures images within the field of view at a fixed sampling time. As shown in FIG. 12, from the upper part of the letter L to the lower left part, draw a first straight line with a gentle slope,
Then, a second straight line with a gentle slope is drawn from the lower left to the lower right. Now, assuming that the position of the pen tip 3 moves to P ₁ , P ₂ , ..., P ₆ , ... For each sampling time, the field of view of the CCD 4 also moves to S ₁ , S ₂ , ..., S ₆ ,. Then, in the memory 11, images S ₁ , S ₂ , ..., S ₆ , ... Of the same size as shown in FIG. 13 are sequentially stored as image data. As described above, the partial images having a constant size are continuously stored, and thus the handwriting as a whole is stored together with the time information.

Next, an example of reproduction of the handwriting stored as image data in the memory 11 of the storage pen units 1, 19 to 25 will be described. The reproduction processing here is performed by "cell", "act", and "scene (s)".
cene) ”and“ page ”are used to construct a hierarchical structure. The “cell” is a partial image composed of fixed-length pixels, and is a basic smallest unit image. _{S 1,} S 2 shown in FIG. 13, ..., _{S 6,} ... corresponds to this.

The term "act" refers to an image formed by superimposing a series of cells which are temporally continuous. Here, a series of cells means a series of cells that are as large as possible as long as the positional relationship between consecutive cells is known, that is, the relative position to the immediately preceding cell is known. A "scene" is an image composed of a plurality of acts whose positional relationship is clear. As described above, which represents a semantic unity, when the positional relationship between consecutive cells is not known, those cells become a part of separate acts. However, there are cases where the positional relationship between these acts is known, and such acts are summarized as a scene.

A "page" is an image representing a unit of work composed of a series of temporally continuous scenes, which is called a page. The scenes that make up one page have no overlap. Further, when the writing pen 2 is moved in the air to restart writing at a position separated from the final image of the previous scene, a scene having no image overlap (common portion) is detected. At any time, it is possible to know in which direction the final handwriting of the final image of the previous scene was moving, and moreover, the time elapsed information during that time is known. Therefore, the speed and distance of the spatial movement are obtained from these information, and there is no common part. The situation of mutual arrangement between two scenes can be judged and reproduced. Of course, the user may explicitly give a scene having no image overlap while looking at the display screen of the playback device 26 or the like. It should be noted that the processing here uses an xy orthogonal coordinate system in which the origin is at the upper left in cells, acts, scenes, and pages, and x increases in the right-hand direction and y increases in the downward direction. However, this method does not depend on the coordinate system. Further, the smallest quadrangle surrounding all the black pixels in each image is called a Bounding Box (BB).

Pixels forming the cell have BLAC as a density.
Take K or WHITE. A method of expanding this to multiple gradations will be described later. In addition, the densities of pixels that make up an act, a scene, and a page are indices that take the possibility that the pixel is black as a non-negative integer, and when the value is large, it is highly possible that the pixel is black. Represent The white density is 0 in the act, the scene, and the page. cell,
FIG. 14 shows an example of the relationship between acts, scenes, and pages.

The overall schematic procedure of the reproduction program is shown in the flowchart of FIG. First, in the step, the cell data stored in the memory 11 is sequentially read, and in the step, the cell and the act are matched,
Build an act. The construction method will be described below with reference to FIG. That is, in the execution of matching in a step, the initial act act _j (0 ≦ j)
Is a blank area, and the act construction is in cell cell _i
This is performed by superimposing (0 ≦ i) on the appropriate position of act act _j . The first cell, cell _0, is act a
It is placed at the appropriate position in ct ₀ . In the next cell cell ₁ , the handwriting appearing in the cell cell ₀ , that is, the handwriting already existing in the act act ₀ (hereinafter referred to as common handwriting) and the cell cell ₀ are fetched, and then the cell cell
Both the new handwritings written before the capture of l ₁ are reflected. The handwriting matching is performed between the cell cell ₁ and the act act ₀ , and the common handwriting of the cell cell ₁ is the position (x ₁ , where the common handwriting overlaps with the act act ₀ without contradiction).
y ₁ ) is obtained, and the cell cell ₁ is superposed on the position (x _i , y _i ) of the act act ₀ .

Similarly, until the cell cell ₁ where the common handwriting does not overlap appears, the common handwriting of each cell cell _i is superposed on the position (x _i , y _i ) at which the handwriting overlaps without any contradiction. Here, when the common handwriting of the cell cell ₁ does not sufficiently overlap with the act act _j , the same work is performed on the cells after the cell cell ₁ for the new act act _{j + 1} .

On the other hand, when constructing an act from a cell, if the positional relationship between the cell and the act is lost, subsequent cells perform reproduction with the new act as the target. At this time, part of the handwriting written in the previous act may appear in the new act, and the positional relationship between the acts may become clear. In such a case, the scene is constructed by superposing the act on the scene by performing matching between the scene constituted by the previous act and the act and detecting a common handwriting. In FIG. 16, a thin dotted line indicates a matching density of 1, a dark concentrated line indicates a matching density of 2, and a thick solid line indicates a matching density of 3 or more. As the density value increases, the matching frequency increases. It shows that there are many.

Next, in the step shown in FIG. 15, the act and the scene are matched. This matching is performed to find the relative positional relationship between the act and the scene.
This is the same matching method as when constructing an act from a cell. That is, an act has a part of the handwriting actually written in the act and part of the handwriting of the character written in the previous act. When the pen movement vector is obtained from the position data of the pen obtained when reproducing the act from the cell, and the handwriting actually written in this act is removed, the existing handwriting previously written remains. Since this existing handwriting exists in the scenes that have been reproduced so far, the positional relationship between the scene sharing the existing handwriting and the act is detected, and the act and the scene are superimposed.

Further, matching is performed in a predetermined matching area defined in advance between the act and the scene, and a position where the act and the scene can be superposed with each other without any contradiction is obtained. Then, all the acts are reproduced on the scene. However, the act may have handwriting common to a plurality of scenes, and the relationship between the scenes whose positional relationship is unknown may be found. In this case, the act and a plurality of scenes are superposed at corresponding positions to be combined into a scene with a small scene number. All other scenes are left blank.

Finally, a page is constructed in the steps shown in FIG. 15 based on each scene constructed as described above. That is, a page is composed of a series of scenes that represent a unit of work. Also, since the acts whose relative positional relationship is known are summarized as a scene,
The positional relationship between pages is unknown. Therefore, the page break is explicitly given by the user by a mechanism separately prepared as hardware or a gesture, or from the time axis, that is, after starting the use of the pen storage unit and ending the example work. Until the power is turned off, use the method of judgment.

Thus, in the reproducing device 26, based on the time-series image data stored in the memory 11, according to the handwriting written with the writing pen 2 on the basis of the image data reproduced with the passage of time. The characters, figures, etc. of can be reproduced. Then, for example, if only the matching density of 1 is reproduced in time series, the reproduction can be performed in the written stroke order.

As shown in the internal structure of the computer 24 in FIG. 11, when the handwriting is reproduced,
Using the handwriting, character recognition software, figure recognition software, communication software, etc. are used for word processing and CAD.
Can be input to. Further, in particular, in the second to seventh embodiments, the image captured by the CCD 8 is
The pen tip 3 portion of the writing pen 2 is located at the center of the visual field and becomes a noise source as a black dot, but this is constantly generated at a fixed position of the image at a fixed size, so this portion is a predetermined erase program. It is automatically erased by.

[0051]

As described above, according to the present invention, the locus of characters, figures, etc. can be stored as it is by mounting it on a commercially available writing pen. Further, since the written content can be reproduced in time series simply by setting the memory in the reproducing device, the history of thoughts can be stored.
In particular, since the writing pen can be attached to an item that the writer uses on a daily basis, anyone can easily use it and no proficiency in using it is required. Therefore, it is extremely useful according to the concept of HMI.

[Brief description of drawings]

FIG. 1 is a perspective view showing a state in which a storage pen unit of a first embodiment according to the present invention is attached to a writing pen.

FIG. 2 is a block diagram illustrating an internal configuration of the storage pen unit.

FIG. 3 is a schematic diagram for explaining a focusing method.

FIG. 4 is a perspective view showing a structural example of a storage pen unit of a second embodiment.

FIG. 5 is a perspective view showing a structural example of a storage pen unit of a third embodiment.

FIG. 6 is a perspective view showing a structural example of a storage pen unit of a fourth embodiment.

FIG. 7 is a perspective view showing a structural example of a storage pen unit of a fifth embodiment.

FIG. 8 is a perspective view showing a structural example of a storage pen unit of a sixth embodiment.

FIG. 9 is a perspective view showing a structural example of a memory pen unit of a seventh embodiment.

FIG. 10 is a perspective view showing a structural example of a memory pen unit of an eighth embodiment.

FIG. 11 is a block diagram illustrating a configuration of a playback device.

FIG. 12 is a conceptual diagram for explaining a handwriting detection method.

13 is a diagram showing an image detected by a CCD corresponding to the handwriting shown in FIG.

FIG. 14 is a diagram illustrating the relationship among cells, acts, scenes, and pages in the playback device.

FIG. 15 is a flowchart illustrating a matching procedure of the playback device.

FIG. 16 is a diagram illustrating an example of act construction.

[Explanation of symbols]

1, 19 to 25 ... Memory pen unit 8 ... CCD 2 ... Writing pen 9 ... Lens 3 ... Nib 10 ... Microcomputer 4 ... Writing surface 11 ... Memory 5 ... Case 12 ... Power supply section 6 ... Clip arm 26 ... Playback device 7 … Gripping unit 28… Computer

Claims (9)

[Claims] 1. An image pickup means for picking up a locus of a character, a figure, etc. drawn by a writing pen, and a storage means for storing information about an image picked up by the image pickup means are detachably attached to the writing pen. A memory pen unit that is equipped in a case. 2. The storage pen unit according to claim 1, wherein the storage means stores time information necessary to reproduce the stroke order in addition to the information about the image. 3. The memory pen unit according to claim 1, further comprising a focusing unit for focusing an image formed on an image pickup device constituting the image pickup unit. 4. The memory pen unit according to claim 3, wherein the focusing means comprises a light irradiating means for irradiating light directed in a predetermined direction from at least two points on the writing surface. 5. The focusing means comprises means for measuring a distance between a predetermined position of the case and the writing surface, and means for adjusting an automatic focal length according to the measured distance. Memory pen unit described. 6. The storage pen unit according to claim 3, wherein the focusing means is a gauge member for adjusting a distance between a predetermined position of the case and the writing surface. 7. The storage pen unit according to claim 1, wherein the memory constituting the storage means is detachably attached to the case. 8. The storage pen unit according to claim 1, wherein a power supply unit for driving the image pickup unit and the storage unit is a solar cell integrally formed with the case. 9. The storage pen unit according to claim 1, wherein a power supply unit for driving the image pickup unit and the storage unit is a rechargeable battery detachably attached to the case.