JP2022020703A - Handwriting device and speech and handwriting communication system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a convenient handwriting device capable of simultaneously transmitting speech information and locus information to a communication counterpart.

SOLUTION: A pen-type device that draws a locus on a medium on which a dot pattern is formed includes: imaging means for imaging the dot pattern; analyzing means for performing code analysis on the imaged dot pattern by the imaging means to output locus information and/or a code value that represents the record of coordinate information obtained by at least coordinate values; speech obtaining means for obtaining speech information; and transmitting means for transmitting at least one of the speech information, the locus information and the code value to a communication device. The coordinate values and the code value are defined in the writing region of the medium on which the dot pattern is formed, and the code value includes a region identifying code to identify the writing region. The analyzing means outputs the locus information in such a way that the region identifying code is contained therein.

SELECTED DRAWING: Figure 17

COPYRIGHT: (C)2022,JPO&INPIT

Description

The present invention relates to a pen-type device having a telephone function and a dot pattern reading function.

The present invention also relates to a pen-type device used in conjunction with a wearable device.

The present inventor has made many epoch-making inventions using a dot pattern.

In these inventions, there is also a technique of connecting a scanner that reads a dot pattern and a mobile phone, reading the dot pattern with the scanner, and outputting information corresponding to the dot pattern from the mobile phone. There is also a mobile phone with a built-in scanner (Patent Document 1).

International Publication WO 2004/029871 Issue

However, the scanner having a telephone function is not disclosed.

In recent years, wearable devices such as Apple Watch and GOOGLE GLASS (registered trademark) are becoming widespread.

The present invention has been made in view of these points, and it is a technical subject to provide a pen-type device having a telephone function and a dot pattern reading function.

Another technical issue is to provide a pen-type device that allows a wearable device to perform various operations by interlocking with the wearable device.

Furthermore, although E-learning is widespread worldwide in the field of education, at present, it is the mainstream to record and distribute lectures by teachers on video. Video recording requires a considerable cost from recording location, cameraman arrangement to editing. In addition, most of the communication between teachers and students is centered on conversation with common teaching materials (including digital contents). Some of them have been corrected, but they cannot be dealt with interactively immediately. Ideally, teachers should be able to easily handwrite lessons and provide immediate voice and handwriting guidance to students' question answers and questions. Also, in the business field, telephone conferences and video conferences are widespread, but in order to share figures, photos, illustrations, etc. and proceed with the conference, participants should use the images taken by the camera as a basis. In order to deepen understanding not only by conversation but also by handwriting, there are major problems in the accuracy, convenience, and cost of touch panels and pen tabs. In addition, social network chat is spreading at a tremendous pace, and there are high expectations for handwritten chat, but at present, there is no inexpensive and easy-to-use system.

(1) The handwriting device according to the present invention is a handwriting device for drawing a locus on a medium on which a dot pattern is formed, and images a coordinate value and a code value, or the dot pattern in which a code value is defined. The image pickup means, the analysis means for outputting the locus information and / or the code value consisting of the history of the coordinate information obtained from at least the coordinate values by code-analyzing the dot pattern captured by the image pickup means, and the voice information. The writing area of the medium provided with the voice acquisition means for acquisition, the voice information, and the transmission means for transmitting at least one of the locus information or the code value to the communication device, and the dot pattern is formed, is described in the writing area. A coordinate value and a code value are defined, the code value has an area specifying code for specifying the writing area, and the analysis means includes the area specifying code in the locus information and outputs the code value. do.

(2) The handwriting device according to the present invention is a handwriting device for drawing a locus on a medium on which a dot pattern is formed, and images a coordinate value and a code value, or the dot pattern in which a code value is defined. An image pickup means, an analysis means for outputting locus information and / or a code value consisting of a history of coordinate information obtained from at least the coordinate values by code analysis of the dot pattern captured by the image pickup means, and the voice information. A coordinate value and a code value are defined in a writing area of a medium having a transmission means for transmitting at least one of the locus information or the code value to a communication device, and the dot pattern is formed. The code value has an area specifying code for specifying the writing area, the analysis means includes the area specifying code in the locus information, and the communication device acquires voice information. And.

(3) Further, the handwriting device or the communication device may further be provided with a synchronization means that adds information defining the passage of time to the locus information and synchronizes with the voice information.

(4) Further, the communication device may be a communication device of the other party used by a third party.

(5) Further, the communication device is a communication device used by itself, and the transmission means transmits the voice information and the locus to the other party communication device used by a third party via the communication device. You may send at least one of the information or the code value.

(6) Further, the other party communication device may be specified by the locus information.

(7) Further, a storage means for storing at least one of the locus information and the voice information is further provided in either the handwriting device or the communication device, and the transmission means provided in the communication device or the transmission is provided. The means may transmit at least one of the voice information and the locus information stored in the storage means as a file.

(8) Further, at least one of the handwriting device and the communication device is further provided with a display means for displaying at least the locus information, which is either built-in or externally connected. May be good.

(9) Further, the display means may output the voice information in synchronization with the locus information.

(10) Further, at least one of the handwriting device and the communication device may further be provided with a voice output means for outputting the received voice information.

(11) Further, a voice receiving device for receiving voice information from the other party communication device may be further provided in at least one of the handwriting device and the communication device.

(12) Further, at least one of the handwriting device and the communication device may further be provided with the track receiving device for receiving the track information from the destination communication device according to any one of claims 4 to 6. good.

(13) Further, the locus information further drawn by the handwriting device with respect to the locus information may be transmitted to the other party communication device or the communication device according to any one of claims 4 to 6.

(14) Further, the communication device may be a smartphone or a PC.

(15) Further, the other party communication device may be a smartphone or a PC.

(16) The voice / handwriting communication system according to the present invention is a handwriting device for drawing a locus on a medium on which a dot pattern is formed, and the coordinate value and the code value, or the dot pattern in which the code value is defined is defined. An image pickup means for capturing an image, and an analysis means for outputting locus information and / or a code value consisting of a history of coordinate information obtained from at least the coordinate values by code-analyzing the dot pattern captured by the image pickup means. A transmission means for transmitting at least one of the locus information or the code value to the communication device is provided, and the coordinate value and the code value are defined in the writing area of the medium on which the dot pattern is formed, and the code is defined. The value has an area specifying code that specifies the writing area, and the analysis means includes a handwriting device that includes the area specifying code in the trajectory information and outputs the handwriting device, and the communication device that acquires voice information. It is characterized by becoming.

(17) Further, the handwriting device or the communication means to add information defining the passage of time to the locus information to at least one of the handwriting device or the communication device to synchronize with the voice information. It may be further equipped with at least one of the devices.

(18) Further, the communication device is a communication device used by itself, and the transmission means transmits the voice information and the locus to the other party communication device used by a third party via the communication device. You may send at least one of the information or the code value.

(19) Further, the other party communication device may be specified by the locus information.

(20) Further, a storage means for storing at least one of the locus information and the voice information in at least one of the handwriting device or the communication device is stored in at least one of the handwriting device or the communication device. Further, the transmission means provided in the communication device or the transmission means may transmit at least one of the voice information and the trajectory information stored in the storage means as a file.

(21) Further, at least one of the handwriting device and the communication device is further provided with a display means for displaying at least the locus information, which is either built-in or externally connected. You may.

(22) Further, the display means may output the voice information in synchronization with the locus information.

(23) Further, the voice output means for outputting the received voice information is provided to the communication device.
You may also prepare further.

(24) Further, the communication device may further be provided with a voice receiving means for receiving voice information from the other party communication device.

(25) Further, the communication device may further be provided with a track receiving means for receiving the track information from the other party communication device.

(26) Further, with respect to the locus information, the locus information further drawn by the handwriting device may be transmitted to the other party communication device via the communication device.

(27) Further, the communication device may be a smartphone or a PC.

(28) Further, the other party communication device may be a smartphone or a PC.

According to the present invention, there is an effect that a telephone call can be made without a complicated operation such as pressing a button.

Further, by linking the wearable device and the pen-type device, there is an effect that it is possible to operate a telephone or the like hands-free.
Further, by providing a highly convenient pen-type device capable of simultaneously transmitting voice information and locus information to the other party, there is a great effect of the invention.

(1) When temporarily storing and transmitting voice information and trajectory information to the other party in a file format.
In any of the following, when the trajectory information is played back in real time in synchronization with the voice, the time elapses on either a pen-type device or one's own smartphone (smartphone), personal computer (PC), wearable computer, etc. It is necessary to add the information indicating that to the locus information.

(1-1) When the pen-type device has a transmission function to the other party's smartphone or PC (including wearable computers, the same applies to others).
● Voice acquisition means, storage means, synchronization means (later, it is useless if there is no need to synchronize the trajectory information with the voice in real time and play it, the same applies to others), and it is equipped with voice information. The trajectory information is recorded on the pen-type device, and later, the voice information and the trajectory information are transmitted as a file to the other party's smartphone or PC.

(1-2) There is no transmission function to the other party's smartphone or PC inside the pen-type device, and the trajectory information and acquired voice information (useless when using the smartphone's voice acquisition / recording means) are sent to the own smartphone or PC. When sending and recording, and then sending the trajectory information and voice information as a file from your smartphone or PC to the other party's smartphone or PC.
● The pen-type device does not have a voice acquisition means or storage means, and the voice is recorded by the voice acquisition means or storage means of the own smartphone or PC, and the trajectory information is transmitted from the pen-type device to the own smartphone or PC in real time. And record.
● The pen-type device is equipped with only the locus information storage means, and the voice is recorded by the voice acquisition means and the storage means of a smartphone or a PC. At this time, in order to synchronize the voice information and the locus information on the own smartphone or PC, information indicating the passage of time may be added to the locus information on either the pen-type device or the own smartphone or PC. In order to ensure the accuracy of synchronization of information and trajectory information, it is better to synchronize with your own smartphone or PC. Since the locus information can be transmitted in almost real time, there is almost no delay. In addition, since the processing speed of smartphones and PCs is overwhelmingly faster than that of pen-type devices, it is desirable to have smartphones and PCs perform complicated processing.
Trajectory information is sent as a file from a pen-type device to a smartphone or PC for recording. If you do not need to synchronize and record the voice information while drawing the trajectory, later send the trajectory information as a file and record it, and then browse the trajectory information on your smartphone or PC and repeat the voice. May be recorded.
● The pen-type device is equipped with voice acquisition means and storage means. It records voice information and trajectory information on the pen, and later sends the voice information and trajectory information as a file to its own smartphone or PC. Record. At this time, it is desirable to add information indicating the passage of time to the locus information in order to synchronize the voice information and the locus information with the pen-type device.

(2) When transmitting voice information and trajectory information to the other party in real time.
In any of the following, when the locus information is played back in real time in synchronization with the voice, it is necessary to add the information indicating the passage of time to the locus information on the pen-type device, the other party's smartphone, or the PC.

(2-1) When the pen-type device has a transmission function to the other party's smartphone or PC.
● Send voice information and trajectory information from a pen-type device to the other party's smartphone or PC in real time and record it. When transmitting voice information and trajectory information using communication means with different speeds, delays occur on both the other party's smartphone and PC, so the passage of time is required to synchronize the voice information and trajectory information with a pen-type device. It is desirable to add the indicated information to the trajectory information. If the speed of both communication means is sufficiently fast, there is no problem.
● A voice output means is provided inside the pen-type device to receive voice information from the other party's smartphone or PC in real time and have a conversation.
● The display can be mounted on a pen-type device, or a display can be provided as an external device of the pen-type device to transmit voice information while viewing trajectory information. In particular, when a watch-type or glasses-type wearable computer or the like is used as the display, one hand can use a writing medium and one hand can use a pen-type device, so that it can be used even while moving, improving convenience.
● You can share the track information with the other party's smartphone or PC, add handwriting to the track information you are browsing, and send and receive each other. You can talk at the same time. By displaying drawings, photographs, and written instructions on the display during work and drawing from above, advanced communication that does not convey only words becomes possible.

(2-2) With the various pen-type devices shown in (1), trajectory information and voice information (useless when using the voice acquisition function of a smartphone or PC) are sent to the other party's smartphone via their own smartphone or PC. , When sending to a PC ● Send voice information and trajectory information in real time from your smartphone or PC to the other party's smartphone or PC.
● You can share track information between your own smartphone, PC and the other party's smartphone, PC, and both sides can add handwriting to the track information you are browsing and send and receive to each other. You can talk at the same time. The same thing can be done with the current smartphones and PCs using the touch panel, but writing is possible without worrying about the accuracy of handwriting and the screen size, and the listability of paper media is clearly superior. Moreover, since it can record handwritten paper media and digital data including voice, it is extremely effective in the field of storing paper media.

It is a perspective view of the pen type device which concerns on 1st Embodiment of this invention. It is a block diagram which shows the internal structure of the pen type device which concerns on 1st Embodiment of this invention. It is explanatory drawing which shows the use state of the pen type device which concerns on 1st Embodiment of this invention. It is a perspective view which shows the modification of the pen type device which concerns on 1st Embodiment of this invention. It is explanatory drawing which shows the paper controller used together with the pen type device. It is explanatory drawing which shows the use state of the pen type device which concerns on 2nd Embodiment of this invention. It is explanatory drawing which shows the state which mail transmission is performed using the pen type device and the wristwatch type wearable device which concerns on the 2nd Embodiment of this invention. It is explanatory drawing which shows the state which makes a telephone call using the pen type device and the wristwatch type wearable device which concerns on the 2nd Embodiment of this invention. It is explanatory drawing which shows the other state of making a telephone call using the pen type device and the wristwatch type wearable device which concerns on the 2nd Embodiment of this invention. It is explanatory drawing which shows the state which makes a transmission from a smartphone using the pen type device and the wristwatch type wearable device which concerns on the 2nd Embodiment of this invention. It is a modification of the 2nd Embodiment of this invention, and is explanatory drawing which shows the state which used the glasses type wearable device. It is a modification of the 2nd Embodiment of this invention, and is explanatory drawing which shows the state which used the earphone type wearable device. It is a flowchart which shows the operation of making a telephone call using a pen type device. It is a figure explaining the use state of the handwriting apparatus which concerns on 3rd Embodiment of this invention. It is a block diagram which shows the internal structure of the handwriting apparatus which concerns on 3rd Embodiment of this invention. It is a figure explaining the use state of the handwriting apparatus which concerns on 3rd Embodiment of this invention. It is a block diagram which shows the internal structure of the handwriting apparatus which concerns on 3rd Embodiment of this invention. It is a figure explaining the sticky note for dot pattern printing used in the 3rd Embodiment of this invention. It is a flowchart explaining the operation in Example 1-1 of the 3rd Embodiment of this invention. It is a flowchart explaining the operation in Example 1-2 of the 3rd Embodiment of this invention. It is a flowchart explaining the operation in Example 1-3 of the 3rd Embodiment of this invention. It is a flowchart explaining the operation in Example 2-1 of the 3rd Embodiment of this invention. It is a flowchart explaining the operation in Example 2-2 of the 3rd Embodiment of this invention. It is a figure explaining the smart note used in the 3rd Embodiment of this invention. It is a figure explaining the synchronization of the locus information.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
<First Embodiment>
1 and 2 are diagrams illustrating a pen-type device according to the present invention.

The pen-type device in the first embodiment has a dot pattern reading function and a telephone function.

As shown in FIG. 2, the pen-type device includes a pen body. Although not shown, the pen body is composed of a CPU, a memory, and the like.

When the CPU reads the program or data stored in the memory of the pen body, the acquisition means, the control means, and the recognition means function.

The acquisition means acquires specific information (telephone number, etc.) that identifies the other party based on the information input by handwriting when making a call to the other party. The control means controls the start of a call when a call from the other party is received. The recognition means recognizes the content handwritten by the user or the content input by voice through the microphone.

As the input unit of the pen body, an imaging means for photographing a dot pattern, a voice acquisition means such as a microphone, an instruction means such as a pen tip, and a receiving means for receiving an incoming call from a call partner are connected to the pen body. There is.

The image pickup means captures a dot pattern when the user performs handwriting input. That is, the handwriting input is performed on the paper on which the dot pattern is printed.

Here, the dot technology, which is a configuration peculiar to the present invention, will be described.

In the present invention, the "dot pattern" is information or numerical value encoded by an arrangement algorithm of a plurality of dots.

Regarding the algorithm of how to encode information or numerical values by dot pattern, Japanese Patent No. 3706385, Japanese Patent No. 3858051, Japanese Patent No. 3771252, Japanese Patent No. 4834872, Japanese Patent The dot pattern disclosed in detail in No. 4899199, etc. can be used. Typical dot pattern standards include Grid Onput (registered trademark), Anoto pattern, and the like.

In addition, although there are several dot pattern standards on the market at the time of the present invention, in the present invention, in addition to the dot patterns existing at present, all dot patterns including dot patterns to be developed in the future shall be used. Is possible.

Further, the dot pattern is preferably an invisible (or hard to see), invisible pattern that can be superimposed on a normal design. However, when printing using so-called stealth ink, a normally visible pattern such as a QR code (registered trademark) may be used.

Further, the dot pattern standard is preferably a standard that can pattern a coordinate value and a code value in one format. The aforementioned Grid Onput (registered trademark) can pattern coordinate values and code values in one format.

In the output unit of the pen body, an infrared irradiation means such as an IR-LED, a voice output means such as a speaker, and a transmission means for transmitting to the other party are connected to the pen body.

FIG. 3 is a diagram illustrating a usage state of the pen-type device.

The user uses a pen-type device to handwrite the telephone number of the other party (calling party) to whom he / she wants to make a call on a paper on which a dot pattern is printed. Then, the image pickup means captures the dot pattern, and the recognition means recognizes the locus. Then, the acquisition means acquires the recognition result. The calling means makes a call to the other party based on the recognition result. For example, when the user manually inputs "080-XXX-XXX", a call is made to "080-XXX-XXX-XXX".

The pen-type device can also receive an incoming call from the other party. In this case, the receiving means receives the incoming call from the other party, and the controlling means controls the start of the call.

In this way, by providing the pen for reading the dot pattern with a telephone function, it is possible to make a telephone call without performing a complicated operation of pressing a button and inputting a telephone number. Of course, this pen-type device can also be used as a normal handwriting pen, which is particularly convenient for those who use the handwriting pen on a daily basis.

The pen-type device is not limited to the above-described embodiment. For example, it can be transformed as follows.
(1) Providing a Display on a Pen-type Device FIG. 4 is a diagram showing a state in which a display is provided on a pen-type device.

This allows the user to see the handwritten phone number on the display. Also, when receiving an incoming call from the other party, the telephone number of the other party is displayed, so it is possible to immediately know who the call is from.
(2) Character Input by Optical Character Recognition (OCR) In the above-described embodiment, character input is performed using a dot pattern, but the present invention is not limited to this, and character input may be performed using optical character recognition.

In this case, the pen-type device has a built-in optical character recognition sensor. When the user scans the handwritten telephone number with the optical character recognition sensor after or while handwriting the telephone number, the recognition means recognizes the scanned image. Then, the acquisition means acquires the recognition result. The calling means makes a call to the other party based on the recognition result.

According to this, it is not necessary to use a dedicated paper such as a paper on which a dot pattern is printed, and it is possible to perform handwriting input using ordinary paper.
(3) Using a paper controller In the present invention, it is also possible to use a paper controller when inputting a telephone number.

The paper controller is a print medium that imitates a controller that gives various operation instructions. On the print medium, an icon on which characters, figures, etc. are drawn and a dot pattern in which information corresponding to the icon is coded are superimposed and printed. When the user touches the icon with the pen-type device, the process corresponding to the content of the icon is performed.

FIG. 5 is a diagram illustrating a paper controller used in this embodiment. This paper controller can be provided in a notebook, for example. Icons such as "call", "end call", and "send mail" are printed on the paper controller. When making a call, the user touches a telephone number in the address book (see FIG. 8) and then touches "call". Then, the telephone number is recognized by the optical recognition sensor, and a call is made to the touched telephone number.

An alphanumerical icon may be provided on the paper controller, and the icon may be touched to enter a telephone number.
(4) Using voice recognition In the present invention, it is also possible to input a telephone number using voice recognition.

The pen-type device is provided with a voice recognition means as the recognition means. When the user speaks a telephone number into the microphone, the voice recognition means analyzes the voice and makes a call to the telephone number.

In the above-described embodiment, the transmission of a telephone number has been described, but the present invention is not limited to this, and of course, it may be used for transmitting an e-mail or transmitting a message using various SNSs or applications. be. This also applies to the second embodiment described below.
<Second embodiment>
Next, a second embodiment of the present invention will be described.

The second embodiment uses a wearable device and a pen-type device.

The pen-type device used in the present embodiment has the same basic structure as that of the first embodiment, but is used as a specific means for specifying input information for the wearable device and for transmitting the input information to the wearable device. The difference is that it is equipped with a means of transmission. Further, it does not have to have a call function (sending means, receiving means).

FIG. 6 is a diagram showing a state in which the content handwritten by the pen-type device is displayed on the wristwatch-type wearable device (hereinafter referred to as “wristwatch”).

The user performs handwriting input using a pen-type device. Then, the image pickup means captures the dot pattern, and the recognition means recognizes the locus. The specific means specifies the input information based on the recognition result. Then, the transmission means transmits the input information to the wearable device.

In FIG. 6, the user is handwriting the characters "apple". When this character is recognized and specified as input information, the transmission means transmits the information of "apple" to the wristwatch. Then, the handwritten characters of "apple" are displayed on the dial (display) of the wristwatch.

The displayed characters may not be handwritten characters but may be text data converted by OCR.

FIG. 7 is a diagram showing a state in which mail is transmitted from the wristwatch.

The user writes a message (for example, "hello") in the notebook and specifies the destination address. Then touch "Send Email" on the paper controller. Then, an email is sent from the wristwatch.

Here, the destination e-mail address can be specified from a smartphone or tablet (hereinafter, "smartphone or the like"). In this case, open the address book of a smartphone or the like, touch the "up" and "down" icons on the paper controller, and specify the e-mail address.

You can also specify your e-mail address from your notebook. In this case, when the destination e-mail address is touched, the optical recognition sensor recognizes the e-mail address and sends the e-mail to the touched e-mail address.

FIG. 8 is a diagram showing a state in which a telephone call is made from the wristwatch.

The user touches the telephone number in the address book and then touches "Call" on the paper controller. Then, the telephone number is recognized by the optical recognition sensor, and the wristwatch makes a call to the touched telephone number.

When the telephone number is specified, it may be specified from a smartphone or the like as in the case of FIG. 7. Open the address book of your smartphone, etc., touch the "up" and "down" icons on the paper controller, and specify the phone number.

FIG. 9 is a diagram illustrating a state in which the wristwatch does not have a call function and only the pen-type device has a call function.

The user specifies a destination telephone number in the same manner as in FIG. 8, and touches "call" on the paper controller. Then, the telephone number is recognized by the optical recognition sensor, and the pen-type device makes a call to the touched telephone number.

At this time, the display of the wristwatch displays the telephone number of the call destination and a picture indicating that the call is being made from the pen-type device.

As a result, the user can accurately recognize that the outgoing call is being made.

FIG. 10 is a diagram illustrating a case where a transmission is performed from a smartphone.

In this embodiment, the call function of the pen-type device or the wristwatch is not used, and the call is made from the smartphone.

In this case as well, the method of inputting the telephone number is the same as the method described with reference to FIGS. 8 and 9. When "Call" on the paper controller is touched, the call is made from the smartphone. At the same time, the wristwatch display shows the destination phone number and a picture showing that the call is being made from the smartphone.

11 and 12 are diagrams illustrating other forms of wearable devices.

FIG. 11 shows a case where a glasses-type wearable device is used. FIG. 12 shows a case where an earphone type wearable device is used.

In both wearable devices, the usage of the pen-type device and the paper controller is the same as that of the wristwatch described above.

In a glasses-type wearable device, when a call is made, the telephone number of the callee can be visually recognized through the glasses. In an earphone-type wearable device, it is possible to make a call through an earphone by being wirelessly connected to a pen-type device.

The structure may be such that the earphone-type wearable device is connected to the glasses-type wearable device. In this case, a bone conduction microphone or speaker may be used.

In this way, by using the pen-type device and the wearable device in conjunction with each other, it becomes possible to make a hands-free call, and the convenience is greatly enhanced.

In the present embodiment, the telephone transmission and the mail transmission have been described, but the present invention is not limited to this, and it is needless to say that the present invention can be applied to all other operations.

Further, in the first and second embodiments, handwriting input is performed on a paper on which a dot pattern is printed, but the present invention is not limited to this, and other media such as an input board in which a dot pattern is formed on a magnetic board. May be used. Further, as the pen, a ballpoint pen that disappears may be used in addition to a normal ballpoint pen.

FIG. 13 is a flowchart showing the operation of making a telephone call.

A user uses a pen-type device to handwrite the telephone number of the other party on a piece of paper on which a dot pattern is printed. Then, the image pickup means captures the dot pattern, and the recognition means recognizes the locus (step S131). For example, as shown in FIG. 3, when the user handwrites "080-XXXX-XXXXX", the recognition means recognizes the locus. Next, the acquisition means identifies the telephone number from the recognized locus (step S132). In the example of FIG. 3, it is specified that the telephone number of the other party is "080-XXXX-XXXXX". Next, the calling means makes a call to the specified telephone number (step S133). In the example of FIG. 3, transmission to "080-XXXX-XXXX" is performed. Next, the control means recognizes that the other party has answered the telephone, that is, the connection has been made to the other party's telephone number (step S134), and the call is started (step S135).

<Third embodiment>
Next, a third embodiment of the present invention will be described.

14 and 16 are diagrams illustrating a third embodiment. In the third embodiment, a pen-type device is used to transmit trajectory information such as a handwritten picture or character on a dot pattern printing paper at the same time as a call.

15 and 17 are views illustrating the pen-type device used in the third embodiment.

FIG. 15 is a pen-type device having only a function as a handwriting pen, and includes a dot pattern imaging means, a control means, an analysis means, a storage means, a synchronization means, and a transmission means. It is not necessary to have a storage means and a synchronization means.

FIG. 17 is a pen-type device capable of directly transmitting and receiving to and from a smartphone of the other party, and is a dot pattern imaging means, a voice acquisition means, a voice information receiving means, a trajectory receiving means, an analysis means, a control means, a synchronization means, and a voice output means. , Equipped with transmission means. In addition, the display device is connected wirelessly or by wire. An example of a display device is a thin display. By connecting a display, it is possible to see handwritten pictures and characters sent from the other party without using a smartphone or PC. As shown in FIG. 4, the display device may be provided in the pen-type device main body.

Further, a plurality of examples can be considered in the third embodiment. This will be described below.
<Example 1>
In the first embodiment, voice information and trajectory information are recorded on either a pen-type device, one's own smartphone (smartphone), or a personal computer (PC), and then voice information and trajectory information are recorded on the other party's smartphone or PC. Will be described below.
(Example 1-1)
In this embodiment, voice information and trajectory information are recorded on one's own smartphone or PC. Voice information is recorded using the voice recording device of the smartphone, and the trajectory information is transmitted and recorded in real time from the pen-type device to the own smartphone or PC.

According to this, it is not necessary to provide a voice acquisition device and a recording device in the pen-type device, which has the effect of reducing the cost and making the size of the pen-type device compact.
(Example 1-2)
In this embodiment, voice information and trajectory information are recorded on the pen-type device. In this case, it is necessary to connect a pen-type device to one's own smartphone or PC later and transmit the information.

According to this, it is possible to record voice information on a pen-type device while handwriting without connecting to a smartphone or a PC. Therefore, even if a smartphone or pen-type device is not carried or connected, it can be used for children's learning and at the work site, which is extremely convenient.
(Example 1-3)
In this embodiment, the locus information is recorded on the pen-type device, only the voice information is recorded on the own smartphone or PC, and the locus information is later transmitted to the own smartphone or PC.

According to this, since it is not necessary to record voice information at the same time, there is no need for a voice acquisition device, there is an effect that the cost can be reduced and the size of the pen can be made compact.

Further, when the voice information is not recorded at the same time as the locus information, the same effect as that of the first and second embodiments can be obtained. In this case, the voice information may be recorded later while reproducing the trajectory information on the own smartphone or PC.

In Examples 1-1 to 1-3, in order to reproduce the locus information in synchronization with the voice information, the locus information may include information indicating the passage of time.

(Sticky note for printing dot patterns)
Next, the dot pattern printing paper used to realize the above-mentioned Examples 1-1 to 1-3 will be described with reference to FIG.

The dot pattern printing paper shown in the figure is configured as a sticky note, and is hereinafter referred to as a “dot pattern printing sticky note”.

The dot pattern printing sticky note is composed of a writing area and an icon area.

The writing area is an area for the user to draw a document or a figure. The locus information handwritten in the writing area is recorded on a pen, one's own smartphone, or a PC. If handwriting input is performed in the paused state, it can be inserted from that position.

The icon area is an area on which icons for the user to perform various operations are printed. In this embodiment, seven icons are printed. The functions of the icons are erase, record, play, pause, color, send, and page number from the left. When the user touches the icon on the pen device, the operation assigned to the icon is executed.

The "erase" icon is an icon for erasing voice information. If you touch during recording, playback, or within 2 seconds after the end of playback, the message "If you want to erase, touch the trash can again to erase all the voice." Is displayed. Alternatively, if voice information is recorded on the pen-type device, a similar message is played back. If you touch it again within 3 seconds, the voice information will be erased. If you do not touch it for 3 seconds or more, it will be in a standby state. When voice information is recorded on a smartphone or a PC, this function may be operated with a touch panel or a mouse by providing a similar icon. When other locus information-related functions are recorded on a smartphone or PC, similar icons may be provided on the smartphone or PC and operated on the smartphone or PC by operating the touch panel or mouse.

The "Record" icon is an icon for starting recording on your smartphone, PC, or handwriting pen. Touch again during recording to end recording. If you touch another icon during recording, recording ends and the operation process of the touched icon is executed. If an audio file already exists, an additional audio file will be generated. When you touch another icon, the operation process of the touched icon is executed, but the operation status is transmitted in real time.

The "playback" icon is an icon for playing back the voice and trajectory information. During the pause, playback is performed in real time from the paused state. At the end (when all trajectory data is displayed), it is played back in real time from the beginning. However, if you touch it again within 1 second after pressing the play button, it will be double speed, and if you touch it again within 1 second, it will be 4x speed.

The "pause" icon is an icon for performing a pause. If you touch it during pause, it will play from the pause position at the speed before that. If you start touching or writing to another icon during pause, the paused state is canceled and the touched icon is processed or written.

The "color" icon is an icon for determining the color of the trajectory drawn by the pen-shaped device. The left is black and the right is red. In addition, although not shown, there may be a function of setting the thickness of the locus.

The "send" icon is an icon for transmitting the locus information and voice information handwritten on the page to the smartphone or PC of the other party. Set the destination address on your smartphone.

The "page number" icon is an icon indicating each page number (1 to 100) when a plurality of pages are provided as a bundle of dot pattern printing sticky notes. Touch this icon to display all trajectory data of the page. Since the locus information includes the area identification code for specifying the page, various processing can be performed in conjunction with the page number.

In addition, in order to execute the play / pause / insert function of this embodiment, the time elapsed information must be added to the locus information.

A general sticky note can be used as a memo for yourself or to convey information to the other party by drawing a simple document or figure. However, many sticky notes have a small area and cannot convey detailed information. With the dot pattern printing sticky note, voice information can be added, so the explanation of the memo can be conveyed in detail. Furthermore, general sticky notes are disposable and it is difficult to organize and store information, but in dot pattern printing sticky notes, the trajectory information is digitized and recorded on a smartphone or PC, which was described in the past. Information can be easily recorded, organized, and utilized. Of course, this embodiment does not need to be a sticky note, and a notebook or the like may be equipped with the same function.

Next, the operations of the pen-type device (pen), the own smartphone / PC, and the other party's smartphone / PC in the first embodiment will be described with reference to FIGS. 19 to 21.

In the figure, although it is described as "own smartphone" and "other party's smartphone", it is needless to say that it is also applied to own PC, wearable computer, partner's PC, and wearable computer.

FIG. 19 is a flowchart illustrating the operation in the first embodiment.

The user uses a pen to input handwriting on a sticky note for printing a dot pattern or the like. Also, at the same time, or immediately before or after, the voice is uttered. For example, draw a map on a sticky note for printing dot patterns, and by voice, "Exit the ticket gate at the south exit of XX station on the XX line and go straight. Turn left at the second intersection, go straight for two blocks, and go straight to the second intersection. Turn right and turn left at the first intersection, and you will see your home on your right. "

In the pen, the image pickup means captures a dot pattern, and the recognition means recognizes the trajectory and acquires the trajectory information. The pen wirelessly or wiredly transmits the acquired trajectory information to its own smartphone (step S1601). Then, the own smartphone receives the locus information (step S1602) and records it in the recording device (step S1604).

On the other hand, the user's voice is recorded in the recording device of his / her smartphone (step S1605).

The own smartphone transmits the recorded voice information and the trajectory information to the smartphone of the other party (step S1607).

The smartphone of the other party receives the voice information and the locus information (step S1608). Then, the voice information is output from the speaker, and the locus information is displayed on the display (step S1609). Synchronization (step S1603) is required to reproduce the audio information and the trajectory information without any contradiction later. Synchronization will be described in detail later.

FIG. 20 is a flowchart illustrating the operation in the first and second embodiments.

The user uses a pen to input handwriting on a sticky note for printing a dot pattern or the like. Also, at the same time, or immediately before or after, the voice is uttered. For example, draw a map on a sticky note for printing dot patterns, and by voice, "Exit the ticket gate at the south exit of XX station on the XX line and go straight. Turn left at the second intersection, go straight for two blocks, and go straight to the second intersection. Turn right and turn left at the first intersection, and you will see your home on your right. "

In the pen, the image pickup means images a dot pattern, the recognition means recognizes the locus, and the locus information is acquired. Then, the locus information is recorded in the recording device built in the pen (step S1701). Further, the voice information uttered by the user is acquired and recorded in the recording device (step S1702). Then, the recorded voice information and the locus information are transmitted to the own smartphone (step S1707).

When the own smartphone receives the voice information and the locus information (step S1706), it transmits the voice information and the locus information to the other party's smartphone (step S1708).

The smartphone of the other party receives the voice information and the locus information (step S1708). Then, the voice information is output from the speaker, and the locus information is displayed on the display (step S1709). Synchronization (step S1701) is required to reproduce the audio information and the trajectory information without any contradiction later. Synchronization will be described in detail later.

FIG. 21 is a flowchart illustrating the operation in the first and third embodiments.

The user uses a pen to input handwriting on a sticky note for printing a dot pattern or the like. Here, the pen and one's own smartphone are not connected, and only the trajectory information needs to be recorded. When the locus information is later reproduced in real time, it is necessary to record the handwritten time lapse information in the locus information together with the coordinate information as shown in FIG. 25. Also, at the same time, or immediately before or after, the voice is uttered. For example, draw a map on a sticky note for printing dot patterns, and by voice, "Exit the ticket gate at the south exit of XX station on the XX line and go straight. Turn left at the second intersection, go straight for two blocks, and go straight to the second intersection. Turn right and turn left at the first intersection, and you will see your home on your right. "

In the pen, the image pickup means images a dot pattern, the recognition means recognizes the locus, and the locus information is acquired. Then, the locus information is recorded in the recording device built in the pen (step S1801). Then, the recorded locus information is transmitted to the own smartphone (step S1802).

The own smartphone receives the locus information (step S1803) and records it in the recording device (step S1805). Further, the voice information uttered by the user is acquired and recorded in the recording device (step S1807). Then, the recorded voice information and the locus information are transmitted to the smartphone of the other party (step S1808).

The smartphone of the other party receives the voice information and the locus information (step S1809). Then, the voice information is output from the speaker, and the locus information is displayed on the display (step S1810).
<Example 2>
Next, the operations of the pen-type device (pen), the own smartphone / PC, and the other party's smartphone / PC in the second embodiment will be described with reference to FIGS. 22 to 23.

In the second embodiment, a case where voice information and trajectory information are transmitted and received in real time from the pen-type device or directly to the other party's smartphone or PC via one's own smartphone or PC will be described.
(Example 2-1)
FIG. 19 is a flowchart illustrating the operation in the second embodiment.

In this embodiment, only the trajectory information is transmitted from the pen-type device to the other party's smartphone or PC in real time via the own smartphone or PC. When the locus information is received by one's own smartphone or PC, it is desirable to record the information indicating the elapsed time for synchronization in the locus information as shown in FIG.

According to this, since the pen uses its own smartphone, the voice acquisition means of the PC, and the voice output means, it is not necessary to provide a voice-related device or a storage device in the pen, the cost is reduced, and the size of the pen is compact. There is an effect that can be done.
(Example 2-2)
FIG. 20 is a flowchart illustrating the operation in the second embodiment.

In this embodiment, voice information and trajectory information are transmitted directly from the pen-type device to the other party's smartphone or PC. In this case, a voice acquisition means and a voice output means are provided inside the pen-type device, and voice information from the other party's smartphone or PC can be received in real time to have a conversation.

According to this, it is possible for children to use it for learning and at the work site without the need for their own smartphone or PC, which is extremely convenient. Further, if the display is mounted on the pen-type device or the display is provided as an external device of the pen-type device, voice information can be transmitted / received while sharing and browsing the trajectory information.

In Examples 2-1 and 2-2, in order to reproduce the locus information in synchronization with the voice information (step S187), the locus information may include information indicating the passage of time. Here, in the case of uttering a voice according to the history of the locus while drawing a locus, if the voice and the point on the locus do not match, a contradiction occurs in the explanation. In particular, when voice information and trajectory information are transmitted to the other party's smartphone by different communication means, a delay occurs depending on the communication speed. Therefore, in the other party's smartphone, in order to output the voice information and the locus information so as to be synchronized, as shown in FIG. 22, the time elapsed information for synchronization is recorded in the locus information. Originally, the voice information is reproduced in real time, so if the time information of the start point is matched by a predetermined method, the other party's smartphone can output the voice information and the trajectory information without delay. If the speeds of the different communication means for the voice information and the locus information are both sufficiently high, it is not necessary to synchronize them. However, it is indispensable to synchronize the voice information and the trajectory information on the smartphone of the self and the other party when the voice information and the trajectory information are recorded and later played back without delay.
(Sticky note for printing dot patterns)
Next, the dot pattern printing paper used to realize the above Examples 2-1 to 2-2 will be described.

The dot pattern printing paper of the figure is configured as a sticky note for dot pattern printing, as in the above-described 1-1 to 1-3.

The sticky note for dot pattern printing in this embodiment has the same appearance as that used in Examples 1-1 to 1-3, that is, FIG. 15, but has some functions peculiar to this embodiment. There is.

Hereinafter, only the functions different from those of Examples 1-1 to 1-3 will be described.

If you touch the "Erase" icon of this example during playback of voice information or within 2 seconds after the end of playback, the message "If you want to erase, touch the trash can again to erase the entire voice." indicate. Alternatively, if voice information is recorded on the pen-type device, a similar message is played back. If you touch it again within 3 seconds, the voice information will be erased. If you do not touch it for 3 seconds or more, it will be in a standby state. You may set a similar icon on your smartphone or PC and operate it with a touch panel or mouse. When other locus information-related functions are recorded on one's own smartphone or PC, similar icons may be provided and operated on the smartphone or PC by operating the touch panel or mouse. Of course, the same operation may be performed on the other party's smartphone or PC.

When you touch the "Record" icon, when you send voice information to the other party's smartphone or PC, you can start recording with your own smartphone, PC, or a pen with a recording function. Touch again during recording to end recording. If you touch another icon during recording, recording ends and the operation process of the touched icon is executed. If an audio file already exists, an additional audio file will be generated. When you touch another icon, the operation process of the touched icon is executed, but the operation status is transmitted in real time. Even if you do not record, voice information is automatically sent to the other party's smartphone or PC.

When the "Send" icon is touched, the track information and voice information handwritten on the page are sent to the other party's smartphone or PC. Set the destination address and phone number with a pen-type device or your own smartphone. When sending directly with a pen-type device, set by touching a paper keyboard or address book on which dedicated dots are superimposed and printed. Paper keyboards and address books are shown, for example, in FIGS. 5, 8-10, and the corresponding specification description.

In addition, when the user writes figures, characters, etc. in the writing area, the handwritten trajectory information is transmitted directly from the pen-type device or via his / her own smartphone / PC to the other party's smartphone / PC. Recorded in.
In this embodiment, a dot pattern printing sticky note is used to transmit information to the other party's smartphone or PC. However, not limited to this, by drawing a simple document or figure and using it as your own memo, or by digitizing the voice information and trajectory information of the dot pattern printing sticky note received by the other party and recording it on your smartphone or PC. , Past information can be easily recorded, organized and utilized. It should be noted that this embodiment does not need to be a dot pattern printing sticky note, and it is needless to say that a notebook or the like may be equipped with the same function.

Next, the operation of the pen-type device, the own smartphone / PC, and the other party's smartphone / PC in the second embodiment will be described with reference to FIGS. 22 to 23.

FIG. 22 is a flowchart illustrating the operation in the second embodiment.

The user uses a pen to input handwriting on a sticky note for printing a dot pattern or the like. Also, at the same time, or immediately before or after, the voice is uttered. For example, draw a map on a sticky note for printing dot patterns, and by voice, "Exit the ticket gate at the south exit of XX station on the XX line and go straight. Turn left at the second intersection, go straight for two blocks, and go straight to the second intersection. Turn right and turn left at the first intersection, and you will see your home on your right. "

In the pen, the image pickup means captures a dot pattern, and the recognition means recognizes the trajectory and acquires the trajectory information. The pen transmits the acquired trajectory information to its smartphone wirelessly or by wire (step S1901). Then, the own smartphone receives the locus information (step S1904) and transmits it to the other party's smartphone (step S1906). The smartphone of the other party receives the locus information (step S196).

On the other hand, the own smartphone acquires the voice information of the user. Then, the voice information is transmitted to the smartphone of the other party (step S1903). The smartphone of the other party receives the voice information (step S1907).

The smartphone of the other party outputs the received voice information from the speaker and displays the received trajectory information on the display (step S1909).

FIG. 23 is a flowchart illustrating the operation in the second embodiment.

The user uses a pen to input handwriting on a sticky note for printing a dot pattern or the like. Also, at the same time, or immediately before or after, the voice is uttered. For example, draw a map on a sticky note for printing dot patterns, and by voice, "Exit the ticket gate at the south exit of XX station on the XX line and go straight. Turn left at the second intersection, go straight for two blocks, and go straight to the second intersection. Turn right and turn left at the first intersection, and you will see your home on your right. "

In the pen, the image pickup means captures a dot pattern, and the recognition means recognizes the trajectory and acquires the trajectory information. The pen transmits the acquired trajectory information to the smartphone of the other party wirelessly or by wire (step S2002). Then, the smartphone of the other party receives the locus information (step S2005). In addition, the voice information uttered by the user is acquired and transmitted to the smartphone of the other party (step S2004). Then, the smartphone of the other party receives the locus information (step S2006).

The smartphone of the other party outputs the received voice information from the speaker and displays the received trajectory information on the display.
<Example 3> (Smart notebook)
Next, another example of the dot pattern printing paper used to realize the above-mentioned Examples 1-1 to 1-3, 2-1 to 2-2 will be described with reference to FIG. 24.

The dot pattern printing paper shown in the figure is configured as a notebook, and is hereinafter referred to as a "smart notebook". In this embodiment, as a lecture / practice note related to education or a notebook / notebook used by the general public, a function of recording trajectory information and voice information, and a function of playing / viewing information transmitted to one's own or the other party's smartphone or PC are provided. Explaining. The method for transmitting and receiving locus information and voice information is the same as in the above-described embodiment of the dot pattern printing sticky note, and a similar transmission icon may be provided. As for the icon arranged on the smart notebook, a similar icon may be arranged on the operation screen of the smartphone or the PC and operated with the touch panel or the mouse.

The smart note consists of a writing area and an icon area.

The writing area is an area for the user to draw a document or a figure.

The icon area is an area on which icons for the user to perform various operations are printed. In this embodiment, icon areas are formed at the top and bottom of the smart notebook. The functions of the icon are from left to right, erase, start, end, play, double speed, pause, back, send, control bar, for link, bottom from left, size, color, eraser, erase display, display, MP4, JPEG, link, link erase.

The "erase" icon is an icon for erasing all the locus information of the page. A confirmation screen will appear on the display, and you can touch it with your finger or press the ○ button on the pen to erase it. Originally, for safety, I would like to use only finger touch, but since there are times when I want to erase data when I am away from the tablet in the classroom, I also enable the deletion operation with a pen. However, erasing with the ○ button on the pen is enabled only when confirmation is displayed on the tablet. Furthermore, the deleted trajectory information can be recovered on a smartphone or PC.

The "start" icon is an icon for starting voice recording. Touch the record icon or the end icon again to end the recording. Touching another icon ends recording and the icon operation is performed. You can create as many audio files and locus files as you like.

The "end" icon is an icon for ending the operation. Touch during voice recording to end recording. Touch the end icon except during recording to display the final trajectory information in the display range.

The "play" icon is an icon for playing the operation. If you touch this icon, it will be played back in real time from the current state (including the paused state). All the gap times for which no locus is drawn are skipped (a mode that does not skip can be set on the setting screen). However, in real-time playback during recording / playback, the gap time is not skipped. If the last image in the display range is displayed, it will be played back in real time from the beginning. During pause (including after stroke movement) or during high-speed playback, playback is performed in real time from that position. If you touch it further, it will be played back at 2x, 4x, 8x, and 16x speeds.

The "double speed" icon is an icon for playing at double speed. If you touch this icon, it will be played back at 4x speed from the beginning. All void times for which no trajectory is drawn are skipped. Furthermore, when touched, it is played back at 8 times and 16 times speed.

The "pause" icon is an icon for pausing. During pause (including after stroke movement), playback is performed from the pause position at the speed before pause. Even if you pause and touch another page, the paused state is saved. Writing is possible even during the pause, and the locus information is inserted from that position and reflected during playback. If you move to another page and return during the pause, the screen at the time of pause is displayed. If you touch Pause during writing and perform real-time playback, playback will start from the position where writing was completed.

The "back" icon is an icon for returning one stroke while displaying the trajectory information.

The "send" icon is an icon for sending one stroke while displaying the trajectory information.

The "control bar" icon is an icon for controlling the display range, and from the left, an icon with a left-pointing triangle (hereinafter referred to as "the leftmost icon"), a time-lapse bar, and a right-pointing triangle are drawn. It is composed of the icon (hereinafter referred to as the "rightmost icon"). Touch the icon at the left end to display the top status of the display range. When the display range is in the initial state, if there is a problem, lecture content, instruction manual, etc. in the background, the background is displayed. When you touch the time elapsed bar, the image up to the touched position is displayed. If you move the pen-type device on the time-lapse bar while the pen-type device is in contact with the touch, the trajectory information to the movement position is displayed one after another. If you move to another page during playback and return, the last image in the display range will be displayed. If you move to another page and return during the pause, the screen at the time of pause is displayed.

The "for link" icon is an icon to which an image or video is linked. Touch this icon to display the linked image / video. The image / video can also be overwritten / recorded. The link icon may be composed of a sticker or the like, and may be added and pasted on the notebook.

The "size" icon is an icon for determining and changing the size of the pen and the eraser. The size of the pen and eraser is not recorded page by page. The last used information is saved. However, in the case of an eraser, if you move to another page, the pen is automatically set and the pen information used last time is adopted.

The "color" icon is an icon for determining / changing the color of the pen and the eraser. Pen and eraser colors are not recorded page by page. The last used information is saved. However, in the case of an eraser, if you move to another page, the pen is automatically set and the pen information used last time is adopted.

The "eraser" icon is an icon for erasing a locus or an area during handwriting. The size is selected by the thickness of the pen. If you do not select the size, it will be the size used in the eraser last time. A yellow circle is displayed while it is being erased, but the traced trajectory information is not displayed.

The "erased display" icon is an icon for displaying the deleted locus or area. The erased trajectory is displayed as it is, and the traced eraser mark is displayed in yellow semi-transparent. When a pen is selected from the eraser state, the color and size of the previously selected pen are selected.

The "display" icon is an icon for canceling the hiding and displaying all the trajectory information.

The "hide" icon is an icon for the reverse display function that hides the currently displayed locus information and the display range is outside the range. You can play, insert, and add operations only in the display range. To display the latter half, play from the beginning, pause at a predetermined position, turn it off, and play. To display the first half, hide the first half, display only the second half, and then turn it off. The display range is saved for each page.

The "MPEG4" icon is an icon for generating a moving image file. When you touch this icon, the setting screen appears, set which holder on your smartphone or PC you want to record, and select one of the following (1) to (3). Select Execute / Stop with the touch panel or mouse, or start recording with the pen ○ button. When you start recording, you will be returned to the smart note screen and the video will be recorded. If you want to finish recording during video recording, touch the MP4 icon again. In that case, a video file until the end is generated. The video may use a format other than MP4.
(1) Animate all operations such as trajectory, recording, and playback in real time.
(2) Animate all the locus and audio information of the page.
(3) Animate only the locus of the display range and audio information.
Audio information that spans the display range is played back from the beginning of the recording, and when it enters the display range, a moving image is recorded. If you make a general video such as MP4, you can view it on any smartphone or PC even if this smart note application is not installed, which enhances convenience.

The "JPEG" icon is an icon for recording the locus information being displayed. When you touch this icon, the recording destination holder is displayed and you can set where to record. Select execute / stop with a finger touch, or record with the pen ○ button.

The "link" icon is an icon for setting a link. The link setting is a process of associating the dot pattern printed on the above-mentioned link icon with an image / moving image. All processing is carried out on smartphones and PCs.

The "delete link" icon is an icon for deleting the display of the link image.

<Example 4>
The fourth embodiment describes the transmission data format of the handwriting pen.

There are various methods such as Bluetooth (registered trademark), BLE, WIFI, RF, various telephone lines, etc. for wireless communication between the handwriting pen and your own smartphone or PC, and wireless communication with the other party's smartphone or PC.

In order to realize handwriting, as a frame rate, trajectory information consisting of coordinate values of 100 times or more per second for characters and 150 times or more per second for figures and pictures is required. Recently, data transmission / reception by BLE, which is a part of the Bluetooth (registered trademark) 4.0 standard installed as standard on smartphones, is widely used. Although BLE has extremely high power saving, it is necessary to minimize the transmission of trajectory information because it can only perform data communication at a maximum of 1 Mbps.

In the present invention, only at the moment when the pen touches the writing area, the captured dot pattern is analyzed by dot code, the code value and XY coordinates defined in one format are read, and the area specifying the area specified below specifies the writing area. The trajectory information is output (8Byte) together with the code. As a result, the trajectory information can be easily managed for each writing area / page, and it is possible to specify a predetermined writing area / page and display / edit / transmit only the handwritten trajectory information. .. Further, in order to minimize the trajectory information, after touching the writing area with the pen, since the stroke is drawn in the same writing area / page, it is not necessary to output the area identification code or the status code. Therefore, only the locus information based on the XY coordinate values is output (4 bytes) to significantly reduce the amount of transmitted information. This made it possible for BLE to transmit the required frame rate trajectory information. As the status code, various information such as pen down / pen up, whether or not it is a writing area, handwritten pen button information, error information, etc. are output.

(1) XY coordinates
8byte: Transmission format when pen down
0 0Xff: Header (1)
1 0xFD: Header (2), a header to indicate the data delimiter at the time of transmission. Since it is read only once at the moment when the pen is touched, 2 bytes are used so that there is no misunderstanding of the data delimiter at the time of transmission.
2 Status code
3 Area identification code (1): 8bit
4 Area specific code (2): 8bit
Area specific code 16bit, about 6500 write area / page can be specified. Since the XY coordinate value can be defined as 2 mm with 1 bit, at present, the XY coordinate is 8 bits each, and the 512 × 512 mm square area is one write area. When the writing area is small as in Post-it, the range of XY coordinate values can be reduced to increase the area identification code. If the writing area is 64 x 64 mm square, the XY coordinates can be defined with 5 bits each, for a total of 10 bits. As a result, 6 bits can be added to the area specific code, and about 4.2 million write areas / pages can be specified with 22 bits.
5 X coordinates: 8bit
6 Y coordinates: 8bit
7 XY coordinates after the decimal point: Bits after the decimal point (4bit x 2) and XY coordinate values are defined as 2mm with 1bit, so the accuracy is low. I have secured it.

4byte: Transmission format during pen down
0 0Xff: Header to indicate data delimiter when sending. Even if the header is mistakenly recognized, it is transmitted 100 times or more per second, so if you read the next header, there is no problem even if one piece of data is missing, so only 1 byte is used. ..
1 X coordinate: 8bit
2 Y coordinates: 8bit
3 XY coordinates after the decimal point: bits after the decimal point (4bit x 2)

The following is the code value output when the dot pattern printed on the dot pattern printing sticky note shown in Examples 1 and 2 and the dot pattern formed by superimposing it on the icon such as the smart note shown in Example 3 are read with a pen. The format. The operation code corresponding to the icon below has 32 bits and can be associated with up to about 4 billion types of operation instructions and contents.
(2) Code value (8-byte transmission)
0 0xFF: Header (1)
1 0Xfd: Header (2), header to indicate data delimiter at the time of transmission. Since it is read only once at the moment when the pen is touched, 2 bytes are used so that there is no misunderstanding of the data delimiter at the time of transmission.
2 status bytes
3 Operation code (1)
4 Operation code (2)
5 Operation code (3)
6 Operation code (4)
7 Angle of rotation (2 degree unit: ± 1 degree) 0 to 179 (0 degree ± 1 degree to 358 degree ± 1 degree)
0 degrees (rotation code 0) is 359 degrees or more and less than 1 degree, 2 degrees (rotation code 1) is 1 degree or more and less than 3 degrees, 358 degrees (rotation code 179) is 357 degrees or more and less than 359 degrees, 2 Send a rotation code every degree. When touching with a pen to output the linked content, it is possible to acquire and output different information depending on the rotation angle, such as when rotating the pen to change the sound and content, or when viewing an image 360 degrees. ..

It should be noted that the above-mentioned embodiment is an example, and it goes without saying that various changes can be made without changing the gist. In addition, each embodiment can be freely combined and selectively used.

Since the pen-type device according to the present invention can be easily used by persons with disabilities and elderly people who are not good at telephone operation, it is considered to have industrial applicability in the welfare field.

Of course, it has the potential to be used by all people and all fields.

Claims (1)

A handwriting device for drawing a trajectory on a medium on which a dot pattern is formed.
An imaging means for imaging the coordinate value and the code value, or the dot pattern in which the code value is defined, and
An analysis means that outputs locus information and / or a code value consisting of a history of coordinate information obtained from at least the coordinate values by code-analyzing the dot pattern captured by the image pickup means.
Voice acquisition means to acquire voice information and
The voice information and the transmission means for transmitting at least one of the locus information or the code value to the communication device are provided.
The coordinate value and the code value are defined in the writing area of the medium on which the dot pattern is formed, and the code value has an area specifying code for specifying the writing area.
The analysis means is a handwriting device that includes the area identification code in the locus information and outputs the information.

Images