WO2017010469A1

WO2017010469A1 - Handwriting device and voice/handwriting communication system

Info

Publication number: WO2017010469A1
Application number: PCT/JP2016/070484
Authority: WO
Inventors: 吉田　健治
Original assignee: 株式会社Ｉ・Ｐソリューションズ
Priority date: 2015-07-10
Filing date: 2016-07-11
Publication date: 2017-01-19
Also published as: JP6973791B2; JPWO2017010469A1; JP2022020703A

Abstract

Provided is a pen device that is very user friendly and is able to simultaneously transmit speech information and locus information to another party.　A handwriting device for drawing a locus on a medium having a dot pattern formed thereon, said device being provided with: an image capturing means that captures an image of a dot pattern; an analysis means that, as a result of performing code analysis of the dot pattern captured by the image capturing means, outputs locus information and/or code values comprising the history of coordinate information obtained from at least coordinate values; a voice acquisition means that acquires voice information; and a transmission means that transmits, to a communication device, the voice information, and the locus information and/or the code values. A writing area of the medium having a dot pattern formed thereon is defined by coordinate values and code values, and the code values have an area identification code that identifies the writing area. The analysis means includes the area identification code in the locus information and outputs said information.

Description

Handwriting device and voice / handwriting communication system

The present invention relates to a pen-type device having a telephone function and a dot pattern reading function.

The present invention also relates to a pen-type device used in conjunction with a wearable device.

The present inventor has made many innovative inventions using dot patterns.

In these inventions, there is also a technology for connecting a scanner for reading a dot pattern and a mobile phone, reading the dot pattern with the scanner, and outputting information corresponding to the dot pattern from the mobile phone. There is also a mobile phone with a built-in scanner (Patent Document 1).

International Publication WO2004 / 029871

However, a scanner having a telephone function is not disclosed.

In recent years, wearable devices such as Apple Watch and GOOGLE GLASS (registered trademark) are becoming popular.

This invention is made in view of such a point, and makes it a technical subject to provide the pen-type device which has a telephone function and a dot pattern reading function.

Also, it is a technical problem to provide a pen-type device that causes the wearable device to execute various operations by linking with the wearable device.

In addition, e-learning is widely used worldwide in the education field, but at present, teacher lectures are recorded on video and distributed. Video recording requires considerable costs from recording location, cameraman arrangement to editing. Most of the communication between teachers and students is based on conversations using common educational materials (including digital contents). Some have been corrected, but they are not interactive. Ideally, it is desirable that teachers can easily deliver classes by hand, and can give guidance by voice and handwriting immediately to students' questions and answers. Also, in the business field, telephone conferences and video conferences are prevalent, but in order to proceed with the conference by sharing diagrams, photos, illustrations, etc., participants must use the images taken with the camera. In order to deepen understanding not only by conversation but also by handwriting, there are major issues in the accuracy, convenience, and cost of touch panels and pen tabs. In addition, social network check is spreading at a tremendous rate, and there is great expectation for handwritten chat, but at present there is no inexpensive and easy-to-use system.

(1) A handwriting device according to the present invention is a handwriting device for drawing a locus on a medium on which a dot pattern is formed, and images the dot pattern in which coordinate values and code values or code values are defined. An image capturing unit; an analysis unit that outputs at least trajectory information and / or a code value including a history of coordinate information obtained from the coordinate value by performing code analysis on the dot pattern captured by the image capturing unit; and audio information. Voice acquisition means for acquiring; and transmission means for transmitting at least one of the voice information and the trajectory information or code value to a communication device, and the writing area of the medium on which the dot pattern is formed includes A coordinate value and a code value are defined, and the code value has an area specifying code for specifying the writing area, and the solution The analyzing means outputs the locus information including the region specifying code.

(2) A handwriting device according to the present invention is a handwriting device for drawing a locus on a medium on which a dot pattern is formed, and images the dot pattern in which coordinate values and code values or code values are defined. Imaging means; analysis means for outputting trajectory information and / or code values comprising at least a coordinate information history obtained from the coordinate values by code analysis of the dot pattern imaged by the imaging means; and the audio information Transmitting means for transmitting at least one of the locus information or the code value to a communication device, and the coordinate value and the code value are defined in a writing area of the medium on which the dot pattern is formed, The code value has an area specifying code for specifying the writing area, and the analysis means specifies the area in the trajectory information. The communication device includes a code and outputs the voice information.

(3) The handwriting device or the communication device may further include synchronization means for adding information defining time passage to the trajectory information to synchronize with the voice information.

(4) Further, the communication device may be a counterpart communication device used by a third party.

(5) Further, the communication device is a communication device used by itself, and the transmission means sends the voice information and the locus to the counterpart communication device used by a third party via the communication device. Information and / or code values may be transmitted.

(6) Further, the counterpart communication device may be specified by the trajectory information.

(7) Further, a storage unit that stores at least one of the trajectory information and the voice information is further provided in either the handwriting device or the communication device, and the transmission unit or the transmission provided in the communication device The means may transmit at least one of the voice information and the trajectory information stored in the storage means as a file.

(8) Further, at least one of the handwriting device and the communication device further includes display means for displaying at least the trajectory information that is either built-in or connected to the outside. Also good.

(9) Further, the display means may output the audio information in synchronization with the trajectory information.

(10) Furthermore, voice output means for outputting the received voice information may be further provided in at least one of the handwriting device and the communication device.

(11) Further, at least one of the handwriting device and the communication device may further include a voice reception device that receives voice information from the counterpart communication device.

(12) Furthermore, a trajectory receiving device for receiving trajectory information from the counterpart communication device according to any one of claims 4 to 6 may be further provided in at least one of the handwriting device and the communication device. Good.

(13) Furthermore, trajectory information further drawn by the handwriting device may be transmitted to the counterpart communication device or the communication device according to any one of claims 4 to 6 with respect to the trajectory information.

(14) Further, the communication device may be a smartphone or a PC.

(15) Further, the counterpart communication device may be a smartphone or a PC.

(16) A voice / handwriting communication system according to the present invention is a handwriting device for drawing a locus on a medium on which a dot pattern is formed, wherein the coordinate value and code value or the dot pattern in which the code value is defined Imaging means for picking up images, and analyzing means for outputting trajectory information and / or code values consisting of at least a history of coordinate information obtained from the coordinate values by code analysis of the dot pattern imaged by the imaging means; Transmitting means for transmitting at least one of the trajectory information and the code value to a communication device, and the coordinate value and the code value are defined in a writing area of the medium on which the dot pattern is formed. The value has an area specifying code for specifying the writing area, and the analysis means adds the area specifying code to the trajectory information. And a communication device for acquiring voice information.

(17) Furthermore, synchronization means for synchronizing information with the audio information by adding information defining time passage to the trajectory information to at least one of the handwriting device or the communication device, the handwriting device or the communication You may further provide in at least any of apparatuses.

(18) Further, the communication device is a communication device used by itself, and the transmission means sends the voice information and the locus to the other party communication device used by a third party via the communication device. Information and / or code values may be transmitted.

(19) Further, the counterpart communication device may be specified by the trajectory information.

(20) Furthermore, storage means for storing at least one of the trajectory information and the voice information in at least one of the handwriting device or the communication device is provided in at least one of the handwriting device or the communication device. Further, the transmission unit or the transmission unit provided in the communication device may transmit at least one of the voice information and the trajectory information stored in the storage unit as a file.

(21) Further, at least one of the handwriting device and the communication device further includes display means for displaying at least the trajectory information that is either built-in or connected to the outside. May be.

(22) Further, the display means may output the audio information in synchronization with the trajectory information.

(23) Furthermore, an audio output means for outputting the received audio information is provided to the communication device.
Further, it may be provided.

(24) Further, the communication device may further include voice receiving means for receiving voice information from the counterpart communication device.

(25) Furthermore, the communication device may further include trajectory receiving means for receiving trajectory information from the counterpart communication device.

(26) Furthermore, the trajectory information further drawn by the handwriting device may be transmitted to the counterpart communication device via the communication device with respect to the trajectory information.

(27) Further, the communication device may be a smartphone or a PC.

(28) Further, the counterpart communication device may be a smartphone or a PC.

According to the present invention, it is possible to make a call without a complicated operation such as pressing a button.

In addition, by linking the wearable device and the pen-type device, there is an effect that it is possible to perform operations such as a hands-free operation.
Furthermore, by providing a pen-type device that can transmit voice information and trajectory information to the other party at the same time and is excellent in convenience, there is a great effect of the invention.

(1) When temporarily storing and transmitting voice information and trajectory information to the other party in a file format.
In any of the following, when the trajectory information is reproduced in real time and synchronized with the sound, the time elapses with either a pen-type device or your own smartphone (smartphone), personal computer (PC), wearable computer, etc. Must be added to the trajectory information.

(1-1) The pen-type device has a transmission function to a partner smartphone or PC (including a wearable computer, etc.).
● Voice acquisition means, storage means, synchronization means (later trace information is synchronized with the voice in real time, useless if it is not necessary to reproduce, etc.) is mounted on the pen-type device, The trajectory information is recorded on the pen-type device, and later the voice information and trajectory information are sent as files to the other party's smartphone or PC.

(1-2) There is no transmission function to the other party's smartphone or PC inside the pen-type device, and the own smartphone, the trajectory information to the PC, and the acquired voice information (unnecessary when using the smartphone's voice acquisition / recording means) When sending and recording, and then sending trajectory information and audio information as a file from your smartphone or PC to the other party's smartphone or PC.
● Pen-type devices do not have voice acquisition means and storage means; they record voice with their own smartphone, PC voice acquisition means, and storage means, and send trajectory information from the pen-type device to their own smartphone or PC in real time. Record.
● The pen-type device is equipped with only the trajectory information storage means, and the voice is recorded by the smartphone, PC voice acquisition means, and storage means. At this time, in order to synchronize the voice information and the trajectory information on the own smartphone / PC, information indicating the passage of time may be added to the trajectory information on either the pen-type device or the own smartphone / PC. To ensure the accuracy of synchronization between information and trajectory information, it is better to synchronize with your own smartphone or PC. Since the trace information can be transmitted almost in real time, there is almost no delay. In addition, since smartphones and PCs are overwhelmingly faster than pen-type devices, it is desirable to have smartphones and PCs perform complex processing.
From the pen-type device, track information is sent as a file to a smartphone or PC for recording. If you do not need to record audio information in sync while drawing the trajectory, you can later send and record the trajectory information as a file, browse the trajectory information on your smartphone or PC, and overdub May be recorded.
● The pen-type device is equipped with voice acquisition means and storage means, which records voice information and trajectory information on the pen, and later sends the voice information and trajectory information as a file to its own smartphone or PC. Record. At this time, in order to synchronize the voice information and the trajectory information with the pen-type device, it is desirable to add information indicating the passage of time to the trajectory information.

(2) When transmitting voice information and trajectory information to the other party in real time.
In any of the following cases, when the trajectory information is reproduced in real time in synchronization with the voice, it is necessary to add information indicating the passage of time to the trajectory information by the pen-type device, the partner smartphone, or the PC.

(2-1) When there is a transmission function to the other party's smartphone or PC inside the pen-type device.
● Send and record voice information and track information in real time from the pen-type device to the other party's smartphone or PC. When sending voice information and trajectory information using communication means with different speeds, the other party's smartphone and PC will be delayed, so the time elapses in order to synchronize the voice information and trajectory information on the pen-type device. It is desirable to add the indicated information to the trajectory information. There is no problem if the speed of both communication means is sufficiently high.
● A voice output means is provided inside the pen-type device to receive voice information from the other party's smartphone and PC in real time and to have a conversation.
-A display can be mounted on a pen-type device, or a display can be provided as an external device of a pen-type device, and voice information can be transmitted while browsing trajectory information. In particular, when a watch-type or glasses-type wearable computer or the like is used as the display, a writing medium can be used with one hand and a pen-type device can be used with one hand, so that it can be used while moving and convenience is improved.
● Share trajectory information with the other party's smartphone and PC, and both sides can add handwriting to the trajectory information being browsed and send and receive each other. You can talk at the same time. By displaying drawings, photographs and document instructions on the display during work, and drawing from above, it is possible to achieve advanced communication that does not convey only words.

(2-2) Using the various pen-type devices shown in (1), track information and voice information (useless when using a smartphone or PC voice acquisition function) via your own smartphone or PC When sending to a PC ● Send voice information and trajectory information in real time from your smartphone or PC to the other party's smartphone or PC.
● You can share the trajectory information between your own smartphone, PC and the other party's smartphone, PC, and both sides can add handwriting to the trajectory information you are browsing and send and receive each other. You can talk at the same time. The same thing can be done with current smartphones and PCs using the touch panel, but writing can be done without worrying about handwriting accuracy and screen size, and the list of paper media is clearly superior. In addition, since handwritten paper media and digital data including voice can be recorded, it is extremely effective in the field of storing paper media.

1 is a perspective view of a pen-type device according to a first embodiment of the present invention. It is a block diagram which shows the internal structure of the pen-type device which concerns on 1st embodiment of this invention. It is explanatory drawing which shows the use condition of the pen-type device which concerns on 1st embodiment of this invention. It is a perspective view which shows the modification of the pen-type device which concerns on 1st embodiment of this invention. It is explanatory drawing which shows the paper controller used with a pen-type device. It is explanatory drawing which shows the use condition of the pen-type device which concerns on 2nd embodiment of this invention. It is explanatory drawing which shows the state which performs mail transmission using the pen type device and wristwatch type wearable device which concern on 2nd embodiment of this invention. It is explanatory drawing which shows the state which performs a telephone call using the pen-type device and wristwatch-type wearable device which concern on 2nd embodiment of this invention. It is explanatory drawing which shows the other state which performs a telephone call using the pen-type device and wristwatch type wearable device which concern on 2nd embodiment of this invention. It is explanatory drawing which shows the state which transmits from a smart phone using the pen type device and wristwatch type wearable device which concern on 2nd embodiment of this invention. It is a modification of 2nd embodiment of this invention, and is explanatory drawing which shows the state using a glasses-type wearable device. It is a modification of 2nd embodiment of this invention, and is explanatory drawing which shows the state which uses the earphone type wearable device. It is a flowchart which shows the operation | movement of the telephone transmission using a pen type device. It is a figure explaining the use condition of the handwriting apparatus which concerns on 3rd embodiment of this invention. It is a block diagram which shows the internal structure of the handwriting apparatus which concerns on 3rd embodiment of this invention. It is a figure explaining the use condition of the handwriting apparatus which concerns on 3rd embodiment of this invention. It is a block diagram which shows the internal structure of the handwriting apparatus which concerns on 3rd embodiment of this invention. It is a figure explaining the sticky note for dot pattern printing used in 3rd embodiment of this invention. It is a flowchart explaining the operation | movement in Example 1-1 of 3rd embodiment of this invention. It is a flowchart explaining the operation | movement in Example 1-2 of 3rd embodiment of this invention. It is a flowchart explaining the operation | movement in Example 1-3 of 3rd embodiment of this invention. It is a flowchart explaining the operation | movement in Example 2-1 of 3rd embodiment of this invention. It is a flowchart explaining the operation | movement in Example 2-2 of 3rd embodiment of this invention. It is a figure explaining the smart note used in 3rd embodiment of this invention. It is a figure explaining the synchronization of locus information.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
<First Embodiment>
1 and 2 are diagrams illustrating a pen-type device according to the present invention.

The pen-type device in the first embodiment has a dot pattern reading function and a telephone function.

As shown in FIG. 2, the pen-type device includes a pen body. Although not shown, the pen body includes a CPU, a memory, and the like.

When the CPU reads a program or data stored in the memory of the pen body, the acquisition unit, the control unit, and the recognition unit function.

The obtaining means obtains specific information (telephone number or the like) for identifying the other party on the basis of information inputted by hand when making a call to the other party. The control means controls the start of the call when receiving a call from the other party. The recognizing unit recognizes contents input by handwriting by the user and contents input by voice through a microphone.

As an input unit of the pen body, an image pickup unit that captures a dot pattern, a voice acquisition unit such as a microphone, an instruction unit such as a pen tip, and a reception unit that receives an incoming call from a call partner are connected to the pen body. Yes.

The imaging means images the dot pattern when the user performs handwriting input. That is, the handwriting input is performed on the paper on which the dot pattern is printed.

Here, the dot technology, which is a configuration unique to the present invention, will be described.

In the present invention, the “dot pattern” is information or a numerical value encoded by a plurality of dot arrangement algorithms.

For information on how to encode information or numerical values using dot patterns, refer to Japanese Patent No. 3706385, Japanese Patent No. 3805551, Japanese Patent No. 3771252, Japanese Patent No. 4834872, Japanese Patent The dot pattern disclosed in detail in No. 4899199, etc. can be used. Typical dot pattern standards include Grid Onput (registered trademark), Anoto pattern, and the like.

In addition, there are several dot pattern standards on the market at the time of the present invention, but in the present invention, in addition to the dot patterns that currently exist, all dot patterns including dot patterns that will be developed in the future should be used. Is possible.

Also, it is preferable that the dot pattern is an invisible pattern that is invisible (or difficult to see) and can be superimposed on a normal design. However, when printing using so-called stealth ink, a normally visible pattern such as a QR code (registered trademark) may be used.

In addition, it is preferable that the standard of the dot pattern is a standard that can pattern the coordinate value and the code value by one format. The Grid Onput (registered trademark) described above can pattern coordinate values and code values in one format.

In the output part of the pen main body, infrared irradiation means such as an IR-LED, sound output means such as a speaker, and transmission means for transmitting to the other party are connected to the pen main body.

FIG. 3 is a diagram illustrating the usage state of the pen-type device.

The user uses the pen-type device to hand-write the telephone number of the other party (calling party) that he / she wants to call on the paper on which the dot pattern is printed. Then, the imaging unit captures a dot pattern, and the recognition unit recognizes the locus. Then, the acquisition unit acquires the recognition result. The transmission means makes a call to the other party on the basis of the recognition result. For example, when the user inputs handwritten as “080-XXXXXX-XXXXXX”, transmission is performed to “080-XXXXXX-XXXXXX”.

Also, the pen type device can receive incoming calls from the other party. In this case, the receiving means receives an incoming call from the call partner, and the control means controls the start of the call.

Thus, by providing a dot function reading pen with a telephone function, it is possible to make a call without performing a complicated operation of pressing a button and inputting a telephone number. Since this pen-type device can be used as a normal handwriting pen, it is particularly convenient for those who use the handwriting pen on a daily basis.

The pen-type device is not limited to the above-described embodiment. For example, the following modifications are possible.
(1) Providing a Display on a Pen-Type Device FIG. 4 is a diagram showing a state where a display is provided on a pen-type device.

As a result, the user can confirm the handwritten telephone number on the display. In addition, when receiving an incoming call from the other party, the telephone number of the other party is displayed, so that it is readily known who the incoming call is from.
(2) Character input by optical character recognition (OCR) In the above-described embodiment, character input is performed using a dot pattern. However, the present invention is not limited to this, and character input may be performed using optical character recognition.

In this case, an optical character recognition sensor is built in the pen-type device. When the user scans the handwritten phone number with the optical character recognition sensor after handwriting the phone number or while handwriting, the recognition means recognizes the scanned image. Then, the acquisition unit acquires the recognition result. The transmission means makes a call to the other party based on the recognition result.

According to this, it is not necessary to use a dedicated sheet such as a sheet on which a dot pattern is printed, and handwriting input can be performed using a normal sheet.
(3) Using a paper controller In the present invention, it is possible to use a paper controller when inputting a telephone number.

A paper controller is a print medium that simulates a controller that performs various operation instructions. On the print medium, an icon on which characters, graphics, and the like are drawn, and a dot pattern in which information corresponding to the icon is encoded are superimposed and printed. When the user touches an icon with a pen-type device, processing corresponding to the content of the icon is performed.

FIG. 5 is a diagram for explaining the paper controller used in this embodiment. This paper controller can be provided in a notebook, for example. On the paper controller, icons such as “call”, “end call”, and “send mail” are printed. When making a call, the user touches a telephone number in the address book (see FIG. 8) and then touches “call”. Then, the telephone number is recognized by the optical recognition sensor, and a call is made to the touched telephone number.

Note that an alphanumeric icon may be provided on the paper controller, and the telephone number may be input by touching the icon.
(4) Using voice recognition In the present invention, it is also possible to input a telephone number using voice recognition.

The pen-type device is provided with voice recognition means as the recognition means. When the user utters a telephone number toward the microphone, the voice is analyzed by the voice recognition means, and a call is made to the telephone number.

In the above-described embodiment, the transmission of the telephone number has been described. However, the present invention is not limited to this, and it is needless to say that the present invention may be used for transmission of e-mails and other types of messages using SNS and applications. is there. The same applies to the second embodiment described below.
<Second Embodiment>
Next, a second embodiment of the present invention will be described.

In the second embodiment, a wearable device and a pen-type device are used.

The pen-type device used in the present embodiment has the same basic structure as that of the first embodiment, but is for specifying the input information for the wearable device and for transmitting the input information to the wearable device. The difference is that the transmission means is provided. Further, the call function (transmitting means, receiving means) may not be provided.

FIG. 6 is a diagram showing a state in which content handwritten with a pen-type device is displayed on a wristwatch type wearable device (hereinafter referred to as “listwatch”).

The user performs handwriting input using a pen-type device. Then, the imaging unit captures a dot pattern, and the recognition unit recognizes the locus. The specifying unit specifies input information based on the recognition result. Then, the transmission means transmits the input information to the wearable device.

In FIG. 6, the user is handwriting the letters “apple”. When this character is recognized and specified as input information, the transmission means transmits information of “apple” to the wristwatch. Then, the handwritten character “apple” is displayed on the dial (display) of the wristwatch.

It should be noted that the displayed characters are not handwritten characters but may be text data converted by OCR.

FIG. 7 is a diagram showing a state where mail transmission is performed from the wristwatch.

The user writes a message (for example, “Konichiwa”) in the notebook and designates the destination address. Then touch “Mail Send” on the paper controller. Then, mail transmission is performed from the wristwatch.

Here, the destination e-mail address can be designated from a smartphone or tablet (hereinafter referred to as “smartphone”). In this case, an address book such as a smartphone is opened, and the “up” and “down” icons of the paper controller are touched to specify the mail address.

You can also specify an email address from your notebook. In this case, when the mail address of the transmission destination is touched, the mail address is recognized by the optical recognition sensor, and transmission to the touched mail address is performed.

FIG. 8 is a diagram showing a state in which a telephone call is made from the wristwatch.

The user touches the telephone number in the address book, and then touches “Call” on the paper controller. Then, the telephone number is recognized by the optical recognition sensor, and a call is made from the wristwatch to the touched telephone number.

In addition, when specifying a telephone number, you may specify from a smart phone etc. similarly to the case of FIG. Open the address book on your smartphone, etc., and touch the “Up” and “Down” icons on the paper controller to specify the phone number.

FIG. 9 is a diagram for explaining a state in which the wristwatch does not have a call function and only a pen-type device has a call function.

The user designates the destination telephone number in the same manner as in FIG. 8, and touches “Call” on the paper controller. Then, the telephone number is recognized by the optical recognition sensor, and a call is made from the pen-type device to the touched telephone number.

At this time, the wristwatch display displays a call destination telephone number and a picture indicating that a call is being made from a pen-type device.

This allows the user to accurately recognize that a call is being made.

FIG. 10 is a diagram for explaining a case of making a call from a smartphone.

In this embodiment, the call function of the pen-type device or wristwatch is not used, and the call is made from the smartphone.

In this case as well, the telephone number input method is the same as the method described in FIGS. When “call” on the paper controller is touched, a call is made from the smartphone. At the same time, the destination watch phone number and a picture indicating that the call is being sent from the smartphone are displayed on the wristwatch display.

11 and 12 are diagrams for explaining other forms of the wearable device.

FIG. 11 shows a case where a glasses-type wearable device is used. FIG. 12 shows a case where an earphone-type wearable device is used.

In both wearable devices, the pen-type device and the paper controller are used in the same manner as the wristwatch described above.

In glasses-type wearable devices, when a call is made, the phone number of the callee can be seen through the glasses. In the earphone-type wearable device, it is possible to make a call through the earphone by being wirelessly connected to the pen-type device.

Note that an earphone-type wearable device may be connected to a glasses-type wearable device. In this case, a bone conduction microphone or speaker may be used.

As described above, by using the pen-type device and the wearable device in conjunction with each other, it becomes possible to make a hands-free call and the convenience is greatly enhanced.

In addition, although this embodiment demonstrated telephone transmission and e-mail transmission, of course, this invention is applicable not only to this but all other operations.

In the first and second embodiments, handwritten input is performed on a paper on which a dot pattern is printed. However, the present invention is not limited to this, and other media such as an input board in which a dot pattern is formed on a magnetic board. May be used. Further, as the pen, other than a normal ballpoint pen, a disappearing ballpoint pen may be used.

FIG. 13 is a flowchart showing an operation of making a call.

The user uses the pen-type device to handwrite the telephone number of the other party on the paper on which the dot pattern is printed. Then, the imaging unit captures a dot pattern, and the recognition unit recognizes the locus (step S131). For example, as shown in FIG. 3, when the user handwrites “080-XXXX-XXXX”, the recognition means recognizes the locus. Next, the acquisition unit specifies a telephone number from the recognized locus (step S132). In the example of FIG. 3, it is specified that the destination telephone number is “080-XXXX-XXXX”. Next, the calling means makes a call to the specified telephone number (step S133). In the example of FIG. 3, transmission to “080-XXXX-XXXX” is performed. Next, the control means recognizes that the other party answered the call, that is, connected to the other party's telephone number (step S134), and starts a call (step S135).

<Third embodiment>
Next, a third embodiment of the present invention will be described.

14 and 16 are diagrams for explaining the third embodiment. In the third embodiment, the pen-type device is used to transmit locus information such as pictures and characters handwritten on the dot pattern printing paper simultaneously with a call.

15 and 17 are diagrams illustrating a pen-type device used in the third embodiment.

FIG. 15 shows a pen-type device having only a function as a handwriting pen, and includes a dot pattern imaging unit, a control unit, an analysis unit, a storage unit, a synchronization unit, and a transmission unit. Note that the storage means and the synchronization means may not be provided.

FIG. 17 shows a pen-type device that can be directly transmitted to and received from the other party's smartphone, etc., and includes dot pattern imaging means, voice acquisition means, voice information receiving means, locus receiving means, analysis means, control means, synchronization means, voice output means. The transmission means is provided. Further, the display device is connected wirelessly or by wire. An example of a display device is a thin display. By connecting a display, you can view handwritten pictures and characters sent from the other party without using a smartphone or PC. As shown in FIG. 4, the display device may be provided in the pen-type device body.

In the third embodiment, a plurality of examples can be considered. This will be described below.
<Example 1>
In the first embodiment, voice information and trajectory information are recorded on either a pen-type device, or one's own smartphone (smartphone) or personal computer (PC), and then the voice information and trajectory information are recorded on the destination smartphone or PC. Will be described.
Example 1-1
In this embodiment, voice information and trajectory information are recorded on the own smartphone and PC. Voice information is recorded using the smartphone's voice recording device, and the trajectory information is sent from the pen-type device to the user's own smartphone or PC in real time for recording.

According to this, it is not necessary to provide the voice acquisition device and the recording device in the pen type device, and there is an effect that the cost can be reduced and the size of the pen type device can be made compact.
Example 1-2
In this embodiment, voice information and trajectory information are recorded on the pen-type device. In this case, it is necessary to connect the pen type device to its own smartphone or PC later and transmit the information.

According to this, voice information can be recorded on the pen-type device while handwriting without connecting to a smartphone or PC. For this reason, it is possible to use a child for learning or at a work site without carrying or connecting a smartphone or a pen-type device, which is extremely convenient.
(Example 1-3)
In this embodiment, trajectory information is recorded on a pen-type device, only voice information is recorded on its own smartphone and PC, and the trajectory information is later transmitted to its own smartphone and PC.

According to this, since it is not necessary to record voice information at the same time, there is no need for a voice acquisition device, and there is an effect that the cost can be reduced and the pen size can be made compact.

Further, when the voice information is not recorded at the same time as the trajectory information, the same effect as the embodiment 1-2 is obtained. In this case, the audio information may be recorded later while the trajectory information is reproduced on its own smartphone or PC.

In Examples 1-1 to 1-3, in order to reproduce the trajectory information in synchronization with the audio information, information indicating the passage of time may be included in the trajectory information.

(Dot pattern printing sticky note)
Next, the dot pattern printing paper used for realizing the above embodiments 1-1 to 1-3 will be described with reference to FIG.

The dot pattern printing paper in the figure is configured as a sticky note, and is hereinafter referred to as “dot pattern printing sticky note”.

The dot pattern printed sticky note is composed of a writing area and an icon area.

The writing area is an area for the user to draw a document or a drawing. The trajectory information handwritten in the writing area is recorded on a pen, own smartphone, or PC. When handwritten input is performed in a paused state, insertion can be performed from that position.

The icon area is an area where icons for the user to perform various operations are printed. In this embodiment, seven icons are printed. From left to right, the icon functions are erase, record, play, pause, color, send, and page number. When the user touches the icon with the pen-type device, the operation assigned to the icon is executed.

“Erase” icon is an icon for erasing audio information. When recording, playing, or touching within 2 seconds after the end of playback, the message “If you want to erase, touch the trash again will erase all the audio.” Appears on the smartphone. Alternatively, when voice information is recorded on the pen-type device, a similar message is reproduced by voice. Touching again within 3 seconds erases the audio information. If you do not touch for more than 3 seconds, it will go into standby mode. When audio information is recorded on a smartphone or PC, this function may be provided with a similar icon and operated with a touch panel or a mouse. When other track information-related functions are also recorded on the smartphone or PC, similar icons may be provided on the smartphone or PC and operated on the smartphone or PC by operating the touch panel or the mouse.

The “Recording” icon is an icon for starting recording on the own smartphone, PC or handwriting pen. Touch again during recording to end recording. If another icon is touched during recording, the recording is terminated and operation processing of the touched icon is executed. If an audio file already exists, an additional audio file is generated. When another icon is touched, operation processing of the touched icon is executed, but the operation status is transmitted in real time.

The “play” icon is an icon for playing back sound and trajectory information. During pause, playback is performed in real time from the paused state. At the end (when all trajectory data is displayed), playback is performed in real time from the beginning. However, if it is touched again within one second after the play button is pressed, it is set to double speed, and if touched again within one second, it is set to quadruple speed.

The “pause” icon is an icon for performing a pause. If touched while paused, playback starts at the previous speed from the pause position. When touching or writing of another icon is started during the pause, the paused state is canceled and the touched icon is processed or written.

The “color” icon is an icon for determining the color of the locus drawn with the pen-type device. The left is black and the right is red. In addition, although not shown, there may be a function of setting the thickness of the locus.

The “Send” icon is an icon for sending the trace information and voice information handwritten on the page to the destination smartphone or PC. Set the destination address on your smartphone.

The “page number” icon is an icon indicating each page number (1 to 100) when a dot pattern printing sticky note is provided in a bundle of a plurality of pages. When this icon is touched, the entire trajectory data of the page is displayed. Since the locus information includes an area specifying code for specifying the page, various processes can be performed in conjunction with the page number.

In addition, in order to execute the playback / pause / insertion function of the present embodiment, time lapse information must be added to the trajectory information.

一般 General sticky notes can be used as your own memos by drawing simple documents and diagrams, and can convey information to the other party. However, since many sticky notes have a small area, detailed information cannot be conveyed. In the dot pattern printing sticky note, since voice information can be added, the explanation of the memo can be conveyed in detail. In addition, general sticky notes are disposable, and it is difficult to organize and store information. However, with dot pattern printing sticky notes, the trajectory information is digitized and recorded on smartphones and PCs in the past. Information can be easily recorded, organized and utilized. Note that this embodiment does not need to be a sticky note, and it is needless to say that a similar function may be installed in a notebook or the like.

Next, the operation of the pen-type device (pen), the own smartphone / PC, and the other party's smartphone / PC in the first embodiment will be described with reference to FIGS.

In addition, in the same figure, although it described as "the own smartphone" and "the other party's smartphone", it is of course applicable to the own PC, wearable computer, the other party's PC, and the wearable computer.

FIG. 19 is a flowchart for explaining the operation in the embodiment 1-1.

The user performs handwriting input on a dot pattern printing tag using a pen. At the same time, or immediately before or after that, the voice is uttered. For example, draw a map on a dot pattern printing sticky note, and by voice, go out the XX line XX station south exit ticket gate and go straight. Turn left at the second intersection and go straight two blocks, then the second intersection Turn right and turn left at the first intersection and you will see your home on your right. "

In the pen, the image pickup means picks up the dot pattern, and the recognition means recognizes the locus and acquires the locus information. The pen transmits the acquired trajectory information to its own smartphone wirelessly or by wire (step S1601). Then, the own smartphone receives the trajectory information (step S1602) and records it in the recording device (step S1604).

Meanwhile, the user's voice is recorded in the recording device of his / her smartphone (step S1605).

The own smartphone transmits the recorded voice information and trajectory information to the destination smartphone (step S1607).

The other party's smartphone receives the voice information and the trajectory information (step S1608). Then, audio information is output from the speaker, and the trajectory information is displayed on the display (step S1609). In order to reproduce the audio information and the trajectory information without any contradiction later, synchronization (step S1603) is required. The synchronization will be described in detail later.

FIG. 20 is a flowchart for explaining the operation in the embodiment 1-2.

In the pen, the imaging means images the dot pattern, the recognition means recognizes the locus, and obtains the locus information. Then, the trajectory information is recorded in a recording device built in the pen (step S1701). Also, the voice information uttered by the user is acquired and recorded in the recording device (step S1702). Then, the recorded voice information and trajectory information are transmitted to the own smartphone (step S1707).

When the own smartphone receives the voice information and the trajectory information (step S1706), it transmits it to the partner smartphone (step S1708).

The other party's smartphone receives voice information and trajectory information (step S1708). Then, audio information is output from the speaker, and the trajectory information is displayed on the display (step S1709). In order to reproduce the audio information and the trajectory information without any contradiction later, synchronization (step S1701) is required. The synchronization will be described in detail later.

FIG. 21 is a flowchart for explaining the operation in the embodiment 1-3.

The user performs handwriting input on a dot pattern printing tag using a pen. Here, the pen and its own smartphone are not connected, and only the trajectory information needs to be recorded. When the trajectory information is reproduced in real time later, it is necessary to record the information indicating the handwritten time passage in the trajectory information together with the coordinate information as shown in FIG. At the same time, or immediately before or after that, the voice is uttered. For example, draw a map on a dot pattern printing sticky note, and by voice, go out the XX line XX station south exit ticket gate and go straight. Turn left at the second intersection and go straight two blocks, then the second intersection Turn right and turn left at the first intersection and you will see your home on your right. "

In the pen, the imaging means images the dot pattern, the recognition means recognizes the locus, and obtains the locus information. Then, the trajectory information is recorded in a recording device built in the pen (step S1801). Then, the recorded trajectory information is transmitted to its own smartphone (step S1802).

The own smartphone receives the trajectory information (step S1803) and records it in the recording device (step S1805). Further, the voice information uttered by the user is acquired and recorded in the recording device (step S1807). Then, the recorded voice information and trajectory information are transmitted to the destination smartphone (step S1808).

The partner's smartphone receives the voice information and the trajectory information (step S1809). Then, audio information is output from the speaker, and the trajectory information is displayed on the display (step S1810).
<Example 2>
Next, operations of the pen-type device (pen), the own smartphone / PC, and the other party's smartphone / PC in the second embodiment will be described with reference to FIGS.

In the second embodiment, a case will be described in which voice information and trajectory information are transmitted and received in real time to a partner smartphone or PC directly from a pen-type device or via the user's smartphone or PC.
Example 2-1
FIG. 19 is a flowchart for explaining the operation in the embodiment 2-1.

In this embodiment, only the trajectory information is transmitted from the pen type device to the other party's smartphone or PC in real time via the own smartphone or PC. When the trajectory information is received by the own smartphone or PC, it is desirable to record information indicating the elapsed time for synchronization in the trajectory information as shown in FIG.

According to this, since it uses its own smartphone, PC voice acquisition means and voice output means, the pen does not need to be equipped with voice-related devices and storage devices, reducing costs and reducing pen size. There is an effect that can be made.
(Example 2-2)
FIG. 20 is a flowchart for explaining the operation in the embodiment 2-2.

In this embodiment, voice information and trajectory information are transmitted directly from the pen-type device to the other party's smartphone or PC. In this case, a voice acquisition means and a voice output means are provided inside the pen-type device, and voice information from the other party's smartphone or PC can be received in real time for conversation.

According to this, there is no need for a self-smartphone or PC, and it is possible to use it for children's learning and work, and it is extremely convenient. Furthermore, if a display is mounted on a pen-type device or a display is provided as an external device of the pen-type device, voice information can be transmitted and received while browsing and sharing trajectory information.

In Examples 2-1 and 2-2, in order to reproduce the trajectory information in synchronization with the audio information (step S187), information indicating the passage of time may be included in the trajectory information. Here, when a voice is uttered in accordance with the history of the trajectory while a trajectory is being drawn, if the voice and a point on the trajectory do not match, a contradiction will occur. In particular, when voice information and trajectory information are transmitted to the other party's smartphone using different communication means, a delay occurs depending on the communication speed. Therefore, in order to output the audio information and the trajectory information in synchronization with each other, the other party's smartphone records time lapse information for synchronization in the trajectory information as shown in FIG. Originally, since the voice information is reproduced in real time, if the time information of the starting point is matched by a predetermined method, the other party's smartphone can output without delay between the voice information and the trajectory information. There is no need to synchronize if the speeds of the communication means for different voice information and trajectory information are sufficiently high. However, it is indispensable to synchronize when recording voice information and trajectory information on the self and the other party's smartphone and reproducing them without delay later.
(Dot pattern printing sticky note)
Next, the dot pattern printing paper used for realizing the above Examples 2-1 and 2-2 will be described.

The dot pattern printing paper shown in the figure is configured as a dot pattern printing sticky note as in Examples 1-1 to 1-3.

The dot pattern printing sticky note in this embodiment has the same appearance as that used in Embodiments 1-1 to 1-3, that is, the same as that shown in FIG. 15, but has some functions specific to this embodiment. Yes.

Hereinafter, only functions different from those in the embodiments 1-1 to 1-3 will be described.

If you touch the “Erase” icon in this example during playback of audio information or within 2 seconds after the end of playback, the message “If you want to delete, touch the trash again will delete all the audio.” Appears on the smartphone. indicate. Alternatively, when voice information is recorded on the pen-type device, a similar message is reproduced by voice. Within 3 seconds, touching again will delete the audio information. If you do not touch for more than 3 seconds, it will be in a standby state. A similar icon may be provided on your smartphone or PC and operated with a touch panel or mouse. When other trajectory information-related functions are recorded on the own smartphone or PC, a similar icon may be provided and operated on the smartphone or PC by touch panel or mouse operation. Of course, the same operation may be performed on the other party's smartphone or PC.

When you touch the “Record” icon, when sending voice information to the other party's smartphone or PC, recording starts with your smartphone, PC, or pen with recording function. Touch again during recording to end recording. If another icon is touched during recording, the recording is terminated and operation processing of the touched icon is executed. If an audio file already exists, an additional audio file is generated. When another icon is touched, operation processing of the touched icon is executed, but the operation status is transmitted in real time. Even without recording, voice information is automatically sent to the other party's smartphone or PC.

When touching the “Send” icon, the trace information and voice information handwritten on the page are sent to the destination smartphone and PC. The destination address and phone number are set using a pen-type device or your own smartphone. When transmitting directly with a pen-type device, settings are made by touching a paper keyboard or address book on which dedicated dots are superimposed and printed. The paper keyboard and the address book are those shown in FIG. 5, FIG. 8 to FIG. 10 and the description of the corresponding specification, for example.

In addition, when the user writes a figure or text in the writing area, the handwritten trajectory information is sent directly to the other party's smartphone or PC from the pen-type device or via their own smartphone or PC. To be recorded.
In this embodiment, a dot pattern printing tag is used to transmit information to the other party's smartphone or PC. However, it is not limited to this, by drawing simple documents and drawings, as a memo, or by the other party digitizing the voice information and locus information of the received dot pattern printed tag and recording it on a smartphone or PC , Past information can be easily recorded, organized and utilized. It should be noted that this embodiment does not have to be a dot pattern printing sticky note, and it is needless to say that a similar function may be installed in a notebook or the like.

Next, the operations of the pen-type device, the own smartphone / PC, and the other party's smartphone / PC in the second embodiment will be described with reference to FIGS.

FIG. 22 is a flowchart for explaining the operation in the embodiment 2-1.

In the pen, the image pickup means picks up the dot pattern, and the recognition means recognizes the locus and acquires the locus information. The pen transmits the acquired trajectory information to its own smartphone wirelessly or by wire (step S1901). Then, the own smartphone receives the trajectory information (step S1904) and transmits it to the other party's smartphone (step S1906). The partner's smartphone receives the trajectory information (step S196).

On the other hand, its own smartphone acquires the user's voice information. Then, the voice information is transmitted to the destination smartphone (step S1903). The partner's smartphone receives the voice information (step S1907).

The other party's smartphone outputs the received voice information from the speaker, and displays the received trajectory information on the display (step S1909).

FIG. 23 is a flowchart for explaining the operation in the embodiment 2-2.

In the pen, the image pickup means picks up the dot pattern, and the recognition means recognizes the locus and acquires the locus information. The pen transmits the acquired trajectory information to the destination smartphone wirelessly or by wire (step S2002). Then, the other party's smartphone receives the trajectory information (step S2005). Also, the voice information uttered by the user is acquired and transmitted to the destination smartphone (step S2004). Then, the other party's smartphone receives the trajectory information (step S2006).

The partner's smartphone outputs the received voice information from the speaker, and displays the received trajectory information on the display.
<Example 3> (Smart Note)
Next, another example of dot pattern printing paper used for realizing the above embodiments 1-1 to 1-3 and 2-1 to 2-2 will be described with reference to FIG.

The dot pattern printing paper in the figure is configured as a notebook and is hereinafter referred to as “smart notebook”. In this example, as a lecture / practice notebook related to education and a notebook / notebook used by ordinary people, it has a function to record trajectory information and voice information, and a function to play back and browse information sent to the user's own or other party's smartphone or PC. Explains. The method for transmitting and receiving the locus information and the sound information is the same as that of the above-described dot pattern printing tag, and a similar transmission icon may be provided. The icons arranged on the smart notebook may be operated with a touch panel or a mouse by arranging similar icons on the operation screen of a smartphone or PC.

Smart note is composed of writing area and icon area.

The writing area is an area for the user to draw a document or a drawing.

The icon area is an area where icons for the user to perform various operations are printed. In the present embodiment, icon areas are formed at the upper and lower parts of the smart notebook. The function of the icon is from the left, erase, start, end, play, double speed, pause, return, send, control bar, link, bottom from left, size, color, eraser, erase display, display, MP4, JPEG, link, link deletion.

The “erase” icon is an icon for erasing all the trajectory information of the page. A confirmation screen appears on the display. Touch it with your finger or press the ○ button on the pen to erase it. Originally, for the sake of safety, only finger touch is desired, but there is a case where it is desired to erase data when away from the tablet in the classroom, so the deletion operation with the pen is also enabled. However, erasing with the ○ button on the pen is valid only when confirmation is displayed on the tablet. In addition, the deleted track information can be restored on a smartphone or PC.

“Start” icon is an icon for starting voice recording. Touch the recording icon or the end icon again to stop recording. When another icon is touched, recording is terminated and the icon operation is performed. You can create any number of audio files / trajectory files.

“End” icon is an icon for ending the operation. Touching during voice recording ends recording. Touching the end icon while not recording will display the last track information of the display range.

The “play” icon is an icon for playing the operation. When this icon is touched, playback is performed in real time from the current state (including the pause state). All gap times in which no trajectory is drawn are skipped (a mode that does not skip in the setting screen can also be set). However, the gap time is not skipped in real-time playback during recording and playback. When the last image in the display range is displayed, playback is performed in real time from the beginning. During pause (including after moving the stroke) or during high-speed playback, playback is performed in real time from that position. When touched further, playback is performed at a speed of 2 times, 4 times, 8 times, or 16 times.

The “double speed” icon is an icon for playing at double speed. When this icon is touched, it is played back at high speed at 4x speed from the beginning. All gap times where no trajectory is drawn are skipped. Furthermore, when touched, it is played back at a speed of 8 or 16 times.

The “pause” icon is an icon for pausing. During pause (including after moving the stroke), playback starts at the speed before pause from the pause position. Even if you pause and touch another page, the paused state is saved. Note that writing is possible even during pause, and the trajectory information is inserted from that position and reflected during playback. If you move to another page and return during a pause, the pause screen is displayed. If pause is touched during writing and playback is performed in real time, playback is started from the position where writing was completed.

The “return” icon is an icon for returning one stroke while the trajectory information is displayed.

The “Send” icon is an icon for sending one stroke while the trajectory information is displayed.

The “control bar” icon is an icon for controlling the display range. Further, from the left, an icon in which a leftward triangle is drawn (hereinafter, “leftmost icon”), a time lapse bar, and a rightward triangle are drawn. Icon (hereinafter “rightmost icon”). Touching the leftmost icon displays the top state of the display range. When the display range is in the initial state, if the problem, lecture content, manual, etc. are in the background, the background is displayed. When the time lapse bar is touched, images up to the touched position are displayed. When the pen type device is moved on the time lapse bar while the pen type device is in contact with the touch, the trajectory information up to the moving position is displayed one after another. Note that when moving to another page during playback and returning, the last image in the display range is displayed. If you move to another page and return during a pause, the pause screen is displayed.

The “Link” icon is an icon in which an image or a movie is linked. Touching this icon displays the linked image / movie. You can also overwrite and record the image / video. The link icon may be composed of a sticker or the like, and may be added and pasted on a note.

The “size” icon is an icon for determining and changing the size of the pen and eraser. Pen and eraser sizes are not recorded on a page-by-page basis. The last used information is saved. However, in the case of the eraser, when moving to another page, the pen is automatically set and the pen information used last time is adopted.

The “color” icon is an icon for determining and changing the color of the pen and eraser. Pen and eraser colors are not recorded on a page-by-page basis. The last used information is saved. However, in the case of the eraser, when moving to another page, the pen is automatically set and the pen information used last time is adopted.

"Eraser" icon is an icon for erasing the locus and area during handwriting. The size is selected by the thickness of the pen. If you do not select a size, it will be the size used for the previous eraser. While erasing, a yellow circle is displayed, but no trace information is displayed.

The “erase display” icon is an icon for displaying an erased locus or area. The erased trace is displayed as it is, and the traced eraser trace is displayed in yellow and translucent. In addition, when a pen is selected from the eraser state, the color and size of the pen selected previously are selected.

The “display” icon is an icon for canceling the hide and displaying all the trajectory information.

The “Hide” icon is an icon for a reverse display function in which the currently displayed trajectory information is hidden and the range other than that is the display range. Playback, insertion, and addition operations can be performed only within the display range. When displaying the second half, playback from the beginning, pause at a predetermined position, turn OFF and play. If you want to display the first half, you can hide the first half, display only the second half, and then turn it off. The display range is saved for each page.

The “MPEG4” icon is an icon for generating a moving image file. When this icon is touched, a setting screen appears. Select which holder of the smartphone or PC to record, and select one of the following (1) to (3). Select execution / stop with the touch panel or mouse, or start recording with the pen ○ button. When recording starts, the screen returns to the smart note screen and a moving image is recorded. Note that the MP4 icon is touched again when recording is ended during moving image recording. In that case, a moving image file until the end is generated. The video may use a format other than MP4.
(1) Animation of all operations such as trajectory, recording and playback in real time.
(2) All the trajectory and audio information of the page are animated.
(3) Only the trajectory of the display range and audio information are animated.
Audio information that spans the display range is played back from the beginning of recording, and when it enters the display range, a moving image is recorded. In the case of a general video such as MP4, even if this smart notebook application is not installed, it can be viewed on any smartphone or PC, increasing convenience.

The “JPEG” icon is an icon for recording the trajectory information being displayed. When this icon is touched, the recording destination holder is displayed and the recording location is set. Select execution / stop by finger touch or record with pen ○ button.

The “link” icon is an icon for performing link setting. The link setting is a process of associating the dot pattern printed on the above-described link icon with an image / moving image. All processing is performed on smartphones and PCs.

The “link deletion” icon is an icon for deleting the display of the link image.

<Example 4>
Example 4 explains the transmission data format of a handwritten pen.

There are various methods such as Bluetooth (registered trademark), BLE, WIFI, RF, and various telephone lines for wireless communication between a handwriting pen and its own smartphone or PC, or wireless communication with a partner smartphone or PC.

In order to realize handwriting, as the frame rate, trajectory information including coordinate values of 100 times or more per second for characters and 150 times or more for 1 second for figures and pictures is required. Recently, data transmission / reception by BLE, which is part of the Bluetooth (registered trademark) 4.0 standard, which is standard on smartphones, is widely used. BLE has extremely high power savings, but since it can only perform data communication at 1 Mbps at the maximum, it is necessary to minimize the transmission of trajectory information.

In the present invention, only when the pen is touched on the writing area, the dot pattern analysis of the captured dot pattern is performed, the code value and XY coordinates defined in one format are read, and the writing area shown below is specified. Outputs trajectory information along with the code (8 bytes). This makes it possible to easily manage the trajectory information for each writing area / page, and to display / edit / transmit only the handwritten trajectory information by designating a predetermined writing area / page. . Further, since the stroke is drawn in the same writing area / page after touching the writing area to minimize the trajectory information, it is not necessary to output the area specifying code or the status code. Therefore, only the trajectory information based on the XY coordinate values is output (4 bytes), and the amount of transmission information is greatly reduced. This makes it possible to send the necessary frame rate trajectory information by BLE. As the status code, various information such as pen down / pen up, writing area, handwritten pen button information, error information, and the like are output.

(1) XY coordinates
8byte: Send format when pen down
0 0Xff: Header (1)
1 0xFD: Header (2), a header to indicate the data delimiter during transmission. Since it is read only once at the moment when the pen is touched, 2 bytes are used so that there is no misunderstanding of the data separator at the time of transmission.
2 Status code
3 Area identification code (1): 8bit
4 Area identification code (2): 8bit
Area specification code 16bit, about 6500 writing area / page can be specified. Since the XY coordinate value can define 2 mm in 1 bit, at present, the XY coordinate is 8 bits each, and a 512 × 512 mm square area is one writing area. As in post-it, when the writing area is small, the area specifying code can be increased by reducing the range of the XY coordinate values. In the case of a 64 × 64mm square writing area, the XY coordinates can be defined with 5 bits each for a total of 10 bits. As a result, 6 bits can be added to the area identification code, and about 4.2 million write areas / pages can be specified with 22 bits.
5 X coordinate: 8bit
6 Y coordinate: 8bit
7 XY coordinates after the decimal point: Since the bit (4 bits x 2) and the XY coordinate value after the decimal point are defined as 2 mm in 1 bit, the accuracy is low, so the accuracy can be reduced to 2 mm / 16 = 0.125 mm by interpolation calculation. Secured.

4byte: Transmission format during pen down
0 0Xff: Header to indicate data delimiter during transmission. Even if the header is misrecognized, it is sent 100 times or more per second, so if you read the next header, there is no problem even if one piece of data is missing, so it is only 1 byte. .
1 X coordinate: 8bit
2 Y coordinate: 8bit
3 XY coordinates below the decimal point: bits below the decimal point (4 bits x 2)

The following is the code value that is output when the dot pattern printed on the dot pattern printed sticky note shown in the first and second embodiments and the smart pattern shown in the third embodiment is superimposed on the icon and read with a pen. Format. The operation codes corresponding to the following icons have 32 bits and can be associated with up to about 4 billion operation instructions and contents.
(2) Code value (8-byte transmission)
0 0xFF: Header (1)
1 0Xfd: Header (2), a header to indicate data delimiters during transmission. Since it is read only once at the moment when the pen is touched, 2 bytes are used so that there is no misunderstanding of the data separator at the time of transmission.
2 Status byte
3 Operation code (1)
4 Operation code (2)
5 Operation code (3)
6 Operation code (4)
7 Rotation angle (2 degree unit: ± 1 degree) 0 to 179 (0 degree ± 1 degree to 358 degree ± 1 degree)
0 degree (rotation code 0) is 359 degrees to less than 1 degree, 2 degrees (rotation code 1) is 1 degree to less than 3 degrees, 358 degrees (rotation code 179) is 357 degrees to less than 359 degrees, 2 Send a rotation code every degree. When touching with a pen to output linked content, it is possible to obtain and output information that varies depending on the rotation angle, such as when the pen is rotated to change the sound or content, or when viewing images 360 degrees .

The above-described embodiment is an example, and it is needless to say that various changes can be made without changing the gist. Further, each embodiment can be freely combined and selectively used.

Since the pen-type device according to the present invention can be easily used by a disabled person or an elderly person who is not good at telephone operation, industrial applicability to the welfare field is conceivable.

Of course, it can be used in all people and fields.

Claims

A handwriting device for drawing a locus on a medium on which a dot pattern is formed,
An imaging unit that images the coordinate pattern and the code value, or the dot pattern in which the code value is defined;
By analyzing the dot pattern imaged by the imaging means, the analyzing means for outputting at least trajectory information and / or code value consisting of a history of coordinate information obtained from the coordinate value;
Audio acquisition means for acquiring audio information;
Transmission means for transmitting the voice information and at least one of the trajectory information or the code value to a communication device,
In the writing area of the medium on which the dot pattern is formed, the coordinate value and the code value are defined, and the code value has an area specifying code for specifying the writing area,
The handwriting device, wherein the analyzing means outputs the locus information including the region specifying code.
A handwriting device for drawing a locus on a medium on which a dot pattern is formed,
An imaging unit that images the coordinate pattern and the code value, or the dot pattern in which the code value is defined;
By analyzing the dot pattern imaged by the imaging means, the analyzing means for outputting at least trajectory information and / or code value consisting of a history of coordinate information obtained from the coordinate value;
Transmission means for transmitting the voice information and at least one of the trajectory information or the code value to a communication device,
In the writing area of the medium on which the dot pattern is formed, the coordinate value and the code value are defined, and the code value has an area specifying code for specifying the writing area,
The analysis unit outputs the locus information including the region specifying code, and the communication device acquires voice information.
Synchronizing means for synchronizing with the voice information by adding information defining the passage of time to the trajectory information, the handwriting device or the communication device,
The handwriting device according to claim 1 or 2, further comprising:
The communication device is a counterpart communication device used by a third party.
The handwriting device according to claim 1 or 3.
The communication device is a communication device used by itself,
The transmission means transmits the voice information and at least one of the trajectory information or code value to the counterpart communication device used by a third party via the communication device.
The handwriting device according to any one of claims 1 to 3.
The trajectory information identifies the counterpart communication device according to claim 4 or 5.
The handwriting device according to any one of claims 1 to 5.
Storage means for storing at least one of the trajectory information and the audio information is stored in either the handwriting device or the communication device.
In addition,
The transmission means or the transmission means provided in the communication device transmits at least one of the voice information and the trajectory information stored in the storage means as a file. Handwriting device.
Display means for displaying at least the trajectory information that is either built-in or externally connected to at least one of the handwriting device or the communication device,
The handwriting device according to any one of claims 1 to 7, further comprising:
The display means outputs the audio information in synchronization with the trajectory information.
The handwriting device according to any one of claims 1 to 8.
Voice output means for outputting received voice information is at least one of the handwriting device or the communication device,
The handwriting device according to any one of claims 1 to 9, further comprising:
A voice receiving device that receives voice information from the counterpart communication device according to any one of claims 4 to 6, and at least one of the handwriting device and the communication device,
The handwriting device according to any one of claims 1 to 10, further comprising:
A trajectory receiving device for receiving trajectory information from the counterpart communication device according to any one of claims 4 to 6, wherein at least one of the handwriting device or the communication device,
The handwriting device according to any one of claims 1 to 11, further comprising:
The trajectory information further drawn by the handwriting device with respect to the trajectory information is transmitted to the counterpart communication device or the communication device according to any one of claims 4 to 6.
The handwriting device according to claim 12.
The handwriting device according to any one of claims 1 to 13, wherein the communication device is a smartphone or a PC.
The handwriting device according to any one of claims 1 to 14, wherein the counterpart communication device according to any one of claims 4 to 6 is a smartphone or a PC.
A handwriting device for drawing a locus on a medium on which a dot pattern is formed,
An imaging unit that images the coordinate pattern and the code value, or the dot pattern in which the code value is defined;
By analyzing the dot pattern imaged by the imaging means, the analyzing means for outputting at least trajectory information and / or code value consisting of a history of coordinate information obtained from the coordinate value;
Transmission means for transmitting at least one of the trajectory information and the code value to a communication device,
The coordinate value and the code value are defined in the writing area of the medium on which the dot pattern is formed, the code value has an area specifying code for specifying the writing area, and the analysis means includes the trajectory information A handwriting device that outputs the region specifying code including
The communication device for obtaining voice information;
A voice / handwritten communication system.
Synchronizing means that synchronizes with the voice information by adding information defining the passage of time to the trajectory information to at least one of the handwriting device or the communication device, and at least one of the handwriting device or the communication device In addition,
The voice / handwritten communication system according to claim 16, further comprising:
The communication device is a communication device used by itself,
The transmission means transmits the voice information and at least one of the trajectory information or code value to the counterpart communication device used by a third party via the communication device.
The voice / handwritten communication system according to claim 16 or 17.
The voice / handwritten communication system according to claim 18, wherein the counterpart communication device is specified by the trajectory information.
The at least one of the handwriting device or the communication device further comprises storage means for storing at least one of the trajectory information and the audio information in at least one of the handwriting device or the communication device,
The transmission means or the transmission means provided in the communication device transmits at least one of the voice information and the trajectory information stored in the storage means as a file. Voice / handwritten communication system.
Display means for displaying at least the trajectory information that is either built-in or connected to the outside, at least one of the handwriting device or the communication device,
The speech / handwritten communication system according to any one of claims 16 to 20, further comprising:
The display means outputs the audio information in synchronization with the trajectory information.
The handwriting device according to claim 21.
Voice output means for outputting the received voice information to the communication device,
The voice / handwritten communication system according to any one of claims 16 to 22, further comprising:
An audio receiving means for receiving audio information from the counterpart communication device according to claim 18 or 19, wherein the communication device includes:
The speech / handwritten communication system according to any one of claims 16 to 23, further comprising:
A trajectory receiving means for receiving trajectory information from the counterpart communication device according to claim 18 or 19 is provided in the communication device.
The speech / handwritten communication system according to any one of claims 16 to 24, further comprising:
With respect to the locus information, the locus information further drawn by the handwriting device is transmitted to the counterpart communication device according to claim 18 or 19 via the communication device.
The voice / handwritten communication system according to any one of claims 16 to 25.
The voice / handwritten communication system according to any one of claims 16 to 26, wherein the communication device is a smartphone or a PC.
28. The voice / handwritten communication system according to any one of claims 16 to 27, wherein the counterpart communication device according to claim 18 or 19 is a smartphone or a PC.