WO2017010469A1 - Dispositif d'écriture manuscrite et système de communication vocale/d'écriture manuscrite - Google Patents

Dispositif d'écriture manuscrite et système de communication vocale/d'écriture manuscrite Download PDF

Info

Publication number
WO2017010469A1
WO2017010469A1 PCT/JP2016/070484 JP2016070484W WO2017010469A1 WO 2017010469 A1 WO2017010469 A1 WO 2017010469A1 JP 2016070484 W JP2016070484 W JP 2016070484W WO 2017010469 A1 WO2017010469 A1 WO 2017010469A1
Authority
WO
WIPO (PCT)
Prior art keywords
information
communication device
voice
handwriting
trajectory information
Prior art date
Application number
PCT/JP2016/070484
Other languages
English (en)
Japanese (ja)
Inventor
吉田　健治
Original Assignee
株式会社Ｉ・Ｐソリューションズ
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 株式会社Ｉ・Ｐソリューションズ filed Critical 株式会社Ｉ・Ｐソリューションズ
Priority to JP2017528681A priority Critical patent/JP6973791B2/ja
Publication of WO2017010469A1 publication Critical patent/WO2017010469A1/fr
Priority to JP2021175186A priority patent/JP2022020703A/ja

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
    • G06F3/042—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by opto-electronic means
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/16—Sound input; Sound output

Definitions

  • the present invention relates to a pen-type device having a telephone function and a dot pattern reading function.
  • the present invention also relates to a pen-type device used in conjunction with a wearable device.
  • the present inventor has made many innovative inventions using dot patterns.
  • This invention is made in view of such a point, and makes it a technical subject to provide the pen-type device which has a telephone function and a dot pattern reading function.
  • a handwriting device is a handwriting device for drawing a locus on a medium on which a dot pattern is formed, and images the dot pattern in which coordinate values and code values or code values are defined.
  • Voice acquisition means for acquiring; and transmission means for transmitting at least one of the voice information and the trajectory information or code value to a communication device, and the writing area of the medium on which the dot pattern is formed includes A coordinate value and a code value are defined, and the code value has an area specifying code for specifying the writing area, and the solution
  • the analyzing means outputs the locus information including the region specifying code.
  • a handwriting device is a handwriting device for drawing a locus on a medium on which a dot pattern is formed, and images the dot pattern in which coordinate values and code values or code values are defined.
  • Imaging means for outputting trajectory information and / or code values comprising at least a coordinate information history obtained from the coordinate values by code analysis of the dot pattern imaged by the imaging means; and the audio information Transmitting means for transmitting at least one of the locus information or the code value to a communication device, and the coordinate value and the code value are defined in a writing area of the medium on which the dot pattern is formed,
  • the code value has an area specifying code for specifying the writing area, and the analysis means specifies the area in the trajectory information.
  • the communication device includes a code and outputs the voice information.
  • the handwriting device or the communication device may further include synchronization means for adding information defining time passage to the trajectory information to synchronize with the voice information.
  • the communication device may be a counterpart communication device used by a third party.
  • the communication device is a communication device used by itself, and the transmission means sends the voice information and the locus to the counterpart communication device used by a third party via the communication device.
  • Information and / or code values may be transmitted.
  • the counterpart communication device may be specified by the trajectory information.
  • a storage unit that stores at least one of the trajectory information and the voice information is further provided in either the handwriting device or the communication device, and the transmission unit or the transmission provided in the communication device
  • the means may transmit at least one of the voice information and the trajectory information stored in the storage means as a file.
  • At least one of the handwriting device and the communication device further includes display means for displaying at least the trajectory information that is either built-in or connected to the outside. Also good.
  • the display means may output the audio information in synchronization with the trajectory information.
  • voice output means for outputting the received voice information may be further provided in at least one of the handwriting device and the communication device.
  • At least one of the handwriting device and the communication device may further include a voice reception device that receives voice information from the counterpart communication device.
  • a trajectory receiving device for receiving trajectory information from the counterpart communication device according to any one of claims 4 to 6 may be further provided in at least one of the handwriting device and the communication device. Good.
  • trajectory information further drawn by the handwriting device may be transmitted to the counterpart communication device or the communication device according to any one of claims 4 to 6 with respect to the trajectory information.
  • the communication device may be a smartphone or a PC.
  • the counterpart communication device may be a smartphone or a PC.
  • a voice / handwriting communication system is a handwriting device for drawing a locus on a medium on which a dot pattern is formed, wherein the coordinate value and code value or the dot pattern in which the code value is defined
  • Imaging means for picking up images, and analyzing means for outputting trajectory information and / or code values consisting of at least a history of coordinate information obtained from the coordinate values by code analysis of the dot pattern imaged by the imaging means;
  • Transmitting means for transmitting at least one of the trajectory information and the code value to a communication device, and the coordinate value and the code value are defined in a writing area of the medium on which the dot pattern is formed.
  • the value has an area specifying code for specifying the writing area, and the analysis means adds the area specifying code to the trajectory information.
  • a communication device for acquiring voice information.
  • synchronization means for synchronizing information with the audio information by adding information defining time passage to the trajectory information to at least one of the handwriting device or the communication device, the handwriting device or the communication You may further provide in at least any of apparatuses.
  • the communication device is a communication device used by itself, and the transmission means sends the voice information and the locus to the other party communication device used by a third party via the communication device.
  • Information and / or code values may be transmitted.
  • the counterpart communication device may be specified by the trajectory information.
  • At least one of the handwriting device and the communication device further includes display means for displaying at least the trajectory information that is either built-in or connected to the outside. May be.
  • the display means may output the audio information in synchronization with the trajectory information.
  • an audio output means for outputting the received audio information is provided to the communication device. Further, it may be provided.
  • the communication device may further include voice receiving means for receiving voice information from the counterpart communication device.
  • the communication device may further include trajectory receiving means for receiving trajectory information from the counterpart communication device.
  • the trajectory information further drawn by the handwriting device may be transmitted to the counterpart communication device via the communication device with respect to the trajectory information.
  • the communication device may be a smartphone or a PC.
  • the counterpart communication device may be a smartphone or a PC.
  • trajectory information When temporarily storing and transmitting voice information and trajectory information to the other party in a file format.
  • the trajectory information when the trajectory information is reproduced in real time and synchronized with the sound, the time elapses with either a pen-type device or your own smartphone (smartphone), personal computer (PC), wearable computer, etc. Must be added to the trajectory information.
  • the pen-type device has a transmission function to a partner smartphone or PC (including a wearable computer, etc.).
  • ⁇ Voice acquisition means, storage means, synchronization means (later trace information is synchronized with the voice in real time, useless if it is not necessary to reproduce, etc.) is mounted on the pen-type device, The trajectory information is recorded on the pen-type device, and later the voice information and trajectory information are sent as files to the other party's smartphone or PC.
  • Pen-type devices do not have voice acquisition means and storage means; they record voice with their own smartphone, PC voice acquisition means, and storage means, and send trajectory information from the pen-type device to their own smartphone or PC in real time. Record.
  • the pen-type device is equipped with only the trajectory information storage means, and the voice is recorded by the smartphone, PC voice acquisition means, and storage means.
  • information indicating the passage of time may be added to the trajectory information on either the pen-type device or the own smartphone / PC.
  • the trace information can be transmitted almost in real time, there is almost no delay.
  • smartphones and PCs are overwhelmingly faster than pen-type devices, it is desirable to have smartphones and PCs perform complex processing. From the pen-type device, track information is sent as a file to a smartphone or PC for recording.
  • the pen-type device is equipped with voice acquisition means and storage means, which records voice information and trajectory information on the pen, and later sends the voice information and trajectory information as a file to its own smartphone or PC. Record. At this time, in order to synchronize the voice information and the trajectory information with the pen-type device, it is desirable to add information indicating the passage of time to the trajectory information.
  • -A display can be mounted on a pen-type device, or a display can be provided as an external device of a pen-type device, and voice information can be transmitted while browsing trajectory information.
  • a watch-type or glasses-type wearable computer or the like is used as the display, a writing medium can be used with one hand and a pen-type device can be used with one hand, so that it can be used while moving and convenience is improved.
  • Share trajectory information with the other party's smartphone and PC, and both sides can add handwriting to the trajectory information being browsed and send and receive each other. You can talk at the same time.
  • FIG. 1 is a perspective view of a pen-type device according to a first embodiment of the present invention. It is a block diagram which shows the internal structure of the pen-type device which concerns on 1st embodiment of this invention. It is explanatory drawing which shows the use condition of the pen-type device which concerns on 1st embodiment of this invention. It is a perspective view which shows the modification of the pen-type device which concerns on 1st embodiment of this invention. It is explanatory drawing which shows the paper controller used with a pen-type device. It is explanatory drawing which shows the use condition of the pen-type device which concerns on 2nd embodiment of this invention. It is explanatory drawing which shows the state which performs mail transmission using the pen type device and wristwatch type wearable device which concern on 2nd embodiment of this invention.
  • First Embodiment> 1 and 2 are diagrams illustrating a pen-type device according to the present invention.
  • the pen-type device in the first embodiment has a dot pattern reading function and a telephone function.
  • the pen-type device includes a pen body.
  • the pen body includes a CPU, a memory, and the like.
  • the CPU reads a program or data stored in the memory of the pen body, the acquisition unit, the control unit, and the recognition unit function.
  • the obtaining means obtains specific information (telephone number or the like) for identifying the other party on the basis of information inputted by hand when making a call to the other party.
  • the control means controls the start of the call when receiving a call from the other party.
  • the recognizing unit recognizes contents input by handwriting by the user and contents input by voice through a microphone.
  • an image pickup unit that captures a dot pattern
  • a voice acquisition unit such as a microphone
  • an instruction unit such as a pen tip
  • a reception unit that receives an incoming call from a call partner
  • the imaging means images the dot pattern when the user performs handwriting input. That is, the handwriting input is performed on the paper on which the dot pattern is printed.
  • the “dot pattern” is information or a numerical value encoded by a plurality of dot arrangement algorithms.
  • Japanese Patent No. 3706385 Japanese Patent No. 3805551, Japanese Patent No. 3771252, Japanese Patent No. 4834872, Japanese Patent
  • the dot pattern disclosed in detail in No. 4899199, etc. can be used.
  • Typical dot pattern standards include Grid Onput (registered trademark), Anoto pattern, and the like.
  • the dot pattern is an invisible pattern that is invisible (or difficult to see) and can be superimposed on a normal design.
  • a normally visible pattern such as a QR code (registered trademark) may be used.
  • the standard of the dot pattern is a standard that can pattern the coordinate value and the code value by one format.
  • the Grid Onput (registered trademark) described above can pattern coordinate values and code values in one format.
  • infrared irradiation means such as an IR-LED
  • sound output means such as a speaker
  • transmission means for transmitting to the other party are connected to the pen main body.
  • FIG. 3 is a diagram illustrating the usage state of the pen-type device.
  • the user uses the pen-type device to hand-write the telephone number of the other party (calling party) that he / she wants to call on the paper on which the dot pattern is printed. Then, the imaging unit captures a dot pattern, and the recognition unit recognizes the locus. Then, the acquisition unit acquires the recognition result.
  • the transmission means makes a call to the other party on the basis of the recognition result. For example, when the user inputs handwritten as “080-XXXXXX-XXXXX”, transmission is performed to “080-XXXXXX-XXXXX”.
  • the pen type device can receive incoming calls from the other party.
  • the receiving means receives an incoming call from the call partner, and the control means controls the start of the call.
  • this pen-type device can be used as a normal handwriting pen, it is particularly convenient for those who use the handwriting pen on a daily basis.
  • FIG. 4 is a diagram showing a state where a display is provided on a pen-type device.
  • Character input by optical character recognition In the above-described embodiment, character input is performed using a dot pattern. However, the present invention is not limited to this, and character input may be performed using optical character recognition.
  • an optical character recognition sensor is built in the pen-type device.
  • the recognition means recognizes the scanned image.
  • the acquisition unit acquires the recognition result.
  • the transmission means makes a call to the other party based on the recognition result.
  • a paper controller is a print medium that simulates a controller that performs various operation instructions.
  • On the print medium an icon on which characters, graphics, and the like are drawn, and a dot pattern in which information corresponding to the icon is encoded are superimposed and printed.
  • processing corresponding to the content of the icon is performed.
  • FIG. 5 is a diagram for explaining the paper controller used in this embodiment.
  • This paper controller can be provided in a notebook, for example.
  • icons such as “call”, “end call”, and “send mail” are printed.
  • the user touches a telephone number in the address book (see FIG. 8) and then touches “call”. Then, the telephone number is recognized by the optical recognition sensor, and a call is made to the touched telephone number.
  • an alphanumeric icon may be provided on the paper controller, and the telephone number may be input by touching the icon.
  • voice recognition it is also possible to input a telephone number using voice recognition.
  • the pen-type device is provided with voice recognition means as the recognition means.
  • voice recognition means as the recognition means.
  • the voice is analyzed by the voice recognition means, and a call is made to the telephone number.
  • a wearable device and a pen-type device are used.
  • the pen-type device used in the present embodiment has the same basic structure as that of the first embodiment, but is for specifying the input information for the wearable device and for transmitting the input information to the wearable device.
  • the difference is that the transmission means is provided.
  • the call function transmitting means, receiving means
  • FIG. 6 is a diagram showing a state in which content handwritten with a pen-type device is displayed on a wristwatch type wearable device (hereinafter referred to as “listwatch”).
  • the user performs handwriting input using a pen-type device. Then, the imaging unit captures a dot pattern, and the recognition unit recognizes the locus. The specifying unit specifies input information based on the recognition result. Then, the transmission means transmits the input information to the wearable device.
  • the transmission means transmits information of “apple” to the wristwatch. Then, the handwritten character “apple” is displayed on the dial (display) of the wristwatch.
  • displayed characters are not handwritten characters but may be text data converted by OCR.
  • FIG. 7 is a diagram showing a state where mail transmission is performed from the wristwatch.
  • the user writes a message (for example, “Konichiwa”) in the notebook and designates the destination address. Then touch “Mail Send” on the paper controller. Then, mail transmission is performed from the wristwatch.
  • a message for example, “Konichiwa”
  • “Mail Send” on the paper controller. Then, mail transmission is performed from the wristwatch.
  • the destination e-mail address can be designated from a smartphone or tablet (hereinafter referred to as “smartphone”).
  • a smartphone or tablet hereinafter referred to as “smartphone”.
  • an address book such as a smartphone is opened, and the “up” and “down” icons of the paper controller are touched to specify the mail address.
  • the mail address of the transmission destination is touched, the mail address is recognized by the optical recognition sensor, and transmission to the touched mail address is performed.
  • FIG. 8 is a diagram showing a state in which a telephone call is made from the wristwatch.
  • the user touches the telephone number in the address book, and then touches “Call” on the paper controller. Then, the telephone number is recognized by the optical recognition sensor, and a call is made from the wristwatch to the touched telephone number.
  • FIG. 9 is a diagram for explaining a state in which the wristwatch does not have a call function and only a pen-type device has a call function.
  • the user designates the destination telephone number in the same manner as in FIG. 8, and touches “Call” on the paper controller. Then, the telephone number is recognized by the optical recognition sensor, and a call is made from the pen-type device to the touched telephone number.
  • the wristwatch display displays a call destination telephone number and a picture indicating that a call is being made from a pen-type device.
  • FIG. 10 is a diagram for explaining a case of making a call from a smartphone.
  • the call function of the pen-type device or wristwatch is not used, and the call is made from the smartphone.
  • the telephone number input method is the same as the method described in FIGS.
  • “call” on the paper controller is touched, a call is made from the smartphone.
  • the destination watch phone number and a picture indicating that the call is being sent from the smartphone are displayed on the wristwatch display.
  • 11 and 12 are diagrams for explaining other forms of the wearable device.
  • FIG. 11 shows a case where a glasses-type wearable device is used.
  • FIG. 12 shows a case where an earphone-type wearable device is used.
  • the pen-type device and the paper controller are used in the same manner as the wristwatch described above.
  • glasses-type wearable devices when a call is made, the phone number of the callee can be seen through the glasses.
  • the earphone-type wearable device it is possible to make a call through the earphone by being wirelessly connected to the pen-type device.
  • an earphone-type wearable device may be connected to a glasses-type wearable device.
  • a bone conduction microphone or speaker may be used.
  • handwritten input is performed on a paper on which a dot pattern is printed.
  • the present invention is not limited to this, and other media such as an input board in which a dot pattern is formed on a magnetic board. May be used.
  • the pen other than a normal ballpoint pen, a disappearing ballpoint pen may be used.
  • FIG. 13 is a flowchart showing an operation of making a call.
  • the user uses the pen-type device to handwrite the telephone number of the other party on the paper on which the dot pattern is printed. Then, the imaging unit captures a dot pattern, and the recognition unit recognizes the locus (step S131). For example, as shown in FIG. 3, when the user handwrites “080-XXXX-XXXX”, the recognition means recognizes the locus. Next, the acquisition unit specifies a telephone number from the recognized locus (step S132). In the example of FIG. 3, it is specified that the destination telephone number is “080-XXXX-XXX”. Next, the calling means makes a call to the specified telephone number (step S133). In the example of FIG. 3, transmission to “080-XXXX-XXX” is performed.
  • control means recognizes that the other party answered the call, that is, connected to the other party's telephone number (step S134), and starts a call (step S135).
  • the pen-type device is used to transmit locus information such as pictures and characters handwritten on the dot pattern printing paper simultaneously with a call.
  • 15 and 17 are diagrams illustrating a pen-type device used in the third embodiment.
  • FIG. 15 shows a pen-type device having only a function as a handwriting pen, and includes a dot pattern imaging unit, a control unit, an analysis unit, a storage unit, a synchronization unit, and a transmission unit. Note that the storage means and the synchronization means may not be provided.
  • FIG. 17 shows a pen-type device that can be directly transmitted to and received from the other party's smartphone, etc., and includes dot pattern imaging means, voice acquisition means, voice information receiving means, locus receiving means, analysis means, control means, synchronization means, voice output means.
  • the transmission means is provided.
  • the display device is connected wirelessly or by wire.
  • An example of a display device is a thin display. By connecting a display, you can view handwritten pictures and characters sent from the other party without using a smartphone or PC. As shown in FIG. 4, the display device may be provided in the pen-type device body.
  • Example 1 In the first embodiment, voice information and trajectory information are recorded on either a pen-type device, or one's own smartphone (smartphone) or personal computer (PC), and then the voice information and trajectory information are recorded on the destination smartphone or PC. Will be described.
  • Example 1-1 In this embodiment, voice information and trajectory information are recorded on the own smartphone and PC. Voice information is recorded using the smartphone's voice recording device, and the trajectory information is sent from the pen-type device to the user's own smartphone or PC in real time for recording.
  • Example 1-2 voice information and trajectory information are recorded on the pen-type device. In this case, it is necessary to connect the pen type device to its own smartphone or PC later and transmit the information.
  • trajectory information is recorded on a pen-type device, only voice information is recorded on its own smartphone and PC, and the trajectory information is later transmitted to its own smartphone and PC.
  • the voice information is not recorded at the same time as the trajectory information, the same effect as the embodiment 1-2 is obtained.
  • the audio information may be recorded later while the trajectory information is reproduced on its own smartphone or PC.
  • Examples 1-1 to 1-3 in order to reproduce the trajectory information in synchronization with the audio information, information indicating the passage of time may be included in the trajectory information. (Dot pattern printing sticky note) Next, the dot pattern printing paper used for realizing the above embodiments 1-1 to 1-3 will be described with reference to FIG.
  • the dot pattern printing paper in the figure is configured as a sticky note, and is hereinafter referred to as “dot pattern printing sticky note”.
  • the dot pattern printed sticky note is composed of a writing area and an icon area.
  • the writing area is an area for the user to draw a document or a drawing.
  • the trajectory information handwritten in the writing area is recorded on a pen, own smartphone, or PC.
  • insertion can be performed from that position.
  • the icon area is an area where icons for the user to perform various operations are printed. In this embodiment, seven icons are printed. From left to right, the icon functions are erase, record, play, pause, color, send, and page number. When the user touches the icon with the pen-type device, the operation assigned to the icon is executed.
  • the “Recording” icon is an icon for starting recording on the own smartphone, PC or handwriting pen. Touch again during recording to end recording. If another icon is touched during recording, the recording is terminated and operation processing of the touched icon is executed. If an audio file already exists, an additional audio file is generated. When another icon is touched, operation processing of the touched icon is executed, but the operation status is transmitted in real time.
  • the “play” icon is an icon for playing back sound and trajectory information. During pause, playback is performed in real time from the paused state. At the end (when all trajectory data is displayed), playback is performed in real time from the beginning. However, if it is touched again within one second after the play button is pressed, it is set to double speed, and if touched again within one second, it is set to quadruple speed.
  • the “pause” icon is an icon for performing a pause. If touched while paused, playback starts at the previous speed from the pause position. When touching or writing of another icon is started during the pause, the paused state is canceled and the touched icon is processed or written.
  • the “color” icon is an icon for determining the color of the locus drawn with the pen-type device. The left is black and the right is red. In addition, although not shown, there may be a function of setting the thickness of the locus.
  • the “Send” icon is an icon for sending the trace information and voice information handwritten on the page to the destination smartphone or PC. Set the destination address on your smartphone.
  • the “page number” icon is an icon indicating each page number (1 to 100) when a dot pattern printing sticky note is provided in a bundle of a plurality of pages. When this icon is touched, the entire trajectory data of the page is displayed. Since the locus information includes an area specifying code for specifying the page, various processes can be performed in conjunction with the page number.
  • time lapse information must be added to the trajectory information.
  • General sticky notes can be used as your own memos by drawing simple documents and diagrams, and can convey information to the other party. However, since many sticky notes have a small area, detailed information cannot be conveyed. In the dot pattern printing sticky note, since voice information can be added, the explanation of the memo can be conveyed in detail. In addition, general sticky notes are disposable, and it is difficult to organize and store information. However, with dot pattern printing sticky notes, the trajectory information is digitized and recorded on smartphones and PCs in the past. Information can be easily recorded, organized and utilized. Note that this embodiment does not need to be a sticky note, and it is needless to say that a similar function may be installed in a notebook or the like.
  • FIG. 19 is a flowchart for explaining the operation in the embodiment 1-1.
  • the user performs handwriting input on a dot pattern printing tag using a pen.
  • the voice is uttered. For example, draw a map on a dot pattern printing sticky note, and by voice, go out the XX line XX station south exit ticket gate and go straight. Turn left at the second intersection and go straight two blocks, then the second intersection Turn right and turn left at the first intersection and you will see your home on your right. "
  • the image pickup means picks up the dot pattern, and the recognition means recognizes the locus and acquires the locus information.
  • the pen transmits the acquired trajectory information to its own smartphone wirelessly or by wire (step S1601). Then, the own smartphone receives the trajectory information (step S1602) and records it in the recording device (step S1604).
  • step S1605 the user's voice is recorded in the recording device of his / her smartphone.
  • the own smartphone transmits the recorded voice information and trajectory information to the destination smartphone (step S1607).
  • the other party's smartphone receives the voice information and the trajectory information (step S1608). Then, audio information is output from the speaker, and the trajectory information is displayed on the display (step S1609). In order to reproduce the audio information and the trajectory information without any contradiction later, synchronization (step S1603) is required. The synchronization will be described in detail later.
  • FIG. 20 is a flowchart for explaining the operation in the embodiment 1-2.
  • the user performs handwriting input on a dot pattern printing tag using a pen.
  • the voice is uttered. For example, draw a map on a dot pattern printing sticky note, and by voice, go out the XX line XX station south exit ticket gate and go straight. Turn left at the second intersection and go straight two blocks, then the second intersection Turn right and turn left at the first intersection and you will see your home on your right. "
  • the imaging means images the dot pattern
  • the recognition means recognizes the locus
  • the trajectory information is recorded in a recording device built in the pen (step S1701).
  • the voice information uttered by the user is acquired and recorded in the recording device (step S1702).
  • the recorded voice information and trajectory information are transmitted to the own smartphone (step S1707).
  • step S1706 When the own smartphone receives the voice information and the trajectory information (step S1706), it transmits it to the partner smartphone (step S1708).
  • the other party's smartphone receives voice information and trajectory information (step S1708). Then, audio information is output from the speaker, and the trajectory information is displayed on the display (step S1709). In order to reproduce the audio information and the trajectory information without any contradiction later, synchronization (step S1701) is required. The synchronization will be described in detail later.
  • FIG. 21 is a flowchart for explaining the operation in the embodiment 1-3.
  • the user performs handwriting input on a dot pattern printing tag using a pen.
  • the pen and its own smartphone are not connected, and only the trajectory information needs to be recorded.
  • the trajectory information is reproduced in real time later, it is necessary to record the information indicating the handwritten time passage in the trajectory information together with the coordinate information as shown in FIG.
  • the voice is uttered. For example, draw a map on a dot pattern printing sticky note, and by voice, go out the XX line XX station south exit ticket gate and go straight. Turn left at the second intersection and go straight two blocks, then the second intersection Turn right and turn left at the first intersection and you will see your home on your right. "
  • the imaging means images the dot pattern
  • the recognition means recognizes the locus
  • the trajectory information is recorded in a recording device built in the pen (step S1801).
  • the recorded trajectory information is transmitted to its own smartphone (step S1802).
  • the own smartphone receives the trajectory information (step S1803) and records it in the recording device (step S1805). Further, the voice information uttered by the user is acquired and recorded in the recording device (step S1807). Then, the recorded voice information and trajectory information are transmitted to the destination smartphone (step S1808).
  • the partner's smartphone receives the voice information and the trajectory information (step S1809). Then, audio information is output from the speaker, and the trajectory information is displayed on the display (step S1810).
  • Example 2 operations of the pen-type device (pen), the own smartphone / PC, and the other party's smartphone / PC in the second embodiment will be described with reference to FIGS.
  • FIG. 19 is a flowchart for explaining the operation in the embodiment 2-1.
  • trajectory information is transmitted from the pen type device to the other party's smartphone or PC in real time via the own smartphone or PC.
  • the trajectory information is received by the own smartphone or PC, it is desirable to record information indicating the elapsed time for synchronization in the trajectory information as shown in FIG.
  • FIG. 20 is a flowchart for explaining the operation in the embodiment 2-2.
  • voice information and trajectory information are transmitted directly from the pen-type device to the other party's smartphone or PC.
  • a voice acquisition means and a voice output means are provided inside the pen-type device, and voice information from the other party's smartphone or PC can be received in real time for conversation.
  • Examples 2-1 and 2-2 in order to reproduce the trajectory information in synchronization with the audio information (step S187), information indicating the passage of time may be included in the trajectory information.
  • information indicating the passage of time may be included in the trajectory information.
  • voice information and trajectory information are transmitted to the other party's smartphone using different communication means, a delay occurs depending on the communication speed. Therefore, in order to output the audio information and the trajectory information in synchronization with each other, the other party's smartphone records time lapse information for synchronization in the trajectory information as shown in FIG.
  • the other party's smartphone can output without delay between the voice information and the trajectory information. There is no need to synchronize if the speeds of the communication means for different voice information and trajectory information are sufficiently high. However, it is indispensable to synchronize when recording voice information and trajectory information on the self and the other party's smartphone and reproducing them without delay later. (Dot pattern printing sticky note) Next, the dot pattern printing paper used for realizing the above Examples 2-1 and 2-2 will be described.
  • the dot pattern printing paper shown in the figure is configured as a dot pattern printing sticky note as in Examples 1-1 to 1-3.
  • the dot pattern printing sticky note in this embodiment has the same appearance as that used in Embodiments 1-1 to 1-3, that is, the same as that shown in FIG. 15, but has some functions specific to this embodiment. Yes.
  • the trace information and voice information handwritten on the page are sent to the destination smartphone and PC.
  • the destination address and phone number are set using a pen-type device or your own smartphone.
  • settings are made by touching a paper keyboard or address book on which dedicated dots are superimposed and printed.
  • the paper keyboard and the address book are those shown in FIG. 5, FIG. 8 to FIG. 10 and the description of the corresponding specification, for example.
  • the handwritten trajectory information is sent directly to the other party's smartphone or PC from the pen-type device or via their own smartphone or PC.
  • a dot pattern printing tag is used to transmit information to the other party's smartphone or PC.
  • Past information can be easily recorded, organized and utilized. It should be noted that this embodiment does not have to be a dot pattern printing sticky note, and it is needless to say that a similar function may be installed in a notebook or the like.
  • FIG. 22 is a flowchart for explaining the operation in the embodiment 2-1.
  • the user performs handwriting input on a dot pattern printing tag using a pen.
  • the voice is uttered. For example, draw a map on a dot pattern printing sticky note, and by voice, go out the XX line XX station south exit ticket gate and go straight. Turn left at the second intersection and go straight two blocks, then the second intersection Turn right and turn left at the first intersection and you will see your home on your right. "
  • the image pickup means picks up the dot pattern, and the recognition means recognizes the locus and acquires the locus information.
  • the pen transmits the acquired trajectory information to its own smartphone wirelessly or by wire (step S1901). Then, the own smartphone receives the trajectory information (step S1904) and transmits it to the other party's smartphone (step S1906).
  • the partner's smartphone receives the trajectory information (step S196).
  • the voice information is transmitted to the destination smartphone (step S1903).
  • the partner's smartphone receives the voice information (step S1907).
  • the other party's smartphone outputs the received voice information from the speaker, and displays the received trajectory information on the display (step S1909).
  • FIG. 23 is a flowchart for explaining the operation in the embodiment 2-2.
  • the user performs handwriting input on a dot pattern printing tag using a pen.
  • the voice is uttered. For example, draw a map on a dot pattern printing sticky note, and by voice, go out the XX line XX station south exit ticket gate and go straight. Turn left at the second intersection and go straight two blocks, then the second intersection Turn right and turn left at the first intersection and you will see your home on your right. "
  • the image pickup means picks up the dot pattern, and the recognition means recognizes the locus and acquires the locus information.
  • the pen transmits the acquired trajectory information to the destination smartphone wirelessly or by wire (step S2002). Then, the other party's smartphone receives the trajectory information (step S2005). Also, the voice information uttered by the user is acquired and transmitted to the destination smartphone (step S2004). Then, the other party's smartphone receives the trajectory information (step S2006).
  • the partner's smartphone outputs the received voice information from the speaker, and displays the received trajectory information on the display.
  • Example 3 Smart Note
  • the dot pattern printing paper in the figure is configured as a notebook and is hereinafter referred to as “smart notebook”.
  • a lecture / practice notebook related to education and a notebook / notebook used by ordinary people it has a function to record trajectory information and voice information, and a function to play back and browse information sent to the user's own or other party's smartphone or PC.
  • the method for transmitting and receiving the locus information and the sound information is the same as that of the above-described dot pattern printing tag, and a similar transmission icon may be provided.
  • the icons arranged on the smart notebook may be operated with a touch panel or a mouse by arranging similar icons on the operation screen of a smartphone or PC.
  • Smart note is composed of writing area and icon area.
  • the writing area is an area for the user to draw a document or a drawing.
  • the icon area is an area where icons for the user to perform various operations are printed.
  • icon areas are formed at the upper and lower parts of the smart notebook.
  • the function of the icon is from the left, erase, start, end, play, double speed, pause, return, send, control bar, link, bottom from left, size, color, eraser, erase display, display, MP4, JPEG, link, link deletion.
  • the “erase” icon is an icon for erasing all the trajectory information of the page.
  • a confirmation screen appears on the display. Touch it with your finger or press the ⁇ button on the pen to erase it.
  • the deletion operation with the pen is also enabled.
  • erasing with the ⁇ button on the pen is valid only when confirmation is displayed on the tablet.
  • the deleted track information can be restored on a smartphone or PC.
  • Start icon is an icon for starting voice recording. Touch the recording icon or the end icon again to stop recording. When another icon is touched, recording is terminated and the icon operation is performed. You can create any number of audio files / trajectory files.
  • End icon is an icon for ending the operation. Touching during voice recording ends recording. Touching the end icon while not recording will display the last track information of the display range.
  • the “play” icon is an icon for playing the operation. When this icon is touched, playback is performed in real time from the current state (including the pause state). All gap times in which no trajectory is drawn are skipped (a mode that does not skip in the setting screen can also be set). However, the gap time is not skipped in real-time playback during recording and playback.
  • playback is performed in real time from the beginning. During pause (including after moving the stroke) or during high-speed playback, playback is performed in real time from that position. When touched further, playback is performed at a speed of 2 times, 4 times, 8 times, or 16 times.
  • the “double speed” icon is an icon for playing at double speed. When this icon is touched, it is played back at high speed at 4x speed from the beginning. All gap times where no trajectory is drawn are skipped. Furthermore, when touched, it is played back at a speed of 8 or 16 times.
  • the “pause” icon is an icon for pausing. During pause (including after moving the stroke), playback starts at the speed before pause from the pause position. Even if you pause and touch another page, the paused state is saved. Note that writing is possible even during pause, and the trajectory information is inserted from that position and reflected during playback. If you move to another page and return during a pause, the pause screen is displayed. If pause is touched during writing and playback is performed in real time, playback is started from the position where writing was completed.
  • the “return” icon is an icon for returning one stroke while the trajectory information is displayed.
  • the “Send” icon is an icon for sending one stroke while the trajectory information is displayed.
  • the “control bar” icon is an icon for controlling the display range. Further, from the left, an icon in which a leftward triangle is drawn (hereinafter, “leftmost icon”), a time lapse bar, and a rightward triangle are drawn. Icon (hereinafter “rightmost icon”). Touching the leftmost icon displays the top state of the display range. When the display range is in the initial state, if the problem, lecture content, manual, etc. are in the background, the background is displayed. When the time lapse bar is touched, images up to the touched position are displayed. When the pen type device is moved on the time lapse bar while the pen type device is in contact with the touch, the trajectory information up to the moving position is displayed one after another. Note that when moving to another page during playback and returning, the last image in the display range is displayed. If you move to another page and return during a pause, the pause screen is displayed.
  • the “Link” icon is an icon in which an image or a movie is linked. Touching this icon displays the linked image / movie. You can also overwrite and record the image / video.
  • the link icon may be composed of a sticker or the like, and may be added and pasted on a note.
  • the “size” icon is an icon for determining and changing the size of the pen and eraser. Pen and eraser sizes are not recorded on a page-by-page basis. The last used information is saved. However, in the case of the eraser, when moving to another page, the pen is automatically set and the pen information used last time is adopted.
  • the “color” icon is an icon for determining and changing the color of the pen and eraser. Pen and eraser colors are not recorded on a page-by-page basis. The last used information is saved. However, in the case of the eraser, when moving to another page, the pen is automatically set and the pen information used last time is adopted.
  • Eraser icon is an icon for erasing the locus and area during handwriting. The size is selected by the thickness of the pen. If you do not select a size, it will be the size used for the previous eraser. While erasing, a yellow circle is displayed, but no trace information is displayed.
  • the “erase display” icon is an icon for displaying an erased locus or area.
  • the erased trace is displayed as it is, and the traced eraser trace is displayed in yellow and translucent.
  • the color and size of the pen selected previously are selected.
  • the “display” icon is an icon for canceling the hide and displaying all the trajectory information.
  • the “Hide” icon is an icon for a reverse display function in which the currently displayed trajectory information is hidden and the range other than that is the display range. Playback, insertion, and addition operations can be performed only within the display range.
  • playback from the beginning pause at a predetermined position, turn OFF and play. If you want to display the first half, you can hide the first half, display only the second half, and then turn it off. The display range is saved for each page.
  • the “MPEG4” icon is an icon for generating a moving image file. When this icon is touched, a setting screen appears. Select which holder of the smartphone or PC to record, and select one of the following (1) to (3). Select execution / stop with the touch panel or mouse, or start recording with the pen ⁇ button. When recording starts, the screen returns to the smart note screen and a moving image is recorded. Note that the MP4 icon is touched again when recording is ended during moving image recording. In that case, a moving image file until the end is generated.
  • the video may use a format other than MP4.
  • All the trajectory and audio information of the page are animated. (3) Only the trajectory of the display range and audio information are animated.
  • Audio information that spans the display range is played back from the beginning of recording, and when it enters the display range, a moving image is recorded.
  • a general video such as MP4
  • MP4 even if this smart notebook application is not installed, it can be viewed on any smartphone or PC, increasing convenience.
  • the “JPEG” icon is an icon for recording the trajectory information being displayed. When this icon is touched, the recording destination holder is displayed and the recording location is set. Select execution / stop by finger touch or record with pen ⁇ button.
  • the “link” icon is an icon for performing link setting.
  • the link setting is a process of associating the dot pattern printed on the above-described link icon with an image / moving image. All processing is performed on smartphones and PCs.
  • the “link deletion” icon is an icon for deleting the display of the link image.
  • Example 4 explains the transmission data format of a handwritten pen.
  • Bluetooth registered trademark
  • BLE BLE
  • WIFI wireless fidelity
  • RF radio frequency
  • trajectory information including coordinate values of 100 times or more per second for characters and 150 times or more for 1 second for figures and pictures is required.
  • BLE which is part of the Bluetooth (registered trademark) 4.0 standard, which is standard on smartphones, is widely used.
  • BLE has extremely high power savings, but since it can only perform data communication at 1 Mbps at the maximum, it is necessary to minimize the transmission of trajectory information.
  • the dot pattern analysis of the captured dot pattern is performed, the code value and XY coordinates defined in one format are read, and the writing area shown below is specified.
  • the XY coordinate value can define 2 mm in 1 bit, at present, the XY coordinate is 8 bits each, and a 512 ⁇ 512 mm square area is one writing area.
  • the area specifying code can be increased by reducing the range of the XY coordinate values.
  • the XY coordinates can be defined with 5 bits each for a total of 10 bits. As a result, 6 bits can be added to the area identification code, and about 4.2 million write areas / pages can be specified with 22 bits.
  • the pen-type device according to the present invention can be easily used by a disabled person or an elderly person who is not good at telephone operation, industrial applicability to the welfare field is conceivable.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • General Health & Medical Sciences (AREA)
  • Telephone Function (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

La présente invention concerne un dispositif de stylo qui est très convivial et peut transmettre en même temps des informations vocales et des informations de lieu à un autre individu. Un dispositif d'écriture manuscrite est destiné à tracer un lieu sur un support sur lequel est formé un motif de points, ledit dispositif étant pourvu : d'un moyen de capture d'image qui capture une image d'un motif de points; d'un moyen d'analyse qui, à la suite de la réalisation d'une analyse de code du motif de points capturé par le moyen de capture d'image, transmet des informations de lieu et/ou des valeurs de code comprenant l'historique des informations de coordonnées obtenues à partir d'au moins des valeurs de coordonnées; d'un moyen d'acquisition de voix qui acquiert des informations vocals; et d'un moyen de transmission qui transmet, à un dispositif de communication, les informations vocales ainsi que les informations de lieu et/ou les valeurs de code. Une zone d'écriture du support sur lequel est formé un motif de points est définie par des valeurs de coordonnées et des valeurs de code et les valeurs de code comportent un code d'identification de zone qui identifie la zone d'écriture. Le moyen d'analyse comprend le code d'identification de zone dans les informations de lieu et transmet lesdites informations.
PCT/JP2016/070484 2015-07-10 2016-07-11 Dispositif d'écriture manuscrite et système de communication vocale/d'écriture manuscrite WO2017010469A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2017528681A JP6973791B2 (ja) 2015-07-10 2016-07-11 手書き装置および手書き通信システム
JP2021175186A JP2022020703A (ja) 2015-07-10 2021-10-27 手書き装置および音声・手書き通信システム

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201562190917P 2015-07-10 2015-07-10
US62/190,917 2015-07-10

Publications (1)

Publication Number Publication Date
WO2017010469A1 true WO2017010469A1 (fr) 2017-01-19

Family

ID=57757418

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2016/070484 WO2017010469A1 (fr) 2015-07-10 2016-07-11 Dispositif d'écriture manuscrite et système de communication vocale/d'écriture manuscrite

Country Status (2)

Country Link
JP (2) JP6973791B2 (fr)
WO (1) WO2017010469A1 (fr)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109426361A (zh) * 2017-09-05 2019-03-05 触信（厦门）智能科技有限公司 手写装置、智能设备及智能手写字条互信系统
JP2021509496A (ja) * 2017-12-28 2021-03-25 ネオラボ コンバージェンス インク 電子機器、電子機器を駆動する方法、及び、データ記録アプリケーションを制御する方法
CN112699977A (zh) * 2020-12-28 2021-04-23 贵州链云科技有限公司 一种轨道车辆手写记录智能采集系统及采集方法
CN113744580A (zh) * 2021-08-06 2021-12-03 廖文雄 点阵笔
WO2022034890A1 (fr) * 2020-08-08 2022-02-17 株式会社Ｉ・Ｐソリューションズ Dispositif d'écriture manuscrite

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005173672A (ja) * 2003-12-08 2005-06-30 Laboratories Of Image Information Science & Technology 用紙、電子ペンおよびプログラム
JP2014529764A (ja) * 2011-12-29 2014-11-13 インテレクチュアル ディスカバリー カンパニー リミテッド ネットワーク上における添削指導サービス提供方法及びこれに用いられるウェブサーバ

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
BRPI0314650B1 (pt) * 2002-09-26 2020-11-10 Kenji Yoshida métodos e dispositivos de reprodução e de inserção/emissão de informação, brinquedo eletrônico portátil, unidade de figura, apoio de mouse, mouse, dispositivo de informação eletrônico, e, mesa gráfica
JP4385169B1 (ja) * 2008-11-25 2009-12-16 健治 吉田 手書き入出力システム、手書き入力シート、情報入力システム、情報入力補助シート
JP5888838B2 (ja) * 2010-04-13 2016-03-22 グリッドマーク株式会社 手書き入力ボードを用いた手書き入力システム、手書き入力ボード、スキャナーペンおよび手書き入力ボードを用いた情報処理システム
JP2012022418A (ja) * 2010-07-12 2012-02-02 Kenji Yoshida ストリームドットを用いた手書き入力シートおよび手書き入出力システム
JP2013016079A (ja) * 2011-07-05 2013-01-24 Dainippon Printing Co Ltd コンピュータ装置及びプログラム、並びにノート

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005173672A (ja) * 2003-12-08 2005-06-30 Laboratories Of Image Information Science & Technology 用紙、電子ペンおよびプログラム
JP2014529764A (ja) * 2011-12-29 2014-11-13 インテレクチュアル ディスカバリー カンパニー リミテッド ネットワーク上における添削指導サービス提供方法及びこれに用いられるウェブサーバ

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
"iPhone Fan Henshubu, au iPhone 6/6 Plus Kanzen Guide", 20 October 2014 (2014-10-20), pages 050 - 051 *
"Muryo Cloud Service ga Marugoto Wakaru Hon", KABUSHIKI KAISHA SHIN'YUSHA, 1 October 2013 (2013-10-01), pages 15 *

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109426361A (zh) * 2017-09-05 2019-03-05 触信（厦门）智能科技有限公司 手写装置、智能设备及智能手写字条互信系统
JP2021509496A (ja) * 2017-12-28 2021-03-25 ネオラボ コンバージェンス インク 電子機器、電子機器を駆動する方法、及び、データ記録アプリケーションを制御する方法
JP7242674B2 (ja) 2017-12-28 2023-03-20 ネオラボ コンバージェンス インク 電子機器、電子機器を駆動する方法、及び、データ記録アプリケーションを制御する方法
US11853544B2 (en) 2017-12-28 2023-12-26 Neolab Convergence Inc. Electronic device, method for driving electronic device, and method for controlling data recording application
WO2022034890A1 (fr) * 2020-08-08 2022-02-17 株式会社Ｉ・Ｐソリューションズ Dispositif d'écriture manuscrite
CN112699977A (zh) * 2020-12-28 2021-04-23 贵州链云科技有限公司 一种轨道车辆手写记录智能采集系统及采集方法
CN113744580A (zh) * 2021-08-06 2021-12-03 廖文雄 点阵笔

Also Published As

Publication number Publication date
JP2022020703A (ja) 2022-02-01
JPWO2017010469A1 (ja) 2018-07-26
JP6973791B2 (ja) 2021-12-01

Similar Documents

Publication Publication Date Title
US10733466B2 (en) Method and device for reproducing content
JP2022020703A (ja) 手書き装置および音声・手書き通信システム
US8358309B2 (en) Animation of audio ink
US6249765B1 (en) System and method for extracting data from audio messages
US20160154482A1 (en) Content Selection in a Pen-Based Computing System
US20160117142A1 (en) Multiple-user collaboration with a smart pen system
US20020079371A1 (en) Multi-moded scanning pen with feedback
WO2020192394A1 (fr) Procédé et dispositif d'affichage de notes, terminal, et support de stockage
JPH06343146A (ja) 再生装置及び再生方法
US9195697B2 (en) Correlation of written notes to digital content
CN110096209A (zh) 书写轨迹显示方法及装置
JP2005010863A (ja) 端末装置、表示システム、表示方法、プログラム、及び記録媒体
EP2632156A2 (fr) Procédé et appareil pour communication bidirectionnelle
EP2947583B1 (fr) Procédé et dispositif de reproduction de contenu
CN112565844A (zh) 视频通信方法、装置和电子设备
US20240146781A1 (en) Systems and methods for managing digital notes for collaboration
Chaudhury et al. Handwriting based interface for communication
JPWO2013005282A1 (ja) 会議用端末装置、及び、会議システム
KR20120044963A (ko) 필기 데이터를 이용한 필기 메시지 재생 장치 및 방법

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16824449

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2017528681

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16824449

Country of ref document: EP

Kind code of ref document: A1