KR20220004924A - Method of synchronizing writing and contents and apparatus therefor - Google Patents

Abstract

Disclosed are a method for synchronizing a handwriting and a content and a device therefor, wherein the method comprises: receiving the handwriting stroke information from an electronic pen or display device; receiving the information for the content being executed from a content execution device; and generating the link data by matching the handwriting stroke information and the information for the content. Therefore, when there are unnecessary contents such as advertisements in the content, the present invention is capable of playbacking the handwriting video without being affected by them.

Description

Method of synchronizing writing and contents and apparatus therefor

The present invention relates to a method for synchronizing writing and contents and an apparatus therefor, and more particularly, to a writing stroke received from an electronic pen and time of contents to synchronize the time of the contents, so that the contents synchronized in time are executed to correspond when the writing stroke is reproduced, or vice versa The present invention relates to a synchronization method and apparatus for reproducing temporally synchronized handwriting strokes when content is executed.

Currently, an electronic pen equipped with a writing means has been released and used. At this time, when the electronic pen takes notes on a note on which a predetermined code is printed, the electronic pen acquires the note code according to the writing trajectory, transmits it to the smartphone through predetermined communication, and the smartphone responds to the received code. The writing trajectory is restored and displayed accordingly.

1 is a view for explaining an example of restoring handwriting in an application when an electronic pen takes notes.

Referring to FIG. 1 , in the dedicated paper, note or textbook 120 , the electronic pen 110 has a predetermined code (eg, dot code) for checking the position on the page of the dedicated paper, note, or textbook 120 . or line segment code) is printed. When taking notes, the electronic pen 110 acquires a code according to the writing trajectory 101 and transmits it to the device 130 in which a dedicated application is installed through predetermined communication. After the dedicated application obtains information about the code from the electronic pen 110 , the location of the code is identified and the writing trace 102 is restored on the screen of the device 130 .

In addition, it is possible to create and store a reproduction file to display the handwriting traces made for a certain period in chronological order. In this case, the handwritten stroke written by the electronic pen is reproduced like a moving picture.

In some cases, taking notes with an electronic pen while watching content such as a video through an image device. In the case of replaying the video, the handwriting stroke image may also be reproduced, but in this case, the user has no choice but to manually reproduce the video, so that the two images are not accurately matched.

The problem to be solved by the present invention is a handwriting and content synchronization method capable of reproducing a synced handwritten image when content is executed, and reproducing a handwritten image without being affected by unnecessary content such as advertisements in the content, and a method for synchronizing the same to provide the device.

According to one embodiment of the present invention for solving the above technical problem, a method for synchronizing writing and content includes receiving handwriting stroke information from an electronic pen or a display device; Receiving information about the content being executed from the content execution device; and generating link data by matching the handwritten stroke information and the information on the content.

The generating link data by matching the handwriting stroke information and the content information may include synchronizing the handwriting stroke time information included in the handwriting stroke information and the content time information included in the content information. ; and generating link data based on the handwritten stroke information, the content information, and the synchronized time information.

The handwriting and content synchronization method includes the steps of extracting time information on the last received content when the time information on the content is not received; resynchronizing the handwriting stroke time information included in the handwriting stroke information and the time information on the extracted last received content; and generating link data based on the handwritten stroke information, the content information, and the resynchronized time information.

The handwriting and content synchronization method includes: determining execution direction information and execution speed information of the content being executed based on a plurality of pieces of handwriting stroke time information and time information on the content; and adding the determined execution direction information and execution speed information of the content being executed to the link data.

The handwriting and content synchronization method may further include, when the content is replayed, reproducing the handwriting stroke based on the link data.

In the handwriting and content synchronization method, when the electronic pen is touched at a predetermined position in the printed matter in which the handwriting stroke exists, or when a predetermined input is received at a location adjacent to the handwriting stroke displayed on the display device, the The method may further include calling and playing the content.

According to another embodiment of the present invention for solving the above technical problem, a method for synchronizing writing and content includes: receiving information about content being executed from a content execution device; extracting time information when receiving information about the content being executed; generating link data by matching the extracted time information and information on the content; receiving handwriting stroke information from the electronic pen or the display device when connected to the electronic pen or the display device; and updating the link data by matching the handwritten stroke information, the extracted time information, and the content information.

According to another embodiment of the present invention for solving the above technical problem, a method for synchronizing writing and content includes receiving handwriting stroke information from an electronic pen or a display device; Receiving information about the content being executed from the content execution device; extracting time information when receiving information about the content being executed; and generating link data by matching the handwritten stroke information, the extracted time information, and the information on the content.

According to another embodiment of the present invention for solving the above technical problem, a writing and content synchronization device receives handwriting stroke information from an electronic pen or a display device, and a receiver configured to receive information about content being executed from the content execution device ; and a synchronizer for generating link data by matching the handwritten stroke information and the information on the content.

According to still another embodiment of the present invention for solving the above technical problem, an apparatus for synchronizing writing and content includes: a receiver configured to receive information about content being executed from a content execution device; a control unit for extracting time information when receiving information about the content being executed; and a synchronization unit that generates link data by matching the extracted time information and the information on the content, wherein when connected to an electronic pen or a display device, the receiving unit receives handwriting stroke information from the electronic pen or the display device , and the synchronization unit updates the link data by matching the handwritten stroke information, the extracted time information, and the information on the content.

According to another embodiment of the present invention for solving the above technical problem, a writing and content synchronization device receives handwriting stroke information from an electronic pen or a display device, and a receiver configured to receive information about content being executed from the content execution device ; a control unit for extracting time information when receiving information about the content being executed; and a synchronization unit generating link data by matching the handwriting stroke information, the extracted time information, and the information on the content.

1 is a view for explaining an example of restoring handwriting in an application when an electronic pen takes notes.
2 is a diagram illustrating a flowchart of a method for synchronizing notes and contents according to an embodiment of the present invention.
3 is a diagram illustrating a flowchart of a method for synchronizing notes and contents according to another embodiment of the present invention.
4 is a flowchart illustrating a method for synchronizing writing and content according to another embodiment of the present invention.
5 is a flowchart illustrating a method for synchronizing writing and content according to another embodiment of the present invention.
6 is a diagram illustrating a block diagram of an apparatus for synchronizing notes and contents according to an embodiment of the present invention.

Advantages and features of the present invention, and methods for achieving them, will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments described herein and may be embodied in other forms. Rather, the embodiments introduced herein are provided so that the disclosed subject matter may be thorough and complete, and that the spirit of the present invention may be sufficiently conveyed to those skilled in the art.

The terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly dictates otherwise. In the present application, terms such as “comprise” or “have” are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, but one or more other features It is to be understood that it does not preclude the possibility of the presence or addition of numbers, steps, operations, components, parts, or combinations thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms such as those defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning in the context of the related art, and should not be interpreted in an ideal or excessively formal meaning unless explicitly defined in the present application. does not

A preferred embodiment of the present invention will be described in detail with reference to the accompanying drawings.

2 is a diagram illustrating a flowchart of a method for synchronizing notes and contents according to an embodiment of the present invention.

Referring to FIG. 2 , in step 210 , an apparatus (hereinafter, referred to as a 'synchronization apparatus') performing a method for synchronizing writing and content receives handwriting stroke information from an electronic pen.

The electronic pen is provided with a writing means, and when the electronic pen writes on a printed matter (for example, paper) on which a code is printed using the writing means, the electronic pen uses an optical means to write the printed matter according to the trajectory of the writing stroke. Read the code information in The code information may include handwritten stroke position information, information on printed matter, and page information of the handwritten print. The electronic pen generates handwriting stroke information including code information and handwriting stroke time information and transmits the generated handwriting stroke information to the synchronization device.

Here, a stroke means one stroke written on a printed matter using an electronic pen. The electronic pen recognizes that writing is performed when the pen tip comes into contact with the printed matter by a predetermined pressure or more through a pressure sensor. Stroke refers to handwriting made during a period in which the pen tip detaches from the printed matter after the pen tip touches it. In the present invention, the stroke time information may be the contact time of the pen tip of the electronic pen based on the stroke or the time of departure from the printed matter, preferably the time at which the pen tip of the electronic pen leaves the printed matter. That is, the electronic pen extracts stroke time information from the electronic pen and transmits the extracted stroke time information to the synchronization device when the pen tip is raised from the printed matter.

As another embodiment of the electronic pen, the electronic pen may generate handwriting information by detecting movement of a laser speckle pattern according to a writing stroke on a surface or paper without a printed code.

In another embodiment, handwriting information may be received from the display device. When a pen such as a stylus writes on the display device, the display device generates writing information.

In step 220, the synchronization device receives information about the content being executed from the content execution device.

The content being executed may be a video, video, or audio being played. An example of the content being executed may be a broadcast program, an application running on the content execution device (eg, MS Word or PowerPoint, etc.) or a service through the Internet (eg, SNS, chat service or video conferencing service, etc.) have. The content may be in any form including online human behavior. The content execution device may be a television, a personal computer, or a smart phone.

When content is being reproduced through the display of the content execution device, the synchronization device receives information about the content being executed or reproduced from the content execution device. The information on the content may include identification information (eg, ID) of the content, provider information, storage information or hyperlink information (eg, URL), and time information on the content. The time information on the content includes total execution (playback) time information and current execution time information of the content.

In another embodiment of the present invention, the synchronization device may receive time information on the content being executed from a communication device connected to the content execution device. The communication device may receive time information on the content being executed from the content execution device, and the synchronization device may again receive time information on the content received by the communication device.

The content execution device and the communication device may be connected via wire or wireless, and the communication device and the synchronization device may also be connected via wire or wireless. Preferably, the content execution device and the communication device are connected through a wireless method such as Bluetooth, and the communication device and the synchronization device are connected through a USB communication method.

In step 230, the synchronization device generates link data by matching the received handwriting stroke information and content information. The synchronization device synchronizes the received handwriting stroke time information and the time information for the content, and then, the synchronization device generates link data based on the received handwriting stroke information, the received content information and the synchronized time information do.

The synchronization device or the device in which the synchronization device is embedded may display the handwriting trace on the screen based on the handwriting information received from the electronic pen or the display device. In addition, after the writing is completed, the writing trajectory can be reproduced in chronological order by saving the writing in a predetermined playback file or document file format and selecting the user.

In one embodiment of the present invention, when the content execution device replays the content, the handwriting stroke may be reproduced based on the link data in the synchronization device, the device connected to the synchronization device, or the device for reproducing the handwriting trace. Content and handwriting strokes are played according to the synchronized time. In this case, the user can confirm that the handwritten content is reproduced according to the synchronized time while viewing the content.

Conversely, when a handwriting stroke is reproduced in a synchronization device, a device connected to the synchronization device, or a device that reproduces a handwriting trace, the content may be reproduced in the content execution device according to time information on the synchronized content based on link data.

In an embodiment of the present invention, when a predetermined icon is selected on a display screen written by an electronic pen or written by an electronic pen, the corresponding content is called according to link data existing on a page in which the selected icon is present. , it can also be played back. In this case, the handwriting stroke synchronized with the content may be reproduced.

In one embodiment of the present invention, when there is an input or touch within a predetermined distance from a handwriting stroke on a printed matter handwritten by an electronic pen or a display screen handwritten by a stylus, corresponding content is called according to link data for the handwritten stroke After that, you can run or play it. In this case, the handwriting stroke matched with the content may also be reproduced.

For example, when a predetermined time elapses after a touch of the electronic pen or stylus has passed since the last stroke, when the touch time is within a predetermined time, and when a stroke with respect to a touch is less than or equal to a predetermined length, if at least two are satisfied, the synchronization device is recognized as a touch. If not satisfied, the content is not called by recognizing it as handwriting.

In another embodiment of the present invention, when a pre-appointed handwriting (for example, replay handwriting) is performed on a print handwritten by an electronic pen or a display screen handwritten by a stylus, according to link data for a handwritten page After calling the corresponding content, it can be played or executed. In this case, the handwriting stroke synchronized with the content may be reproduced.

In another embodiment of the present invention, when the display device connected to the electronic pen or stylus performs a predetermined gesture, the corresponding content may be called according to link data existing in the written page, and then the corresponding content may be played or executed. . In this case, the handwriting stroke synchronized with the content may be reproduced.

In the case of reproducing content, particularly, in the case of reproducing video or audio, the video may be rewinded or reproduced at double speed.

When reproducing content, time information on content is sequentially received and comparing them, it is possible to recognize whether content reproduction is in the forward or reverse direction.

In addition, when time differences for a plurality of contents are taken into account based on a time difference between a plurality of handwriting strokes, information on the speed of the contents can be calculated.

The synchronization device may add the recognized content execution direction information and speed information to the link data as metadata. Alternatively, link data may be updated by creating these fields.

According to another embodiment of the present invention, the content execution direction information and the speed information may be received from the content execution device. In this case, the synchronization device may add the execution direction information and the speed information of the received content to the link data as metadata.

In step 240, when the synchronization device does not receive time information on the content, it extracts the time information on the last received content.

Recently, there are many cases in which content providers insert advertisement images at the beginning or in the middle of content in order to increase revenue. In this case, it is common that the advertisement image has nothing to do with the content of the content. Accordingly, there is a problem in that the contents are not accurately matched because the handwriting made during the reproduction of the advertisement image is not related to the advertisement image.

When there is an advertisement in the content, the content execution device does not transmit information about the content, in particular, time information about the content. In this case, the synchronization device extracts time information about the last content that was received.

In another embodiment, the synchronization device may extract time information about the next received content.

In step 250, the synchronization device resynchronizes the received handwriting stroke time information and the extracted time information on the content. After correcting the time of the handwriting stroke received during the reproduction of the advertisement video to the time before or after the reproduction of the advertisement, it is resynchronized by matching the corresponding time information of the content being reproduced.

Thereafter, link data is generated or updated based on the handwritten stroke information, the content information, and the resynchronized time information.

3 is a diagram illustrating a flowchart of a method for synchronizing notes and contents according to another embodiment of the present invention.

Unlike FIG. 2, FIG. 3 relates to a content execution device and a synchronization device that are first connected to generate link data, and then, a synchronization device and an electronic pen or display device are connected to update link data based on handwriting information. .

Referring to FIG. 3 , in step 310, the synchronization device receives information about the content being executed from the content execution device.

The content being executed may be a video, video, or audio being played. An example of the content being executed may be a broadcast program, an application running on the content execution device (eg, MS Word or PowerPoint, etc.) or a service through the Internet (eg, SNS, chat service or video conferencing service, etc.) have. The content may be in any form including online human behavior. The content execution device may be a television, a personal computer, or a smart phone.

When content is being reproduced through the display of the content execution device, the synchronization device receives information about the content being executed or reproduced from the content execution device. The information on the content may include identification information (eg, ID) of the content, provider information, storage information or hyperlink information (eg, URL), and time information on the content. The time information on the content includes total execution (playback) time information and current execution time information of the content.

In another embodiment of the present invention, the synchronization device may receive time information on the content being executed from a communication device connected to the content execution device. The communication device may receive time information on the content being executed from the content execution device, and the synchronization device may again receive time information on the content received by the communication device.

The content execution device and the communication device may be connected via wire or wireless, and the communication device and the synchronization device may also be connected via wire or wireless. Preferably, the content execution device and the communication device are connected through a wireless method such as Bluetooth, and the communication device and the synchronization device are connected through a USB communication method.

In step 320, when the synchronization device receives information about the content being executed, it extracts time information in the synchronization device. The synchronization device extracts time information from an internal clock in real time when receiving information about content.

In step 330, the synchronization device generates link data by matching the information on the received content and the extracted time information. The synchronization device synchronizes the extracted time information and the time information about the content, and then, the synchronization device generates link data based on the received time information, the information about the received content, and the synchronized time information.

In operation 340, when the synchronization device is connected to the electronic pen, handwriting stroke information is received from the electronic pen. Here, the electronic pen should generate and store handwriting stroke information in a state that is not connected to the synchronization device.

The electronic pen is provided with a writing means, and when the electronic pen writes on a printed matter (for example, paper) on which a code is printed using the writing means, the electronic pen uses an optical means to write the printed matter according to the trajectory of the writing stroke. Read the code information in The code information may include handwritten stroke position information, information on printed matter, and page information of the handwritten print. The electronic pen generates code information and handwriting stroke time information and transmits them to the synchronization device.

Here, a stroke means one stroke written on a printed matter using an electronic pen. The electronic pen recognizes that writing is performed when the pen tip comes into contact with the printed matter by a predetermined pressure or more through a pressure sensor. Stroke refers to handwriting made during a period in which the pen tip detaches from the printed matter after the pen tip touches it. In the present invention, the stroke time information may be the contact time of the pen tip of the electronic pen based on the stroke or the time that the pen tip of the electronic pen departs from the printed matter, preferably the time at which the pen tip of the electronic pen leaves the printed matter. That is, the electronic pen extracts time information from the electronic pen and transmits it to the synchronization device when the pen tip is raised from the printed matter.

As another embodiment of the electronic pen, the electronic pen may generate handwriting information by detecting movement of a laser speckle pattern according to a writing stroke on a surface or paper without a printed code.

In another embodiment, handwriting information may be received from the display device. When a pen such as a stylus writes on the display device, the display device generates writing information.

In step 350, the synchronization device updates the link data by matching the handwritten stroke information, the extracted time information, and the content information.

The synchronization device resynchronizes the extracted time information with the handwriting stroke time information included in the handwriting stroke information. Since the extracted time information and the time information about the content are already synchronized, the handwriting stroke time information and the extracted internal time information need only be synchronized. Then, the synchronization device updates the link data by matching the handwritten stroke information, the extracted time information, and the information on the content.

The synchronization device or the device in which the synchronization device is embedded may display the handwriting trace on the screen based on the handwriting information received from the electronic pen or the display device. In addition, after the writing is completed, the writing trajectory can be reproduced in chronological order by saving the writing in a predetermined playback file or document file format and selecting the user.

In one embodiment of the present invention, when the content execution device replays the content, the handwriting stroke may be reproduced based on the link data in the synchronization device, the device connected to the synchronization device, or the device for reproducing the handwriting trace. Content and handwriting strokes are played according to the synchronized time. In this case, the user can confirm that the handwritten content is reproduced according to the synchronized time while viewing the content.

Conversely, when a handwriting stroke is reproduced in a synchronization device, a device connected to the synchronization device, or a device that reproduces a handwriting trace, the content may be reproduced in the content execution device according to time information on the synchronized content based on link data.

In an embodiment of the present invention, when a predetermined icon is selected on a display screen written by an electronic pen or written by an electronic pen, the corresponding content is called according to link data existing on a page in which the selected icon is present. , it can also be played back. In this case, the handwriting stroke synchronized with the content may be reproduced.

In one embodiment of the present invention, when there is an input or touch within a predetermined distance from a handwriting stroke on a printed matter handwritten by an electronic pen or a display screen handwritten by a stylus, corresponding content is called according to link data for the handwritten stroke After that, you can run or play it. In this case, the handwriting stroke matched with the content may also be reproduced.

For example, when a predetermined time elapses after a touch of the electronic pen or stylus has passed since the last stroke, when the touch time is within a predetermined time, and when a stroke with respect to a touch is less than or equal to a predetermined length, if at least two are satisfied, the synchronization device is recognized as a touch. If not satisfied, the content is not called by recognizing it as handwriting.

In another embodiment of the present invention, when a pre-appointed handwriting (for example, replay handwriting) is performed on a print handwritten by an electronic pen or a display screen handwritten by a stylus, according to link data for a handwritten page After calling the corresponding content, it can be played or executed. In this case, the handwriting stroke synchronized with the content may be reproduced.

In another embodiment of the present invention, when the display device connected to the electronic pen or stylus performs a predetermined gesture, the corresponding content may be called according to link data existing in the written page, and then the corresponding content may be played or executed. . In this case, the handwriting stroke synchronized with the content may be reproduced.

In the case of reproducing content, particularly, in the case of reproducing video or audio, the video may be rewinded or reproduced at double speed.

When reproducing content, time information on content is sequentially received and comparing them, it is possible to recognize whether content reproduction is in the forward or reverse direction.

In addition, when time differences for a plurality of contents are taken into account based on a time difference between a plurality of handwriting strokes, information on the speed of the contents can be calculated.

The synchronization device may add the recognized content execution direction information and speed information to the link data as metadata. Alternatively, link data may be updated by creating these fields.

According to another embodiment of the present invention, the content execution direction information and the speed information may be received from the content execution device. In this case, the synchronization device may add the execution direction information and the speed information of the received content to the link data as metadata.

In step 360, when the synchronization device does not receive time information on the content, it extracts the time information on the last received content.

Recently, there are many cases in which content providers insert advertisement images at the beginning or in the middle of content in order to increase revenue. In this case, it is common that the advertisement image has nothing to do with the content of the content. Accordingly, there is a problem in that the contents are not accurately matched because the handwriting made during the reproduction of the advertisement image is not related to the advertisement image.

When there is an advertisement in the content, the content execution device does not transmit information about the content, in particular, time information about the content. In this case, the synchronization device extracts time information about the last content that was received.

In another embodiment, the synchronization device may extract time information about the next received content.

In step 370, the synchronization device resynchronizes the extracted time information and the extracted time information for the last received content. After correcting the internal time extracted while playing the advertisement video to the time before or after the advertisement playing, it is resynchronized by matching it with the time information of the corresponding running content.

Thereafter, link data is generated or updated based on the extracted time information, content information, and resynchronized time information.

4 is a flowchart illustrating a method for synchronizing writing and content according to another embodiment of the present invention.

4 is a case in which the electronic pen or the display device does not transmit time information, but only transmits location information, unlike FIGS. 2 and 3 .

Referring to FIG. 4 , in step 410, handwriting stroke information is received from the electronic pen.

The electronic pen is provided with a writing means, and when the electronic pen writes on a printed matter (for example, paper) on which a code is printed using the writing means, the electronic pen uses an optical means to write the printed matter according to the trajectory of the writing stroke. Read the code information in The code information may include handwritten stroke position information, information on printed matter, and page information of the handwritten print. The electronic pen generates code information as handwriting stroke information and transmits it to the synchronization device.

Here, a stroke means one stroke written on a printed matter using an electronic pen. The electronic pen recognizes that writing is performed when the pen tip comes into contact with the printed matter by a predetermined pressure or more through a pressure sensor. Stroke refers to handwriting made during a period in which the pen tip detaches from the printed matter after the pen tip touches it.

As another embodiment of the electronic pen, the electronic pen may generate handwriting information by detecting movement of a laser speckle pattern according to a writing stroke on a surface or paper without a printed code.

In another embodiment, handwriting information may be received from the display device. When a pen such as a stylus writes on the display device, the display device generates writing information.

In step 420, the synchronization device receives information about the content being executed from the content execution device.

The content being executed may be a video, video, or audio being played. Examples of the content being executed may be a broadcast program, an application running on the content execution device (eg, MS Word or PowerPoint, etc.) or a service through the Internet (eg, SNS, chat service or video conferencing service, etc.) have. The content may be in any form including online human behavior. The content execution device may be a television, a personal computer, or a smart phone.

When content is being reproduced through the display of the content execution device, the synchronization device receives information about the content being executed or reproduced from the content execution device. The information about the content may include identification information (eg, ID) of the content, provider information, storage information or hyperlink information (eg, URL), and time information about the content. The time information on the content includes total execution (playback) time information and current execution time information of the content.

According to another embodiment of the present invention, the information on the content may include only location information within the content, not including the execution time information.

In this case, the information on the content being executed may include identifier information, location information, and storage path information of the content being executed. Identifier information of content includes file name, file type, account, ID, login information, creation date, creator, modification date, modified person, file size, tag, owner or copyright holder, for example, and location information is a video or broadcast program In the case of an application, it may mean time information, in the case of an application, it may be a cursor or a point position in the application when receiving, and the storage path information indicates storage location information in the case of an image display device, and is stored in an external device. may be hyperlink information.

In another embodiment of the present invention, the synchronization device may receive time information on the content being executed from a communication device connected to the content execution device. The communication device may receive time information on the content being executed from the content execution device, and the synchronization device may again receive time information on the content received by the communication device.

The content execution device and the communication device may be connected via wire or wireless, and the communication device and the synchronization device may also be connected via wire or wireless. Preferably, the content execution device and the communication device are connected through a wireless method such as Bluetooth, and the communication device and the synchronization device are connected through a USB communication method.

In step 430, when the synchronization device receives information about the content being executed, it extracts time information in the synchronization device. The synchronization device extracts time information from an internal clock in real time when receiving information about content. Since the electronic pen or the display device does not transmit the handwriting stroke time information, when the handwriting stroke information is first received, the handwriting stroke time information may be matched according to the internal time information. That is, the internal time information and the handwriting stroke time information are the same.

In step 440, the synchronization device generates link data by matching the information on the received content and the extracted time information. The synchronization device synchronizes the extracted time information and time information for content. Then, the synchronization device generates link data based on the received time information, the information on the received content, and the synchronized time information.

In step 450, the synchronization device updates the link data by matching the handwritten stroke information, the extracted time information, and the content information.

The synchronization device resynchronizes the extracted time information with the handwriting stroke time information included in the handwriting stroke information. Since the extracted time information and the time information about the content are already synchronized, the handwriting stroke time information and the extracted internal time information need only be synchronized. Then, the synchronization device updates the link data by matching the handwritten stroke information, the extracted time information, and the information on the content.

The synchronization device or the device in which the synchronization device is embedded may display the handwriting trace on the screen based on the handwriting information received from the electronic pen or the display device. In addition, after the writing is completed, the writing trajectory can be reproduced in chronological order by saving the writing in a predetermined playback file or document file format and selecting the user.

In one embodiment of the present invention, when the content execution device replays the content, the handwriting stroke may be reproduced based on the link data in the synchronization device, the device connected to the synchronization device, or the device for reproducing the handwriting trace. Content and handwriting strokes are played according to the synchronized time. In this case, the user can confirm that the handwritten content is reproduced according to the synchronized time while viewing the content.

Conversely, when a handwriting stroke is reproduced in a synchronization device, a device connected to the synchronization device, or a device that reproduces a handwriting trace, the content may be reproduced in the content execution device according to time information on the synchronized content based on link data.

In an embodiment of the present invention, when a predetermined icon is selected on a display screen written by an electronic pen or written by an electronic pen, the corresponding content is called according to link data existing on a page in which the selected icon is present. , it can also be played back. In this case, the handwriting stroke synchronized with the content may be reproduced.

In one embodiment of the present invention, when there is an input or touch within a predetermined distance from a handwriting stroke on a printed matter handwritten by an electronic pen or a display screen handwritten by a stylus, corresponding content is called according to link data for the handwritten stroke After that, you can run or play it. In this case, the handwriting stroke matched with the content may also be reproduced.

For example, when a predetermined time elapses after a touch of the electronic pen or stylus has passed since the last stroke, when the touch time is within a predetermined time, and when a stroke with respect to a touch is less than or equal to a predetermined length, if at least two are satisfied, the synchronization device is recognized as a touch. If not satisfied, the content is not called by recognizing it as handwriting.

In another embodiment of the present invention, when a pre-appointed handwriting (for example, replay handwriting) is performed on a print handwritten by an electronic pen or a display screen handwritten by a stylus, according to link data for a handwritten page After calling the corresponding content, it can be played or executed. In this case, the handwriting stroke synchronized with the content may be reproduced.

In another embodiment of the present invention, when the display device connected to the electronic pen or stylus performs a predetermined gesture, the corresponding content may be called according to link data existing in the written page, and then the corresponding content may be played or executed. . In this case, the handwriting stroke synchronized with the content may be reproduced.

In the case of reproducing content, particularly, in the case of reproducing video or audio, the video may be rewinded or reproduced at double speed.

When reproducing content, time information on content is sequentially received and comparing them, it is possible to recognize whether content reproduction is in the forward or reverse direction.

In addition, when time differences for a plurality of contents are taken into account based on a time difference between a plurality of handwriting strokes, information on the speed of the contents can be calculated.

The synchronization device may add the recognized content execution direction information and speed information to the link data as metadata. Alternatively, link data may be updated by creating these fields.

According to another embodiment of the present invention, the content execution direction information and the speed information may be received from the content execution device. In this case, the synchronization device may add the execution direction information and the speed information of the received content to the link data as metadata.

In step 460, when the synchronization device does not receive the time information on the content, it extracts the time information on the last received content.

Recently, there are many cases in which content providers insert advertisement images at the beginning or in the middle of content in order to increase revenue. In this case, it is common that the advertisement image has nothing to do with the content of the content. Accordingly, there is a problem in that the contents are not accurately matched because the handwriting made during the reproduction of the advertisement image is not related to the advertisement image.

When there is an advertisement in the content, the content execution device does not transmit information about the content, in particular, time information about the content. In this case, the synchronization device extracts time information about the last content that was received.

In another embodiment, the synchronization device may extract time information about the next received content.

In step 470, the synchronization device resynchronizes the extracted time information and the extracted time information for the last received content. After correcting the internal time extracted while playing the advertisement video to the time before or after the advertisement playing, it is resynchronized by matching it with the time information of the corresponding running content.

Thereafter, link data is generated or updated based on the extracted time information, content information, and resynchronized time information.

5 is a flowchart illustrating a method for synchronizing writing and content according to another embodiment of the present invention.

Referring to FIG. 5 , in step 510 , an apparatus (hereinafter, referred to as a 'synchronization apparatus') performing a method for synchronizing writing and content receives handwriting stroke information from an electronic pen.

The electronic pen is provided with a writing means, and when the electronic pen writes on a printed matter (for example, paper) on which a code is printed using the writing means, the electronic pen uses an optical means to write the printed matter according to the trajectory of the writing stroke. Read the code information in The code information may include handwritten stroke position information, information on printed matter, and page information of the handwritten print. The electronic pen generates handwriting stroke information including code information and handwriting stroke time information and transmits the generated handwriting stroke information to the synchronization device.

Here, a stroke means one stroke written on a printed matter using an electronic pen. The electronic pen recognizes that writing is performed when the pen tip comes into contact with the printed matter by a predetermined pressure or more through a pressure sensor. Stroke refers to handwriting made during a period in which the pen tip detaches from the printed matter after the pen tip touches it. In the present invention, the stroke time information may be the contact time of the pen tip of the electronic pen based on the stroke or the time that the pen tip of the electronic pen departs from the printed matter, preferably the time at which the pen tip of the electronic pen leaves the printed matter. That is, the electronic pen extracts time information from the electronic pen and transmits it to the synchronization device when the pen tip is raised from the printed matter.

As another embodiment of the electronic pen, the electronic pen may generate handwriting information by detecting movement of a laser speckle pattern according to a writing stroke on a surface or paper without a printed code.

In another embodiment, handwriting information may be received from the display device. When a pen such as a stylus writes on the display device, the display device generates writing information.

In step 520, the synchronization device receives information about the content being executed from the content execution device.

The content being executed may be a video, video, or audio being played. Examples of the content being executed may be a broadcast program, an application running on the content execution device (eg, MS Word or PowerPoint, etc.) or a service through the Internet (eg, SNS, chat service or video conferencing service, etc.) have. The content may be in any form including online human behavior. The content execution device may be a television, a personal computer, or a smart phone.

When content is being reproduced through the display of the content execution device, the synchronization device receives information about the content being executed or reproduced from the content execution device. The information on the content may include identification information (eg, ID) of the content, provider information, storage information or hyperlink information (eg, URL), and time information on the content. The time information on the content includes total execution (playback) time information and current execution time information of the content.

According to another embodiment of the present invention, the synchronization device may receive time information on the content being executed from a communication device connected to the content playback device. The communication device may receive time information on the content being executed from the content execution device, and the synchronization device may again receive time information on the content received by the communication device.

The content execution device and the communication device may be connected via wire or wireless, and the communication device and the synchronization device may also be connected via wire or wireless. Preferably, the content execution device and the communication device are connected through a wireless method such as Bluetooth, and the communication device and the synchronization device are connected through a USB communication method.

In step 530, the synchronization device extracts handwriting stroke time information from the received handwriting stroke information, and moves the extracted handwriting stroke time information back by a predetermined time to correct it. That is, the synchronization device corrects the handwriting stroke information.

In general, after viewing the content, the user takes notes about the content. There is a problem in that when the content is re-executed later, it is not accurately matched with the written content. Accordingly, in order to match them, the handwritten stroke information is corrected earlier by the time set by the user. When the handwriting stroke time information is corrected, the contents of the two are matched to be reproduced or executed.

In operation 540, the synchronization device generates link data by matching the corrected handwriting stroke information and the content information. The synchronization device synchronizes the modified handwriting stroke time information and the time information for the content. Thereafter, the synchronization device generates link data based on the corrected handwriting stroke information, the information on the received content, and the synchronized time information.

The synchronization device or the device in which the synchronization device is embedded may display the handwriting trace on the screen based on the handwriting information received from the electronic pen or the display device. In addition, after the writing is completed, the writing trajectory can be reproduced in chronological order by saving the writing in a predetermined playback file or document file format and selecting the user.

In one embodiment of the present invention, when the content execution device replays the content, the handwriting stroke may be reproduced based on the link data in the synchronization device, the device connected to the synchronization device, or the device for reproducing the handwriting trace. Content and handwriting strokes are played according to the synchronized time. In this case, the user can confirm that the handwritten content is reproduced according to the synchronized time while viewing the content.

Conversely, when a handwriting stroke is reproduced in a synchronization device, a device connected to the synchronization device, or a device that reproduces a handwriting trace, the content may be reproduced in the content execution device according to time information on the synchronized content based on link data.

In an embodiment of the present invention, when a predetermined icon is selected on a display screen written by an electronic pen or written by an electronic pen, the corresponding content is called according to link data existing on a page in which the selected icon is present. , it can also be played back. In this case, the handwriting stroke synchronized with the content may be reproduced.

In one embodiment of the present invention, when there is an input or touch within a predetermined distance from a handwriting stroke on a printed matter handwritten by an electronic pen or a display screen handwritten by a stylus, corresponding content is called according to link data for the handwritten stroke After that, you can run or play it. In this case, the handwriting stroke matched with the content may also be reproduced.

For example, when a predetermined time elapses after a touch of the electronic pen or stylus has passed since the last stroke, when the touch time is within a predetermined time, and when a stroke with respect to a touch is less than or equal to a predetermined length, if at least two are satisfied, the synchronization device is recognized as a touch. If not satisfied, the content is not called by recognizing it as handwriting.

In another embodiment of the present invention, when a pre-appointed handwriting (for example, replay handwriting) is performed on a print handwritten by an electronic pen or a display screen handwritten by a stylus, according to link data for a handwritten page After calling the corresponding content, it can be played or executed. In this case, the handwriting stroke synchronized with the content may be reproduced.

In another embodiment of the present invention, when the display device connected to the electronic pen or stylus performs a predetermined gesture, the corresponding content may be called according to link data existing in the written page, and then the corresponding content may be played or executed. . In this case, the handwriting stroke synchronized with the content may be reproduced.

In the case of reproducing content, particularly, in the case of reproducing video or audio, the video may be rewinded or reproduced at double speed.

When reproducing content, time information on content is sequentially received and comparing them, it is possible to recognize whether content reproduction is in the forward or reverse direction.

In addition, when time differences for a plurality of contents are taken into account based on a time difference between a plurality of handwriting strokes, information on the speed of the contents can be calculated.

The synchronization device may add the recognized content execution direction information and speed information to the link data as metadata. Alternatively, link data may be updated by creating these fields.

According to another embodiment of the present invention, the content execution direction information and the speed information may be received from the content execution device. In this case, the synchronization device may add the execution direction information and the speed information of the received content to the link data as metadata.

In step 550, when the synchronization device does not receive time information on content, it extracts time information on the last received content.

Recently, there are many cases in which content providers insert advertisement images at the beginning or in the middle of content in order to increase revenue. In this case, it is common that the advertisement image has nothing to do with the content of the content. Accordingly, there is a problem in that the contents are not accurately matched because the handwriting made during the reproduction of the advertisement image is not related to the advertisement image.

When there is an advertisement in the content, the content execution device does not transmit information about the content, in particular, time information about the content. In this case, the synchronization device extracts time information about the last content that was received.

In another embodiment, the synchronization device may extract time information about the next received content.

In step 560, the synchronization device resynchronizes the received handwriting stroke time information and the extracted time information for the E content. After correcting the time of the handwriting stroke received during the reproduction of the advertisement video to the time before or after the reproduction of the advertisement, it is resynchronized by matching the corresponding time information of the content being reproduced.

Thereafter, link data is generated or updated based on the handwritten stroke information, the content information, and the resynchronized time information.

6 is a diagram illustrating a block diagram of an apparatus for synchronizing notes and contents according to an embodiment of the present invention.

Referring to FIG. 6 , the synchronization device 610 includes a reception unit 611 , a synchronization unit 612 , a control unit 613 , and a playback unit 614 , and the synchronization device 610 includes an electronic pen 620 and content execution. The device 630 and the communication device 640 may be connected, and the communication device 640 may be plugged into an image reproducing device or an image display device 630 .

The receiver 611 receives handwriting stroke information from the electronic pen 620 .

The electronic pen 620 is provided with a writing means, and when the electronic pen 620 writes on a printed matter (eg, paper) on which a code is printed using the writing means, the electronic pen 620 is an optical means. is used to read the code information in the printed matter according to the trajectory of the handwriting stroke. The code information may include handwritten stroke position information, information on printed matter, and page information of the handwritten print. The electronic pen 620 generates handwriting stroke information including code information and handwriting stroke time information and transmits the generated handwriting stroke information to the synchronization device.

Here, the stroke means one stroke written on the printed matter using the electronic pen 620 . The electronic pen 620 recognizes that writing is performed when the pen tip comes into contact with the printed matter by a predetermined pressure or more through the pressure sensor. Stroke refers to handwriting made during a period in which the pen tip detaches from the printed matter after the pen tip touches it. In the present invention, the stroke time information may be the contact time of the pen tip of the electronic pen 620 based on the stroke or the time at which the pen tip of the electronic pen 620 departs from the print. Preferably, the pen tip of the electronic pen 620 leaves the print. it's time to That is, the electronic pen 620 extracts time information from the electronic pen and transmits it to the synchronization device when the pen tip is up from the printed matter.

As another embodiment of the electronic pen 620 , the electronic pen may generate handwriting information by detecting movement of a laser speckle pattern according to a writing stroke on a surface or paper without a printed code.

In another embodiment, handwriting information may be received from a display device (not shown). When a pen such as a stylus writes on the display device, the display device generates writing information. In FIG. 6 , it is assumed that the display device is included in the electronic pen 620 .

The receiver 611 receives information about the content being executed from the content execution device 630 .

The content being executed may be a video, video, or audio being played. Examples of the content being executed include a broadcast program, an application running on the content execution device 630 (eg, MS Word or PowerPoint, etc.) or a service through the Internet (eg, SNS, chat service or video conference service, etc.) ) can be The content may be in any form including online human behavior. The content execution device 630 may be a television, a personal computer, or a smart phone.

When content is being reproduced through the display of the content execution device 630 , the synchronization device 610 receives information about the content being executed or reproduced from the content execution device 630 . The information on the content may include identification information (eg, ID) of the content, provider information, storage information or hyperlink information (eg, URL), and time information on the content. The time information on the content includes total execution (playback) time information and current execution time information of the content.

As another embodiment of the present invention, the receiver 611 may receive time information about the content being executed from the communication device 640 connected to the content execution device 630 . The communication device 640 may receive time information about the content being executed from the content execution device 630 , and the receiver 611 may receive time information about the content received by the communication device 640 again.

The content execution device 630 and the communication device 640 may be connected via wire or wireless, and the communication device 640 and the synchronization device 610 may also be connected via wire or wireless. Preferably, the content execution device 630 and the communication device 640 are connected through a wireless method such as Bluetooth, and the communication device 640 and the synchronization device 620 are connected through a USB communication method.

The synchronization unit 612 generates link data by matching the received handwriting stroke information and content information. The synchronization unit 612 synchronizes the received handwriting stroke time information and the time information for the content, and then, the synchronization unit 612 is configured to synchronize the received handwriting stroke information, the received content information, and the synchronized time information based on the to create link data.

The synchronization device 610 or the device in which the synchronization device 610 is embedded may display the handwriting trace on the screen based on the handwriting information received from the electronic pen 620 or the display device. In addition, after the writing is completed, the playback unit 614 may store the file in a predetermined playback file or document file format, and then reproduce the writing trace in chronological order according to the user's selection.

In one embodiment of the present invention, when the content execution device 630 replays the content, the playback unit 614 may reproduce the handwriting stroke based on the link data together. The content and the handwriting stroke are reproduced according to the synchronized time in the reproduction unit 614 . In this case, the user can confirm that the handwritten content is reproduced according to the synchronized time while viewing the content.

Conversely, when a handwriting stroke is reproduced in the synchronization device 610, a device connected to the synchronization device 610, or a device that reproduces a handwriting trajectory, the content is transferred to the content execution device according to time information on the synchronized content based on the link data. It may be played back at 630 .

In an embodiment of the present invention, when a predetermined icon is selected from a print written by the electronic pen 620 or a display screen written by a stylus, the control unit 613 is configured to add the selected icon to link data existing in a page. After calling the corresponding content accordingly, it can also be played. In this case, the handwriting stroke synchronized with the content may be reproduced.

According to an embodiment of the present invention, when there is an input or touch within a predetermined distance from a writing stroke on a printed matter written by the electronic pen 620 or a display screen written by a stylus, the controller 613 controls a link to the corresponding writing stroke After calling the corresponding content according to the data, it can be executed or played back. In this case, the handwriting stroke matched with the content may also be reproduced.

For example, when the electronic pen 620 or the stylus touch satisfies at least two of a case where a predetermined time elapses after the last stroke, a case where the touch time is within a predetermined time, and a case where a stroke with respect to the touch is less than or equal to a predetermined length In this case, the controller 613 recognizes it as a touch. If not satisfied, the controller 613 does not call the content by recognizing it as handwriting.

According to another embodiment of the present invention, when a pre-appointed handwriting (eg, replay handwriting) is performed on a print written by the electronic pen 620 or a display screen handwritten by the stylus, the controller 613 controls the handwriting After calling the corresponding content according to the link data for the page, it may be played or executed. In this case, the reproducing unit 614 may reproduce the handwriting stroke synchronized with the content.

In another embodiment of the present invention, when the display device connected to the electronic pen 620 or the stylus performs a predetermined gesture, the controller 613 calls the corresponding content according to link data existing in the handwritten page. After that, it can be played or executed. In this case, the reproducing unit 614 may reproduce the handwriting stroke synchronized with the content.

In the case of reproducing content, particularly, in the case of reproducing video or audio, the video may be rewinded or reproduced at double speed.

During content reproduction, when time information on content is sequentially received and compared, the controller 613 may recognize whether content reproduction is in a forward or reverse direction.

Also, the controller 613 may calculate the speed information of the content by considering the time difference for the plurality of contents based on the time difference of the plurality of handwriting strokes.

The synchronization unit 612 may add information on the execution direction and speed of the recognized content to the link data as metadata. Alternatively, link data may be updated by creating these fields.

In another embodiment of the present invention, the receiving unit 611 may receive the content execution direction information and the speed information from the content execution device. In this case, the synchronization unit 612 may add the received content execution direction information and the speed information as metadata to the link data.

When the control unit 613 does not receive time information on the content, the controller 613 extracts the time information on the last received content.

Recently, there are many cases in which content providers insert advertisement images at the beginning or in the middle of content in order to increase revenue. In this case, it is common that the advertisement image has nothing to do with the content of the content. Accordingly, there is a problem in that the contents are not accurately matched because the handwriting made during the reproduction of the advertisement image is not related to the advertisement image.

When there is an advertisement in the content, the content execution device 630 does not transmit information on the content, in particular, time information on the content. In this case, the control unit 613 extracts time information about the last received content.

In another embodiment, the controller 613 may extract time information about the content to be received next.

The synchronization unit 612 resynchronizes the received handwriting stroke time information and the extracted time information on the content. After correcting the handwriting stroke time received during advertisement image reproduction to the time before or after advertisement reproduction, the synchronization unit 612 matches and resynchronizes the corresponding time information on the content being reproduced.

Thereafter, the synchronization unit 612 generates or updates the link data based on the handwritten stroke information, the content information, and the resynchronized time information.

Hereinafter, the content execution device 630 and the synchronization device 610 are first connected to generate link data, and then, the synchronization device 610 and the electronic pen 620 or the display device are connected to each other based on the handwriting information. An embodiment related to updating link data. Descriptions of portions overlapping with FIG. 3 or the above embodiment will be omitted.

The receiver 611 receives information about the content being executed from the content execution device 630 . When content is being reproduced through the display of the content execution device 630 , the receiver 611 receives information about the content being executed or reproduced from the content execution device 630 . The information on the content may include identification information (eg, ID) of the content, provider information, storage information or hyperlink information (eg, URL), and time information on the content. The time information on the content includes total execution (playback) time information and current execution time information of the content.

When receiving information on the content being executed, the controller 613 extracts time information in the synchronization device 610 . The control unit 613 extracts time information from the internal clock in real time when receiving information about content.

The synchronization unit 612 generates link data by matching the information on the received content and the extracted time information. The synchronization unit 612 synchronizes the extracted time information and time information on content. Thereafter, the synchronization unit 612 generates link data based on the received time information, the information on the received content, and the synchronized time information.

When the synchronization device 610 is connected to the electronic pen 620 , the receiver 611 receives handwriting stroke information from the electronic pen 620 . Here, the electronic pen 620 should generate and store handwriting stroke information in a state in which it is not connected to the synchronization device 610 .

The synchronization unit 612 updates the link data by matching the handwritten stroke information, the extracted time information, and the content information. The synchronization unit 612 resynchronizes the extracted time information with the handwriting stroke time information included in the handwriting stroke information. Since the extracted time information and the time information about the content are already synchronized, the handwriting stroke time information and the extracted internal time information need only be synchronized. Thereafter, the synchronizer 612 updates the link data by matching the handwritten stroke information, the extracted time information, and the content information.

When the control unit 613 does not receive time information on the content, the controller 613 extracts the time information on the last received content.

When there is an advertisement in the content, the content execution device 630 does not transmit information on the content, in particular, time information on the content. In this case, the control unit 613 extracts time information about the last received content.

In another embodiment, the controller 613 may extract time information about the content to be received next.

The synchronization unit 612 resynchronizes the extracted time information and the extracted time information about the last received content. The synchronization unit 612 corrects the internal time extracted during advertisement image reproduction to a time before or after advertisement reproduction, and resynchronizes by matching the corresponding time information with respect to the content being executed. Thereafter, the synchronization unit 612 generates or updates link data based on the extracted time information, content information, and resynchronized time information.

The following is an embodiment of a case in which the electronic pen 620 or the display device transmits only location information without transmitting time information. Here too, a description of the parts overlapping with FIG. 4 or the above embodiment will be omitted.

The receiver 611 receives handwriting stroke information from the electronic pen 620 . The handwriting stroke information may include handwriting stroke position information excluding time information, information on printed matter, and page information of the handwritten print.

The receiver 611 receives information about the content being executed from the content execution device 630 . When content is being reproduced through the display of the content execution device 630 , the receiver 611 receives information about the content being executed or reproduced from the content execution device 630 . The information on the content may include identification information (eg, ID) of the content, provider information, storage information or hyperlink information (eg, URL), and time information on the content. The time information on the content includes total execution (playback) time information and current execution time information of the content.

According to another embodiment of the present invention, the information on the content may include only location information within the content, not including the execution time information. Information on the content being executed may include identifier information, location information, and storage path information of the content being executed. Identifier information of content includes file name, file type, account, ID, login information, creation date, creator, modification date, modified person, file size, tag, owner or copyright holder, for example, and location information is a video or broadcast program In the case of an application, it may mean time information, in the case of an application, it may be a cursor or a point position in the application when receiving, and the storage path information indicates storage location information in the case of an image display device, and is stored in an external device. may be hyperlink information.

When receiving information on the content being executed, the controller 613 extracts time information in the synchronization device 610 . The control unit 613 extracts time information from the internal clock in real time when receiving information about content. Since the electronic pen 620 or the display device does not transmit the handwriting stroke time information, when the handwriting stroke information is first received, the handwriting stroke time information may be matched according to the internal time information. That is, the internal time information and the handwriting stroke time information are the same.

The synchronization unit 612 generates link data by matching the information on the received content and the extracted time information. The synchronization unit 612 synchronizes the extracted time information and time information on content. Thereafter, the synchronization unit 612 generates link data based on the received time information, the information on the received content, and the synchronized time information.

The synchronization unit 612 updates the link data by matching the handwritten stroke information, the extracted time information, and the content information. The synchronization unit 612 resynchronizes the extracted time information with the handwriting stroke time information included in the handwriting stroke information. Since the extracted time information and the time information about the content are already synchronized, the handwriting stroke time information and the extracted internal time information need only be synchronized. Thereafter, the synchronizer 612 updates the link data by matching the handwritten stroke information, the extracted time information, and the content information.

When the control unit 613 does not receive time information on the content, the controller 613 extracts the time information on the last received content. When there is an advertisement in the content, the content execution device 630 does not transmit information on the content, in particular, time information on the content. In this case, the control unit 613 extracts time information about the last received content. In another embodiment, the controller 613 may extract time information about the content to be received next.

The synchronization unit 612 resynchronizes the extracted time information and the extracted time information about the last received content. After correcting the internal time extracted during advertisement image reproduction to the time before or after advertisement reproduction, the synchronization unit 612 matches and resynchronizes the corresponding time information on the content being executed. Thereafter, the synchronization unit 612 generates or updates link data based on the extracted time information, content information, and resynchronized time information.

Hereinafter, an embodiment in which a writing stroke time received from the electronic pen 620 is changed and synchronized is related. Here too, a description of a part overlapping with FIG. 5 or the above embodiment will be omitted.

The receiver 611 receives handwriting stroke information from the electronic pen 620 .

The receiver 611 receives time information about the content being executed from the content execution device 630 .

The controller 613 extracts handwriting stroke time information from the received handwriting stroke information, and corrects the extracted time information by moving it back by a predetermined amount of time. In general, after viewing the content, the user takes notes about the content. There is a problem in that when the content is re-executed later, it is not accurately matched with the written content. Accordingly, the handwriting stroke time information is modified by the time set by the user in advance for matching them. When the handwriting stroke time information is corrected, the contents of the two are matched to be reproduced or executed.

The synchronization unit 612 generates link data by matching the corrected handwriting stroke time information and the content information. The synchronization unit 612 synchronizes the corrected handwriting stroke time information and time information on content. Thereafter, the synchronization unit 612 generates link data based on the corrected handwriting stroke information, information on the received content, and synchronized time information.

When the control unit 613 does not receive time information on the content, the controller 613 extracts the time information on the last received content. When there is an advertisement in the content, the content execution device 630 does not transmit information on the content, in particular, time information on the content. In this case, the control unit 613 extracts time information about the last received content. In another embodiment, the controller 613 may extract time information about the content to be received next.

The synchronization unit 612 resynchronizes the received handwriting stroke time information and the extracted time information on the content. After correcting the handwriting stroke time received during advertisement image reproduction to the time before or after advertisement reproduction, the synchronization unit 612 matches and resynchronizes the corresponding time information on the content being reproduced. Thereafter, the synchronization unit 612 generates or updates the link data based on the handwritten stroke information, the content information, and the resynchronized time information.

The writing and content synchronization method as described above can also be implemented as computer-readable codes on a computer-readable recording medium. The computer-readable recording medium includes any type of recording medium in which data readable by a computer system is stored. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage device. In addition, the computer-readable recording medium is distributed in a computer system connected to a network, so that the computer-readable code can be stored and executed in a distributed manner. In addition, functional programs, codes, and code segments for implementing the disk management method can be easily inferred by programmers in the art to which the present invention pertains.

So far, the present invention has been looked at with respect to preferred embodiments thereof. Those of ordinary skill in the art to which the present invention pertains will understand that the present invention can be implemented in a modified form without departing from the essential characteristics of the present invention. Therefore, the disclosed embodiments are to be considered in an illustrative rather than a restrictive sense. The scope of the present invention is indicated in the claims rather than the foregoing description, and all differences within the scope equivalent thereto should be construed as being included in the present invention.

610: synchronizer
611: receiving unit 612: synchronizing unit
613: control unit 614: playback unit
620: electronic pen
630: content playback device
640: communication device

Claims (36)

receiving handwriting stroke information from an electronic pen or a display device;
Receiving information about the content being executed from the content execution device; and
and generating link data by matching the handwriting stroke information with the content information. The method of claim 1,
generating link data by matching the handwriting stroke information and the information on the content;
synchronizing handwriting stroke time information included in the handwriting stroke information and time information on content included in the information on the content; and
and generating link data based on the handwriting stroke information, the content information, and the synchronized time information. 3. The method of claim 2,
extracting time information on the last received content when the time information on the content is not received;
resynchronizing the handwriting stroke time information included in the handwriting stroke information and the time information on the extracted last received content; and
and generating link data based on the handwriting stroke information, the content information, and the resynchronized time information. The method of claim 1,
determining execution direction information and execution speed information of the currently running content based on the plurality of pieces of handwriting stroke time information and time information on the content; and
and adding the determined execution direction information and execution speed information of the content being executed to the link data. The method of claim 1,
and replaying the handwriting stroke based on the link data when the content is replayed. The method of claim 1,
Calling and playing the content based on the link data when the electronic pen is touched at a predetermined position in the printed matter in which the handwriting stroke exists, or when a predetermined input is received at a position adjacent to the handwriting stroke displayed on the display device A playback time synchronization method, characterized in that it further comprises a step. Receiving information about the content being executed from the content execution device;
extracting time information when receiving information about the content being executed;
generating link data by matching the extracted time information and information on the content;
receiving handwriting stroke information from the electronic pen or the display device when connected to the electronic pen or the display device; and
and updating the link data by matching the handwriting stroke information, the extracted time information, and the content information. 8. The method of claim 7,
The step of generating link data by matching the extracted time information and the information on the content includes:
synchronizing the extracted time information and time information on content included in the information on the content; and
and generating link data based on the extracted time information, information on the content, and the synchronized time information,
The updating of the link data by matching the handwritten stroke information, the extracted time information, and the information on the content may include:
resynchronizing the handwriting stroke time information included in the handwriting stroke information, the extracted time information, and time information on the content; and
and updating link data based on the handwriting stroke information, the extracted time information, the content information, and the resynchronized time information. 8. The method of claim 7,
extracting time information on the last received content when the time information on the content is not received;
resynchronizing the extracted time information and the extracted time information for the last received content; and
and generating link data based on the extracted time information, the content information, and the resynchronized time information. 8. The method of claim 7,
determining execution direction information and execution speed information of the currently executing content based on a plurality of the extracted time information and time information on the content; and
and adding the determined execution direction information and execution speed information of the content being executed to the link data. 8. The method of claim 7,
and replaying the handwriting stroke based on the link data when the content is replayed. 8. The method of claim 7,
Calling and playing the content based on the link data when the electronic pen is touched at a predetermined position in the printed matter in which the handwriting stroke exists, or when a predetermined input is received at a position adjacent to the handwriting stroke displayed on the display device A playback time synchronization method, characterized in that it further comprises a step. receiving handwriting stroke information from an electronic pen or a display device;
Receiving information about the content being executed from the content execution device;
extracting time information when receiving information about the content being executed; and
and generating link data by matching the handwriting stroke information, the extracted time information, and the content information. 14. The method of claim 13,
generating link data by matching the handwritten stroke information, the extracted time information, and the information on the content;
synchronizing the extracted time information and time information on content included in the information on the content; and
and generating link data based on the extracted time information, the content information, and the synchronized time information. 14. The method of claim 13,
extracting time information on the last received content when the time information on the content is not received;
resynchronizing the extracted time information and the extracted time information for the last received content; and
and generating link data based on the handwriting stroke information, the extracted time information, the content information, and the resynchronized time information. 14. The method of claim 13,
determining execution direction information and execution speed information of the currently executing content based on a plurality of the extracted time information and time information on the content; and
and adding the determined execution direction information and execution speed information of the content being executed to the link data. 14. The method of claim 13,
and replaying the handwriting stroke based on the link data when the content is replayed. 14. The method of claim 13,
Calling and playing the content based on the link data when the electronic pen is touched at a predetermined position in the printed matter in which the handwriting stroke exists, or when a predetermined input is received at a position adjacent to the handwriting stroke displayed on the display device A playback time synchronization method, characterized in that it further comprises a step. a receiver configured to receive handwriting stroke information from an electronic pen or a display device, and to receive information on content being executed from a content execution device; and
and a synchronization unit to generate link data by matching the handwriting stroke information and the content information. 20. The method of claim 19,
The synchronization unit synchronizes the handwriting stroke time information included in the handwriting stroke information and the time information on the content included in the content information, based on the handwriting stroke information, the content information, and the synchronized time information Handwriting and content synchronization device, characterized in that for generating link data. 21. The method of claim 20,
When the time information for the content is not received, the control unit further comprises a control unit for extracting the time information about the last received content,
The synchronization unit resynchronizes the handwriting stroke time information included in the handwriting stroke information and the time information on the extracted last received content, based on the handwriting stroke information, the content information, and the resynchronized time information Handwriting and content synchronization device, characterized in that for generating link data. 20. The method of claim 19,
Execution direction information and execution speed information of the content being executed are determined based on the plurality of pieces of handwriting stroke time information and time information on the content, and the determined execution direction information and execution speed information of the content being executed are combined with the link data Handwriting and content synchronization device, characterized in that it further comprises a control unit added to the. 20. The method of claim 19,
and a reproducing unit that reproduces the handwriting stroke based on the link data when the content is replayed. 20. The method of claim 19,
Calling and playing the content based on the link data when the electronic pen is touched at a predetermined position in the printed matter in which the handwriting stroke exists, or when a predetermined input is received at a position adjacent to the handwriting stroke displayed on the display device Playback time synchronization device, characterized in that it further comprises a control unit. a receiving unit for receiving information on the content being executed from the content execution device;
a control unit for extracting time information when receiving information about the content being executed; and
A synchronization unit for generating link data by matching the extracted time information and information on the content;
When connected to an electronic pen or a display device, the receiver receives handwriting stroke information from the electronic pen or the display device, and the synchronizer matches the handwriting stroke information, the extracted time information, and the information on the content to obtain the information on the content. A device for synchronizing writing and contents, characterized in that the link data is updated. 26. The method of claim 25,
The synchronization unit generates link data based on the extracted time information, the information on the content, and the synchronized time information by synchronizing the time information on the content included in the extracted time information and the information on the content, and , by resynchronizing the handwriting stroke time information included in the handwriting stroke information, the extracted time information, and the time information about the content, the handwriting stroke information, the extracted time information, the information about the content, and the resynchronized A device for synchronizing writing and contents, characterized in that the link data is updated based on time information. 26. The method of claim 25,
When the control unit does not receive the time information for the content, extracts the time information for the last received content,
The synchronization unit resynchronizes the extracted time information and the extracted time information on the last received content to generate link data based on the extracted time information, the information on the content, and the resynchronized time information Handwriting and content synchronization device, characterized in that it further comprises a step. 26. The method of claim 25,
The control unit determines execution direction information and execution speed information of the content being executed based on a plurality of the extracted time information and time information on the content, and the synchronization unit determines execution direction information and execution direction information of the content being executed A device for synchronizing writing and content, characterized in that adding speed information to the link data. 26. The method of claim 25,
and a reproducing unit that reproduces the handwriting stroke based on the link data when the content is replayed. 26. The method of claim 25,
The control unit calls the content based on the link data when the electronic pen is touched at a predetermined position in the printed matter in which the handwriting stroke exists, or when there is a predetermined input at a position adjacent to the handwriting stroke displayed on the display device A playback time synchronization device, characterized in that the playback. a receiver configured to receive handwriting stroke information from an electronic pen or a display device, and to receive information on content being executed from a content execution device;
a control unit for extracting time information when receiving information about the content being executed; and
and a synchronization unit to generate link data by matching the handwriting stroke information, the extracted time information, and the content information. 32. The method of claim 31,
The synchronization unit synchronizes the extracted time information and the time information on the content included in the information on the content to generate link data based on the extracted time information, the information on the content, and the synchronized time information Handwriting and content synchronization device, characterized in that. 32. The method of claim 31,
When the control unit fails to receive time information on the content, the controller extracts time information on the last received content, and the synchronization unit resynchronizes the extracted time information and the extracted time information on the last received content to generate link data based on the handwriting stroke information, the extracted time information, the content information, and the resynchronized time information. 32. The method of claim 31,
The control unit determines execution direction information and execution speed information of the content being executed based on a plurality of the extracted time information and time information on the content, and the synchronization unit determines execution direction information and execution direction information of the content being executed A device for synchronizing writing and content, characterized in that adding speed information to the link data. 32. The method of claim 31,
and a reproducing unit that reproduces the handwriting stroke based on the link data when the content is replayed. 32. The method of claim 31,
The control unit calls the content based on the link data when the electronic pen is touched at a predetermined position in the printed matter in which the handwriting stroke exists, or when there is a predetermined input at a position adjacent to the handwriting stroke displayed on the display device A playback time synchronization device, characterized in that the playback.

Images