CN107077245B

CN107077245B - Electronic pen, electronic device linked to electronic pen, and method for controlling electronic device

Info

Publication number: CN107077245B
Application number: CN201580056358.9A
Authority: CN
Inventors: 李相奎; 文疋滥; 韩侑辰; 朴鋕垸
Original assignee: Neolab Convergence Inc
Current assignee: Neolab Convergence Inc
Priority date: 2014-10-17
Filing date: 2015-10-08
Publication date: 2020-04-24
Anticipated expiration: 2035-10-08
Also published as: KR20160045540A; KR20160045539A; CN111459308A; CN111459312A; CN107077245A; CN111459309A; KR20160045537A; CN111459311A; KR102286589B1; KR102487643B1; CN111459310A; KR102286587B1

Abstract

The present invention provides an electronic device for reproducing notes written by a user on a medium. The electronic device includes: a communication unit for communicating with an electronic pen that detects information about a note-taking of a user; a display for displaying at least a portion of a media image corresponding to the media, on which notes recorded by a user are reproduced and visually represented; and a control unit for determining whether a position of a note recording of the user satisfies a predetermined condition for a first area corresponding to at least a portion of the media image being displayed through the display based on the pen data received from the communication unit, moving the portion of the media image being displayed through the display from the first area to a second area different from the first area and displaying the second area through the display when the predetermined condition is satisfied, and displaying a reproduction of the note recorded by the user at a position corresponding to the position of the note recording of the user on the second area to reproduce the note recorded by the user.

Description

Electronic pen, electronic device linked to electronic pen, and method for controlling electronic device

Technical Field

The present invention relates to an electronic pen, an electronic device linked to the electronic pen, and a method for controlling the electronic device, and more particularly, to an electronic pen, an electronic device linked to the electronic pen, and a method for controlling the electronic device that recognizes and displays writing of a user.

Background

In recent years, technologies and products for recognizing writing of a user using an electronic pen and digitizing and storing notes have been commercialized. However, a method of providing a user with notes that are recognized using such an electronic pen and digitized in a meaningful manner has not been satisfactorily developed. Thus, there is a need to develop methods that enable users to effectively utilize digitized notes.

In general, a conventional electronic pen has a lead (lead) at the center of a body portion gripped by a user, and adopts the configuration of an existing writing tool. A conventional electronic pen is inserted and held in its cover when not in use.

Here, according to the related art, when the body part is inserted into the cap, the body part is inserted and fixed into the cap by forming a protrusion on an outer surface of the body part, forming a groove on an inner surface of the cap, and inserting the protrusion into the groove, so that the body part and the cap are prevented from being separated from each other.

In such a conventional method, the protrusion is frequently damaged by an external factor that may occur when the user uses the main body, which results in degradation of the coupling force between the main body and the cover, thereby causing a reduction in the service life of the electronic pen.

Disclosure of Invention

[ problem ] to provide a method for producing a semiconductor device

An object of the present invention is to provide an electronic pen, an electronic device linked to the electronic pen, and a control method of the electronic device configured to recognize writing of a user in real time and reproduce the recognized writing of the user.

The objects intended to be achieved by the present invention are not limited to the above, and other objects not described herein will be clearly understood by those skilled in the art from the following description and the accompanying drawings.

[ technical solution ] A

According to an aspect of the present invention, there is provided an electronic device for reproducing a note written on a medium by a user, the electronic device including: a communication unit configured to communicate with an electronic pen that detects information about writing of a user; a display configured to display at least a portion of a media image corresponding to the media, reproduce a note written by a user on the media image, and visually represent the reproduced note; and a control unit configured to determine whether a position of writing by the user satisfies a predetermined condition for a first region that is at least a part of the medium image being displayed through the display based on the pen data received from the communication unit, configured to move the part of the medium image being displayed through the display from the first region to a second region different from the first region and display the second region through the display when the predetermined condition is satisfied, and configured to display a reproduction of a note written by the user at a position on the second region corresponding to the position of writing by the user to reproduce the note written by the user.

According to another aspect of the present invention, there is provided a control method of an electronic device for reproducing a note written by a user on a medium, the control method including determining whether a position of the user's writing to a first area, which is at least a part of a medium image being displayed through a display, satisfies a predetermined condition based on pen data received from a communication unit for communicating with an electronic pen for detecting information on the user's writing; moving a portion of the media image being displayed by the display from a first area to a second area different from the first area and displaying the second area by the display when a predetermined condition is satisfied such that a reproduction of a note written by a user is displayed by the display; and displaying a reproduction of the note written by the user at a position on the second area corresponding to the position of the user's writing to reproduce the note written by the user.

According to still another aspect of the present invention, there is provided a recording medium having recorded thereon a program for executing the above-described control method of an electronic apparatus.

[ advantageous effects of the invention ]

According to the present invention, by performing a shift on an area corresponding to a position of writing by a user and reproducing a note written by the user in real time, it is possible to effectively and efficiently provide the user with real-time reproduction of the note written by the user even when the note written by the user is reproduced on an enlarged screen.

Drawings

Fig. 1 is a schematic diagram illustrating a system including an electronic pen, an electronic device linked to the electronic pen, and the like, associated with an embodiment of the present invention.

Fig. 2 is a diagram showing an example of media associated with an embodiment of the present invention.

Fig. 3 is a diagram showing an example appearance of an electronic pen according to an embodiment of the present invention.

FIG. 4 is a block diagram illustrating elements of an electronic pen according to an embodiment of the present invention.

FIG. 5 is a block diagram illustrating elements of an electronic device according to an embodiment of the invention.

Fig. 6 and 7 are diagrams illustratively showing unit cells constituting a code pattern according to an embodiment of the present invention.

Fig. 8 is a diagram illustrating a method of encoding 2-bit information into an information code according to an embodiment of the present invention.

Fig. 9 is a diagram illustrating a method of generating information on a position of an electronic pen with respect to a medium according to an embodiment of the present invention.

Fig. 10 is a diagram illustrating an embodiment of reproducing notes written by a user in real time according to the present invention.

Fig. 11 to 15 are diagrams illustrating another embodiment of reproducing notes written by a user in real time according to the present invention.

Fig. 16 is a diagram illustrating a playback section according to the present invention.

Fig. 17 is a diagram illustrating a playback reproduction time determined according to an embodiment of the present invention for the playback section shown in fig. 16.

Fig. 18 and 19 are diagrams illustrating a playback reproduction time determined according to another embodiment of the present invention for the playback section shown in fig. 16.

20-24 are diagrams illustrating other examples of determining a playback reproduction time for replaying strokes written by a user according to some embodiments of the present invention.

FIG. 25 is a diagram illustrating a method of determining a playback reproduction time based on a stroke number according to some embodiments of the invention.

FIG. 26 is a diagram illustrating an example of determining a playback reproduction speed based on the number of strokes included in a playback target according to some embodiments of the invention.

FIG. 27 is a diagram illustrating an example of determining the number of strokes played back per unit time based on playback reproduction speed according to some embodiments of the present invention.

FIG. 28 is a diagram illustrating a method of performing a playback operation on a user's writing according to some embodiments of the invention.

Fig. 29 is a diagram illustrating an entire note and a target note at the time of a capture operation of an electronic device according to the present invention.

Fig. 30 to 32 show examples of user interfaces for receiving a target section according to the present invention.

Fig. 33 to 35 show another example of a user interface for receiving a target section according to the present invention.

Fig. 36 and 37 show still another example of a user interface for receiving a target section according to the present invention.

Fig. 38 is a diagram illustrating a screen to output an entire note to receive a target section according to the present invention.

Fig. 39 and 40 are diagrams illustrating a screen for recognizing a target note according to the present invention.

Fig. 41 is a diagram showing a captured image of the present invention.

Fig. 42 is a schematic perspective view illustrating an electronic pen having a cap part and a body part separated from each other according to an embodiment of the present invention.

Fig. 43 is a schematic perspective view illustrating an electronic pen in which a cap part is inserted into an opposite side of a main body part according to an embodiment of the present invention.

Fig. 44 is a schematic perspective view showing an internal configuration of a cap portion and a body portion of an electronic pen according to an embodiment of the present invention.

Fig. 45 is a schematic perspective view taken along line a-a of fig. 44.

Fig. 46 is a schematic perspective view of a modification of a cap portion of an electronic pen according to an embodiment of the present invention.

Fig. 47 is a schematic front view showing a main body portion of an electronic pen according to an embodiment of the present invention.

Fig. 48 and 49 are schematic side views illustrating a process of inserting a cap part into one side of a main body part of an electronic pen according to an embodiment of the present invention.

Fig. 50 and 51 are schematic exploded perspective views illustrating a main body portion of an electronic pen according to an embodiment of the present invention.

Fig. 52 to 54 are schematic perspective views showing a substrate section, a camera module, and an illumination module fixed in a main body section of an electronic pen according to an embodiment of the present invention.

Fig. 55 to 57 are schematic perspective views illustrating a process of inserting a lead part into a main body part of an electronic pen according to an embodiment of the present invention.

FIG. 58 is a flowchart of a first embodiment of a method for real-time rendering of notes written by a user in accordance with the present invention.

FIG. 59 is a flowchart of a second embodiment of a method for real-time rendering of notes written by a user in accordance with the present invention.

FIG. 60 is a flowchart of a first embodiment of a playback method for written notes in accordance with the present invention.

FIG. 61 is a flow chart of a second embodiment of a playback method for written notes in accordance with the present invention.

FIG. 62 is a flow diagram of a writing capture method according to an embodiment of the present invention.

Detailed Description

The embodiments described in this specification are given to clearly explain the scope of the present invention to those skilled in the art and are not intended to limit the present invention. It should be understood that the present invention may include substitutions and modifications within the technical scope of the present invention.

Terms used in the present specification are selected from general terms widely used based on functions of the present invention, and may have meanings changed according to intentions of those skilled in the art, conventions of the art, or advances of new technologies. If a specific term having a specific meaning is used, the meaning of the term will be described specifically. Thus, the terms used in the present specification should not be defined as simple names of components but defined based on the actual meanings of the terms and the entire context throughout the present specification.

The drawings are for facilitating the explanation of the present invention, and the shape in the drawings may be exaggerated for the convenience of explanation, so that the present invention is not limited to the drawings.

In addition, details of generally known functions and structures which make the subject matter of the present invention unclear will be omitted.

Electronic devices linked to the electronic pen described herein may include mobile terminals such as cellular phones, smart phones, laptop computers, digital broadcast terminals, Personal Digital Assistants (PDAs), Portable Multimedia Players (PMPs) and navigation devices, and stationary terminals such as digital TVs and desktop computers.

According to an aspect of the present invention, there may be provided an electronic device for reproducing a note written on a medium by a user, the electronic device including: a communication unit configured to communicate with an electronic pen that detects information about writing of a user; a display configured to display at least a portion of a media image corresponding to the media, reproduce a note written by a user on the media image, and visually represent the reproduced note; and a control unit configured to determine whether a position of writing by the user satisfies a predetermined condition for a first region that is at least a part of the medium image being displayed through the display based on the pen data received from the communication unit, configured to move the part of the medium image being displayed through the display from the first region to a second region different from the first region and display the second region through the display when the predetermined condition is satisfied, and configured to display a reproduction of a note written by the user at a position on the second region corresponding to the position of writing by the user to reproduce the note written by the user.

The predetermined condition may be whether the location of the user's writing is outside the first region.

Alternatively, the predetermined condition may be whether the position of the user's writing is included in a boundary area of the first area.

The second region may be determined based on a location of the user's writing on the medium.

In this case, the second area may be determined such that the position of the user's writing on the medium is placed at the center portion.

Optionally, the second region may be determined further taking into account the position of the edge of the media image.

In particular, the second region may be determined such that the position of the edge of the medium image is not separated from the boundary region of the screen being displayed on the display by a predetermined distance or more.

When determining whether the predetermined condition is satisfied, the control unit may determine whether the predetermined condition is satisfied using at least some of the plurality of pieces of pen data for one stroke included in the writing of the user.

Also, the control unit may determine whether a predetermined condition is satisfied using first stroke data corresponding to the pen-down time point among the plurality of stroke data for one stroke.

Also, the control unit may determine the second area based on position information included in the first data when a predetermined condition is satisfied.

The control unit may determine whether the predetermined condition is satisfied using a second pen data corresponding to the pen-up time point among the plurality of pen data for one stroke.

Also, when a predetermined condition is satisfied, the control unit may determine the second area based on the position information included in the second pen data.

Also, the control unit may determine whether a predetermined condition is satisfied for each of a plurality of strokes included in the user's writing.

According to another aspect of the present invention, there may be provided an electronic device for reproducing a note written by a user on a medium, the electronic device including: a communication unit configured to communicate with an electronic pen that detects information about writing of a user; a display configured to display at least a portion of a media image corresponding to the media, reproduce a note written by a user on the media image, and visually represent the reproduced note; and a control unit configured to display a first stroke included in the user's writing at a corresponding position in the medium image, receive pen data constituting a second stroke included in the user's writing, determine whether a position of the second stroke satisfies a predetermined condition based on at least one piece of the pen data constituting the second stroke, and move at least a partial area of the medium image being displayed through the display according to a distance and a direction with respect to the first stroke based on the at least one piece of the pen data constituting the second stroke when the predetermined condition is satisfied.

The predetermined condition may be whether or not the position information of at least one piece of pen data constituting the second stroke is outside a partial area of the medium image being displayed through the display.

The control unit may move at least a partial region of the medium image such that a position corresponding to the at least one piece of pen data is placed in the center portion.

The at least one piece of pen data may be at least one of pen data corresponding to the pen-down time point and pen data corresponding to the pen-up time point among the plurality of pieces of pen data constituting the second stroke.

According to still another aspect of the present invention, there may be provided an electronic device for reproducing a note written on a medium by a user, the electronic device including: a communication unit configured to communicate with an electronic pen that detects information about writing of a user; a display configured to display a medium image corresponding to the medium, reproduce a note written by a user on the medium image, and visually represent the reproduced note; and a control unit configured to determine whether a note written by a user is reproduced through the display in a normal mode or an enlarged mode, reproduce the note written by the user in a first reproduction mode when reproducing in the normal mode, and reproduce the note written by the user in a second reproduction mode when reproducing in the enlarged mode, wherein the control unit moves a partial area of the medium image being displayed through the display in the enlarged mode according to a position of the user's writing when reproducing the note written by the user in the second reproduction mode.

According to still another aspect of the present invention, there is provided a control method of an electronic device for reproducing a note written by a user on a medium, the control method including determining whether a position of the user's writing to a first area, which is at least a portion of an image of the medium being displayed through a display, satisfies a predetermined condition based on pen data received from a communication unit for communicating with an electronic pen for detecting information on the user's writing; moving a portion of the media image being displayed by the display from a first area to a second area different from the first area and displaying the second area by the display when a predetermined condition is satisfied such that a reproduction of a note written by a user is displayed by the display; and displaying a reproduction of the note written by the user at a position on the second area corresponding to the position of the user's writing to reproduce the note written by the user.

The predetermined condition may be whether a position of the user's writing is included in a boundary area of the first area.

The second region may be determined such that the user's writing position on the medium is placed in the center portion.

The determining may include determining whether a predetermined condition is satisfied using at least some of the plurality of pieces of pen data for one stroke included in the user's writing.

The determining may include determining whether a predetermined condition is satisfied using first pen data corresponding to the pen-down time point among the plurality of pen data for one stroke.

When a predetermined condition is satisfied, the second area may be determined based on the position information included in the first data.

The determining may include determining whether a predetermined condition is satisfied using a second pen data corresponding to the pen-up time point among the plurality of pen data for one stroke.

When a predetermined condition is satisfied, the second area may be determined based on the position information included in the second pen data.

The determining may include determining whether a predetermined condition is satisfied for each of a plurality of strokes included in the user's writing.

According to still another aspect of the present invention, there may be provided a control method of an electronic device for reproducing a note written by a user on a medium, the control method including receiving pen data constituting a first stroke included in the user's writing from an electronic pen; displaying a first stroke included in writing of a user at a corresponding position of the medium image; receiving pen data constituting a second stroke included in writing of the user from the electronic pen; determining whether a position of the second stroke satisfies a predetermined condition based on at least one piece of pen data constituting the second stroke; moving at least a partial area of the medium image being displayed through the display according to a distance and a direction of at least one piece of pen data constituting the second stroke with respect to the first stroke when a predetermined condition is satisfied; and displaying the second stroke in the moved partial area.

According to still another aspect of the present invention, there may be provided a control method of an electronic device for reproducing a note written by a user on a medium, the control method including determining whether the note written by the user is reproduced through a display in a normal mode or an enlarged mode; and reproducing the note written by the user in the first reproduction mode when the reproduction is performed in the normal mode, and reproducing the note written by the user in the second reproduction mode when the reproduction is performed in the enlargement mode. When reproducing the pen notes written by the user in the second reproduction mode, the control unit may move a partial area of the medium image being displayed in the enlargement mode through the display according to the position of the user's writing.

According to still another aspect of the present invention, there may be provided a recording medium having a program recorded thereon for executing the above-described control method of an electronic apparatus provided by the present invention.

First, an operating environment of an electronic pen and an electronic device linked to the electronic pen associated with some embodiments of the present invention will be briefly described, and then the electronic pen, the electronic device linked to the electronic pen, and a control method of the electronic device according to embodiments of the present invention will be described in detail.

< System configuration >

The system may include at least one medium 1000, at least one electronic pen 2000, at least one electronic device 3000, and at least one server 4000.

The medium 1000 provides a surface for writing notes of items desired by a user using the electronic pen according to an embodiment of the present invention.

The medium 1000 may be made of materials such as paper, plastic, and metal, but is not limited thereto.

When the electronic pen according to an embodiment of the present invention is an electronic pen implemented using an optical element including a camera module or the like (hereinafter, referred to as an optical electronic pen), the medium 1000 may include a code pattern that can be recognized by the electronic pen. The code pattern will be described in detail later. When the code pattern is included in the medium 1000, the code pattern may be provided to the medium 1000 by a method such as printing.

The electronic pen 2000 may detect information about the user's writing and generate pen data. The pen data may be used to generate stroke data regarding the user's writing. The pen data will be described in detail later.

The electronic device 3000 may receive pen data from the electronic pen 2000 and interpret the received pen data. The stroke data may be generated based on an interpretation of the pen data.

From the stroke data, the electronic device 3000 may appropriately process the interpreted stroke data to reproduce and display the user's writing in real time or to replay and display the user's writing at a later time through a display included in the electronic device 3000.

Also, the electronic device 3000 may store the pen data and/or stroke data generated based on the interpretation of the pen data in a memory or the like included in the electronic device 3000, and may appropriately process the pen data and/or stroke data according to a user's request.

The electronic device 3000 may be connected to the server 4000 via a network to transmit or receive necessary information through the network.

The server 4000 may be connected to the electronic device 3000 via a network to receive or transmit necessary information.

The server 4000 may be connected with a plurality of electronic devices 3000 to construct a database of pen data (or stroke data) received from the electronic devices 3000.

The server 4000 may appropriately process pen data (or stroke data) stored in the database according to requests from a plurality of electronic devices 3000 or requests of users.

The server 4000 may perform some or all of the functions of the electronic device 3000 that have been described above. For example, the electronic device 3000 may receive pen data from the electronic pen 2000 and immediately transmit the pen data to the server 4000 without performing a specific process on the pen data. In this case, the server 4000 may receive the pen data and perform operations of generating, storing, and additionally processing the pen data, as described above.

Fig. 1 shows that one medium 1000-1 corresponds to one electronic pen 2000-1. However, the correspondence between the medium 1000 and the electronic pen 2000 may be 1: n or N: 1.

also, fig. 1 shows that one electronic pen 2000-1 corresponds to one electronic device 3000-1. However, the correspondence between the electronic pen 2000 and the electronic device 3000 may be 1: n or N: 1.

further, fig. 1 shows that a plurality of electronic devices 3000 are connected to one server 4000. However, a system according to an embodiment of the present invention may include two or more servers.

Fig. 1 shows that the electronic pen 2000 communicates with the electronic device 3000 and that the electronic device 3000 communicates with the electronic pen 2000 and the server 4000. However, the electronic pen 2000 may directly communicate with the server 4000 as necessary.

< Medium >

Subsequently, the medium 1000 associated with the embodiment of the present invention will be described.

Fig. 2 shows that a notebook or book includes a plurality of media 1000. As shown in fig. 2, according to some embodiments of the invention, at least some or all of the pages making up a notebook or book may be implemented as a medium 1000 according to embodiments of the invention. For example, when the electronic pen 2000 is implemented as an optical electronic pen, a code pattern according to an embodiment of the present invention may be included in at least some or all of the pages constituting a notebook or book.

Only at least a partial region of the entire surface of the page constituting the notebook or book may be implemented as the medium 1000 according to an embodiment of the present invention. For example, when the electronic pen 2000 is implemented as an optical electronic pen, the code pattern may not be provided on the entire surface of the page constituting the notebook or book but in a partial area thereof.

Although not shown, the medium 1000 according to an embodiment of the present invention may be implemented in the form of a blackboard. That is, the medium 1000 according to an embodiment of the present invention may be implemented in the form of a blackboard (electronic blackboard) including a code pattern.

< electronic pen >

Subsequently, an electronic pen according to an embodiment of the present invention will be described.

Fig. 3 is a diagram showing an example appearance of an electronic pen according to an embodiment of the present invention, and fig. 4 is a block diagram showing elements of the electronic pen according to an embodiment of the present invention.

Referring to fig. 3, the electronic pen 2000 may mainly include: a body portion 2020 configured to fix a lead portion 2010 (see fig. 42) capable of writing notes on the medium 1000; a cap portion 2030 configured to protect the lead portion 2010 when the electronic pen 2000 is in an unused state or the like.

Here, the unused state may refer to a state in which the electronic pen 2000 according to an embodiment of the present invention is held. In this state, the lead part 2010 is not exposed to the outside, and thus it is impossible to write notes on the medium.

Lead portion 2010 is an element capable of writing notes on media 1000 and may include a ball and ink. In this case, the ink may include various forms such as oil, water, and neutral inks.

Also, the lead part 2010 may be a graphite lead used in a pencil, or may be a lead used in a general writing instrument.

The body portion 2020 is an element configured to fix the lead portion 2010 so as to be partially exposed when the lead portion 2010 is inserted therein, and may refer to a kind of grip portion that can be gripped by a user's hand.

The cap portion 2030 may have a predetermined inner space into which one side of the body portion 2020 may be inserted in an unused state to protect the lead portion 2010. When the cap portion 2030 is inserted into the other side of the body portion 2020, the lead portion 2010 may be partially exposed to the outside, enabling a note to be written.

However, when the cap portion 2030 enables writing with the lead portion 2010, the cap portion 2030 does not necessarily have to be inserted into the other side of the main body portion 2020, and may be separated from the main body portion 2020 and placed in a predetermined space.

Referring to fig. 4, an electronic pen according to an embodiment of the present invention may include an input unit 2100, an output unit 2200, a pattern detection unit 2300, a communication unit 2400, a sensor unit 2500, a memory 2600, a power supply unit 2700, a control unit 2800, and the like. However, the elements shown in FIG. 4 are not required, and thus the electronic pen may be implemented with more elements or fewer elements than shown in FIG. 4.

The elements of the electronic pen 2000 will be described in turn below.

The input unit 2100 may receive a user input from a user. The user input may include a button input, a key input, a touch input, a voice input, and various other inputs. For example, the electronic pen 2000 may receive a button input from a user to control power on/off of the electronic pen 2000.

As an example, the input unit 2100 is a broad concept including a touch sensor for detecting a user's touch, a microphone for receiving a voice signal, a camera for recognizing a gesture through image recognition, a proximity sensor composed of an illumination sensor or an infrared sensor for detecting a user's proximity, a motion sensor for recognizing a user's motion through an acceleration sensor, a gyro sensor, and the like, and various input units for detecting or receiving various user inputs, as well as typical buttons, typical scroll wheels, and the like.

The output unit 2200 may output various information and provide the information to a user. The output unit 2200 should be construed as a broad concept including a display for outputting visible light recognizable by the vision of a user, a speaker for outputting sound recognizable by the hearing of a user, a haptic device for generating vibration recognizable by the sense of touch of a user, and various output units. For example, the electronic pen 2000 may output a predetermined specific sound through a buzzer included in the electronic pen 2000 in order to provide information to a user as to whether the electronic pen 2000 is switched from a power-on state to a power-off state, whether the electronic pen 2000 is switched from a power-off state to a power-on state, and the like. As another example, the electronic pen 2000 may output a predetermined visible light through a display such as an LED lamp in order to provide information on an operation state of the electronic pen 2000 to a user.

The pattern detection unit 2300 is configured to detect a code pattern into which positional information for determining a position of the electronic pen 2000 with respect to the medium 1000 is encoded, and includes a camera module 2320 and an illumination module 2340.

The camera module 2320 is configured to acquire an image of the code pattern provided in the medium 1000. The camera module 2320 acquires images of partial areas of the code pattern provided in the medium 1000 according to a predetermined time interval (e.g., 1/240 to 1/60 seconds) while the electronic pen 2000 moves on the medium 1000 according to the user's writing. Accordingly, the electronic pen 2000 may interpret information about the unit cells included in the acquired image according to an interpretation method of a code pattern to be described below to interpret the position of the electronic pen 2000 with respect to the medium 1000 in the form of coordinates.

Illumination module 2340 emits light of a predetermined specific frequency toward medium 1000 so that an image of the code pattern can be acquired by camera module 2320. The code pattern provided in the medium 1000 according to an embodiment of the present invention may be printed using ink for absorbing infrared rays. In this case, the lighting module 2340 may emit light to the medium 1000 at a wavelength that may be well absorbed by the ink. In this case, the camera module 2320 may be configured to capture an image for an infrared band. Accordingly, the camera module 2320 may include an infrared filter for selectively passing only an infrared band.

The communication unit 2400 may communicate with an external device. For example, the electronic pen 2000 may communicate with the electronic device 3000 or the server 4000 through the communication unit 2400. Accordingly, the electronic pen 2000 may transmit or receive various information to or from an external device. For example, the electronic pen 2000 may transmit pen data generated by the electronic pen 2000 to the electronic device 3000 or the server 4000 through the communication unit 2400. Instructions and/or data for controlling the operation of the electronic pen 2000 or providing necessary information to the electronic pen 2000 may be received from the electronic device 3000 or the server 4000 through the communication unit 2400.

The communication unit 2400 may include a wired communication module 2420 and a wireless communication module 2440.

The wired communication module 2420 is a module for transmitting or receiving data to or from an external device in a wired manner, and may include elements such as a USB port, a PS/2 port, a parallel port, a serial port, etc., which may transmit or receive data to or from the external device, and also an ethernet port for accessing the internet through a Local Area Network (LAN).

The wireless communication module 2440 may include a mobile communication module accessing a mobile communication network via a mobile communication base station to transmit or receive data; a short range communication module using a Wireless Local Area Network (WLAN) communication scheme such as Wi-Fi or a Wireless Personal Area Network (WPAN) communication scheme such as bluetooth and Zigbee, or the like.

The sensor unit 2500 may sense various information. Here, the various information may be utilized to generate a control signal for controlling the operation of the electronic pen 2000, and may be used to generate pen data generated by the electronic pen 2000. Various information sensed by the sensor unit 2500 may be provided to the electronic device 3000 or the server 4000.

The sensor unit 2500 may include a pressure sensor 2520, a pen lead sensor 2540, a sleep sensor 2560, and the like.

Pressure sensor 2520 may sense pressure generated while the user is writing. For example, when a user writes notes using the electronic pen 2000, the electronic pen 2000 is in contact with the note surface, and the pressure generated at this point may be sensed by the pressure sensor 2520. In particular, the pressure sensor 2520 may be provided to the rear end of a pen lead (lead part 2010) installed in the electronic pen 2000 (i.e., the end opposite to the pen tip among both ends of the pen lead) to sense pressure. The electronic pen 2000 may use the pressure value sensed by the pressure sensor 2520 to generate pen data, which will be described in detail later. In this case, the pressure value included in the pen data may be processed as pen pressure information regarding pen pressure applied to the medium 1000 at the time of writing by the user and then may be used to reproduce a note written by the user. Alternatively, the electronic pen 2000 that has been in the closed state may use a signal output through the pressure sensor 2520 as a trigger signal for opening the electronic pen 2000.

The pen lead sensor 2540 may detect a pen lead identifier indicating the kind of pen lead (lead unit 2010) installed in the electronic pen 2000. For example, a pen lead identifier for indicating a color of the pen lead (lead element 2010) may be provided outside of the pen lead (lead element 2010) installed in the electronic pen 2000. In this case, the pen lead sensor 2540 may detect the pen lead identifier in various other available methods besides the optical method and the magnetic method. The value sensed by the pen lead sensor 2540 or information about the pen lead (lead element 2010) interpreted from the sensed value may be provided to the electronic device 3000. The electronic device 3000 may use the provided information to reproduce a note written by the user.

The sleep sensor 2560 may detect the state of the electronic pen 2000 to determine whether the electronic pen 2000, which has been in the on state, will automatically switch to the off state. For example, the sleep sensor 2560 may include an acceleration sensor or the like. Whether the electronic pen 2000 is moved may be detected by an acceleration sensor or the like. When it is determined that the electronic pen 2000 is not moved for a predetermined time based on the value sensed by the sleep sensor 2560, the electronic pen 2000 may be automatically switched to the off state.

According to some embodiments of the present invention, when the electronic pen 2000 is in an on state, power is normally supplied to elements included in the electronic pen 2000 and the elements may normally operate as needed, and when the electronic pen 2000 is in an off state, power to some or all of the elements included in the electronic pen 2000 is cut off and some or all of the elements included in the electronic pen 2000 may not normally operate. In particular, according to some embodiments of the present invention, although the electronic pen 2000 is in an off state, power may be normally supplied to the pressure sensor 2520 and/or an interpretation module for interpreting a signal output from the pressure sensor 2520, and the like. Accordingly, the electronic pen 2000, which has been in the off state, may be automatically switched to the on state based on the sensed value output from the pressure sensor 2520.

The memory 2600 may store various information. The memory 2600 may store data temporarily or semi-permanently. Operating programs, such as firmware for driving the electronic pen 2000, may be stored in the memory 2600. Information, instructions, and data required to drive the electronic pen 2000, such as pen data generated by the electronic pen 2000 and sensed values output by the sensor unit 2500, may be stored as needed.

Examples of the memory 2600 may include a Hard Disk Drive (HDD), a Solid State Drive (SSD), flash memory, Read Only Memory (ROM), Random Access Memory (RAM), and the like.

The power supply unit 2700 may supply power required to operate elements of the electronic pen 2000. The power supply unit 2700 may be implemented as a rechargeable battery. When the communication unit 2400 of the electronic pen 2000 includes a charging port (e.g., a USB port) and the electronic pen 2000 is connected to the electronic device 3000 through the charging port, power may be supplied from the electronic device 3000, and thus the battery may be charged.

The control unit 2800 controls the overall operation of the electronic pen 2000. For this reason, the control unit 2800 may calculate and process various information, and may control the operations of the elements of the electronic pen 2000.

Also, the control unit 2800 may analyze the image of the code pattern acquired by the pattern detection unit 2300 to decode information of the unit cells encoded into the image of the acquired code pattern. Further, the control unit may acquire position coordinates of the electronic pen 2000 with respect to the medium 1000 and identification information (e.g., book information or page information) of the medium 1000 from the decoded information.

The control unit 2800 may generate pen data based on information acquired by decoding information encoded into a unit cell. As described above, the generated pen data may be stored in the memory 2600 or may be transmitted to the electronic device 3000 and/or the server 4000 through the communication unit 2400. The generation of pen data will be described in detail later.

The control unit 2800 may be classified into a first control unit for controlling general operations of the electronic pen 2000 and a second control unit for analyzing an image acquired by the pattern detection unit 2300, decoding a unit cell having necessary information, and generating the above-described pen data. The control unit 2800 may be logically divided into a first control unit and a second control unit. However, the first control unit and the second control unit may be physically separated from each other.

The control unit 2800 may be implemented as a computer or the like in hardware, software, or a combination thereof. The control unit implemented in hardware may be provided in the form of an electronic circuit for processing an electric signal to perform a control function, and the control unit implemented in software may be provided in the form of a program for driving the control unit implemented in hardware.

In the following description, unless otherwise specified, the operation of the electronic pen 2000 may be construed as being performed by the control of the control unit 2800.

< electronic apparatus >

Subsequently, the electronic apparatus 3000 according to the embodiment of the present invention will be described.

Referring to fig. 5, the electronic device 3000 may include an input module 3100, an output module 3200, a communication module 3300, a memory module 3400, and a control module 3500. However, the elements shown in FIG. 5 are not required, and thus an electronic device may be implemented with more elements or fewer elements than shown in FIG. 5.

The elements of the electronic device 3000 will be described in turn.

The input module 3100 may receive user input from a user. The user input may include various forms such as key input, touch input, voice input, and various other inputs. As an example, the input module 3100 for receiving such user input is a broad concept including a touch sensor for detecting a touch of a user, a microphone for receiving a voice signal, a camera for recognizing a gesture through image recognition, a proximity sensor composed of an illumination sensor or an infrared sensor for detecting a proximity of the user, a motion sensor for recognizing a motion of the user through an acceleration sensor, a gyro sensor, and the like, and various input units for detecting or receiving various user inputs, as well as a typical keypad, a typical keyboard, and a typical mouse. Here, the touch sensor may be implemented as a resistive or capacitive touch sensor for detecting a touch through a touch panel or a touch film attached to a display panel, an optical touch sensor for optically detecting a touch, or the like.

The output module 3200 may output various information and provide the information to a user. The output module 3200 is a broad concept including a display for outputting visible light recognizable by the vision of a user, a speaker for outputting sound recognizable by the hearing of a user, a haptic device for generating vibration recognizable by the tactile sense of a user, and various other output units.

The display may output an image for reproducing a note written by a user using the electronic pen 2000 and/or the electronic device 3000 according to an embodiment of the present invention.

Also, the display is a broad concept of an image display device, including a Liquid Crystal Display (LCD), a Light Emitting Diode (LED) display, an Organic Light Emitting Diode (OLED) display, a Flat Panel Display (FPD), a transparent display, a curved display, a flexible display, a three-dimensional (3D) display, a holographic display, a projector, and various other devices for performing an image output function. The display may be a touch display that is integrally formed with the touch sensor of the input module 3100.

The communication module 3300 may communicate with an external device. For example, the electronic device 3000 may communicate with the electronic pen 2000 or the server 4000 through the communication module 3300. Accordingly, the electronic apparatus 3000 can transmit or receive various information to or from an external device. For example, through the communication module 3300, the electronic device 3000 may receive pen data or the like generated by the electronic pen 2000 and may transmit necessary information to the electronic pen 2000. Also, the electronic device 3000 may transmit pen data received from the electronic pen 2000 or stroke data acquired by interpreting the pen data to the server 4000 through the communication module 3300, and may receive necessary information from the server 4000.

The communication module 3300 may be configured as a wired communication module including a serial port, a parallel port, a PS/2 port, a USB port, and a Local Area Network (LAN) port for accessing the internet through a LAN, a mobile communication module for accessing a mobile communication network via a mobile communication base station to transmit or receive data, a short-range communication module using a Wireless Local Area Network (WLAN) communication method such as Wi-Fi or a Wireless Personal Area Network (WPAN) communication method such as bluetooth and Zigbee, a satellite communication module using a Global Navigation Satellite System (GNSS) such as Global Positioning System (GPS), or a combination thereof.

The memory module 3400 may store various information. The memory module 3400 may temporarily or semi-permanently store data. An Operating System (OS) or the like for driving the electronic device 3000 may be stored in the memory module 3400, and various data received from the electronic pen 2000, an application program for processing various data received from the electronic pen 2000, and the like may be stored in the memory module 3400 as necessary.

Examples of the memory module 3400 may include a Hard Disk Drive (HDD), a Solid State Drive (SSD), a flash memory, a Read Only Memory (ROM), a Random Access Memory (RAM), and the like.

The control module 3500 controls the overall operation of the electronic device 3000. For this, the control module 3500 may calculate and process various information, and may control operations of elements of the electronic device 3000.

The control module 3500 may be implemented as a computer, etc. in hardware, software, or a combination thereof. The control module implemented in hardware may be provided in the form of an electronic circuit for processing an electric signal to perform a control function, and the control unit implemented in software may be provided in the form of a program for driving the control module implemented in hardware.

In the following description, unless otherwise specified, the operation of the electronic device 3000 may be interpreted as being performed by the control of the control module 3500.

< operation of electronic Pen >

Subsequently, the operation of the electronic pen according to the embodiment of the present invention will be described.

A user may write notes on a medium 1000 according to some embodiments of the present invention using an electronic pen 2000 according to embodiments of the present invention.

Further, the electronic pen 2000 according to an embodiment of the present invention may acquire an image of at least a partial region of a code pattern included in the medium 1000 according to some embodiments of the present invention, and may interpret information encoded to a unit cell included in the acquired image to acquire position information. When the user writes notes while the electronic pen 2000 continuously acquires and processes an image of the code pattern and position information interpreted from the image and stores the processing result. As a result, the electronic pen 2000 may later generate basic data for interpreting a trace of a user writing a note on the medium 1000.

A code pattern into which position information and the like can be encoded according to an embodiment of the present invention will be described first, and then a method in which the electronic pen 2000 interprets the position information encoded in the code pattern using an image of the code pattern will be described.

However, since the electronic pen 2000 according to some embodiments of the present invention uses a commonly used pen lead (lead part 2010), the electronic pen 2000 does not necessarily have to be used to write notes only on the medium 1000 according to some embodiments, but may be used to write notes on a common medium (e.g., a piece of paper without a code pattern). However, in this case, since the electronic pen 2000 cannot acquire position information for explaining the user's writing trace, it is impossible to implement an operation of reproducing and reproducing the user's writing trace in real time by the electronic device 3000 or the like as described later.

1. Code pattern

The code pattern according to an embodiment of the present invention is configured by repeatedly arranging unit cells according to a desired two-dimensional size.

Referring to fig. 6 and 7, the unit cell includes a segmented code, and the segmented code may be mainly classified into a reference code and an information code.

The reference codes include horizontal reference codes, vertical reference codes, and cross-reference codes.

The reference code has a function of identifying an area of the unit cell when decoding the code pattern.

In a method to be described later, the information code performs a function of representing information intended to be encoded into the unit cell.

The reference code and the information code, except for the cross-reference code, have the same geometry. That is, the reference code other than the cross-reference code and the information code have a rotationally symmetric shape and a "segment shape".

The cross reference code has a geometrical cross shape unlike other reference codes and information codes.

The virtual line may be defined in a unit cell, and the virtual line is composed of a plurality of horizontal virtual lines arranged in a horizontal direction and a plurality of vertical virtual lines arranged in a vertical direction. In the description of some embodiments of the present invention, elements referred to as "virtual lines" may or may not actually be printed on the medium 1000.

The plurality of horizontal virtual lines are parallel to and spaced apart from each other by a certain distance.

Also, the plurality of vertical virtual lines are parallel to and spaced apart from each other by a certain distance.

Any one of the plurality of horizontal virtual lines is orthogonal to any one of the plurality of vertical virtual lines.

A plurality of intersections are formed by intersecting a plurality of horizontal virtual lines and vertical virtual lines. Each of the plurality of intersections is referred to as a virtual reference point. In the description of some embodiments of the present invention, elements referred to as "virtual points" may or may not actually be printed on the medium 1000.

Referring to fig. 6, one unit cell is shown to have a total of seven horizontal virtual lines and a total of seven vertical virtual lines. The horizontal reference codes are shown placed on the top horizontal virtual line, while the vertical reference codes are shown placed on the leftmost vertical virtual line.

The reference code is placed at a virtual reference point. In particular, the reference code is placed such that its centroid is located at the virtual reference point. That is, the horizontal reference codes and the vertical reference codes are placed such that the virtual reference points are located at the centers of both ends of each reference code. Furthermore, a cross-reference code having the shape of a geometric cross is placed such that the virtual reference point is located at the center of the cross.

The information code is also placed at the virtual reference point. In particular, unlike the reference code, the information code is placed such that its centroid is not located at the virtual reference point. That is, the virtual reference points are not placed at the centers of both ends of each information code. One end of each information code is placed near the virtual reference point, and the other end is placed at a position (different from the virtual reference point) determined in accordance with a value to be encoded into the information code. When a method of encoding information into an information code is described, it will be described in detail.

A code pattern having the shapes and elements already described above may be formed on the medium 1000. The actual formation of the code pattern on the medium 1000 may be implemented in a printing method or the like. In this case, when the code pattern is printed on the medium 1000, only the above-described reference code and information code may be printed, as shown in fig. 7. That is, among elements constituting the code pattern described above, the horizontal virtual line, the vertical dotted line, and the virtual reference point may not be actually printed.

2. Coding/decoding of information

The basic principle for encoding information into an information code is to place one end of the information code near a virtual reference point and to place the other end of the information code so that the information code passes through the virtual reference point. In this case, desired information is encoded into the information code by using the relationship between one end and the other end.

That is, desired information is encoded into the information code according to the direction of the information code (hereinafter also referred to as information code direction) from one end (end placed close to the virtual reference point) to the other end (end placed far from the virtual reference point) of the information code.

A method of encoding information into an information code will be described in detail. However, for convenience of description, an example in which information to be encoded into one information code is 2-bit information will be described.

For clarity of description, a virtual reference point where an information code into which information is to be encoded is to be placed is regarded as an origin, and a cartesian coordinate system defined according to directions of a horizontal reference code and a vertical reference code included in a unit cell is considered.

In this case, when the information intended to be encoded by the information code is "00", the direction of the information code may be determined such that one end (the end placed close to the virtual reference point) of the information code is placed in the third quadrant of the cartesian coordinate system and the other end (the end placed far from the virtual reference point) is placed in the first quadrant of the cartesian coordinate system (see (a) of fig. 8).

When information intended to be encoded by an information code is "01", the direction of the information code may be determined such that one end (an end placed close to a virtual reference point) of the information code is placed in a fourth quadrant of the cartesian coordinate system and the other end (an end placed far from the virtual reference point) is placed in a second quadrant of the cartesian coordinate system (see (b) of fig. 8).

Also, when the information intended to be encoded by the information code is "10", the direction of the information code may be determined such that one end (an end placed close to the virtual reference point) of the information code is placed in a first quadrant of the cartesian coordinate system and the other end (an end placed far from the virtual reference point) is placed in a third quadrant of the cartesian coordinate system (see (c) of fig. 8).

Also, when the information intended to be encoded by the information code is "11", the direction of the information code may be determined such that one end (an end placed close to the virtual reference point) of the information code is placed in the second quadrant of the cartesian coordinate system and the other end (an end placed far from the virtual reference point) is placed in the fourth quadrant of the cartesian coordinate system (see (d) of fig. 8).

When the information code is designed such that 2-bit information can be encoded into the information code (as shown in fig. 8) and 36 information codes are also included in one unit cell (as shown in fig. 6 and 7), information to be encoded into one unit cell may be 72 bits (═ 2 bits × 36).

However, the method of encoding information into an information code according to some embodiments of the present invention is performed in a direction from one end of the information code (e.g., an end placed close to a virtual reference point) to the other end of the information code (e.g., an end placed far from the virtual reference point). The number of bits of information to be encoded into the information code is not limited to two bits, and may be three or more bits. For example, when the direction of an information code is interpreted every 45 degrees in order to encode information into the information code, the total number of individual directions of the information code is eight, and thus 3 bits of information can be encoded into one information code.

3. Configuration of unit cell information

Subsequently, information encoded into the unit cell will be described.

The unit cell information encoded into the unit cell may include error verification information, medium identification information, coordinate information, and other additional information.

The error verification information is information for determining whether an error occurs when encoding is performed on the unit cell.

The medium identification information is identification information of the medium 1000 having the unit cell supplied thereto. The medium identification information may include book information, page information, and the like.

When each of the plurality of media 1000 is manufactured in the form of a notebook or a book, the book information refers to identification information assigned to the notebook or the book.

When each of the plurality of media 1000 is manufactured in the form of a notebook or a book, page information refers to information for identifying on which page of the notebook or the book the media 1000 having the unit cell provided thereto is located.

Further, when the medium 1000 is implemented to have a specific format (e.g., paper including a security check list, test paper, etc.) according to a specific use, the medium identification information may include information for identifying which format the medium 1000 has.

The coordinate information refers to two-dimensional coordinate information indicating where the unit cell is located on the medium 1000 having the unit cell supplied thereto. Thus, the coordinate information may include an x-coordinate and a y-coordinate.

The other additional information may include various other necessary information. For example, license information of the coordinate code provided to the medium 1000 may be included in other additional information.

As described above, the electronic pen 2000 may acquire the unit cell information by analyzing at least one unit cell included in the image acquired by the pattern detection unit 2300 and decoding information encoded into the unit cell, and thus may acquire the medium identification information, the coordinate information, and other additional information included in the unit cell information.

4. Generation of pen data

Subsequently, a method of generating pen data by the electronic pen 2000 will be briefly described based on the unit cell information.

The pen data refers to information for generating stroke data required to reproduce a user's writing trace in real time or to replay the user's writing trace later, in addition to the unit cell information acquired by the electronic pen 2000.

The pen data may include time information, pen pressure information, location information, media identification information, and the like.

The time information is information on a point of time at which an image for generating one piece of unit pen data is acquired or a point of time at which one piece of unit pen data is generated.

The pen pressure information is pressure information output from the pressure sensor 2520 at a point in time when an image for generating one piece of unit pen data is acquired or a point in time when one piece of unit pen data is generated.

The medium identification information is the same as or similar to information included in the unit cell information, and thus a detailed description thereof will be omitted.

The position information is information on the position of the electronic pen 2000 with respect to the medium 1000, which is generated by performing necessary correction processing on the coordinate information acquired from the unit cell information. The location information may or may not match the coordinate information acquired from the unit cell information, and thus a method of generating the location information will be described in detail below.

Fig. 9 illustratively shows an image I acquired by the camera module 2320 of the electronic pen 2000. As shown in fig. 9, at least one unit cell UC may be included in the acquired image I, and the position of the unit cell UC may be acquired in the form of coordinates by interpreting information encoded in the unit cell via the above-described method. In this case, as shown in the drawing, the coordinate information obtained by interpreting the unit cell UC may be somewhat different from the exact position of the electronic pen 2000 with respect to the medium 1000, because the unit cell UC for interpreting the coordinate information may not be located at the center of the acquired image I. That is, it may be assumed that the center IC of the acquired image I is closest to the position of the electronic pen 2000 with respect to the medium 1000. The center IC of the acquired image I may be different from the coordinates (x, y) acquired from the coordinate information acquired from the unit cell UC.

Therefore, in order to acquire the coordinates (x ', y') of the center IC of the acquired image I, the electronic pen 2000 calculates an offset value between the center IC of the acquired image I and the coordinate information acquired from the unit cell UC, adds or subtracts the calculated offset value (Δ x, Δ y) to or from the coordinate information (x, y) acquired from the unit cell UC, and calculates the coordinates (x ', y') of the center IC of the acquired image I.

The position of the electronic pen 2000 with respect to the medium 1000 may be more accurately calculated in the above-described method, and the coordinates (x ', y') corrected by the above-described method are the above-described position information.

As described above, the electronic pen 2000 may continuously generate pen data. Accordingly, the change in the position of the electronic pen 2000 with respect to the medium 1000 may be recorded in real time in the form of pen data according to the user's writing, and the pen data may be appropriately processed later by the electronic device 3000 or the server 4000 if necessary.

< operation of electronic apparatus >

As described above, when generating pen data, the electronic pen 2000 may store the pen data in the memory 2600 or may transmit the pen data to the electronic device 3000 through the communication unit 2400.

For example, when the electronic pen 2000 is enabled to communicate with the electronic device 3000, the electronic pen 2000 may generate pen data and transmit the generated pen data directly to the electronic device 3000. On the other hand, when the electronic pen 2000 is prohibited from communicating with the electronic device 3000, the electronic pen 2000 may generate pen data and store the generated pen data in the memory 2600. Then, when the electronic pen 2000 becomes capable of communicating with the electronic device 3000, the electronic pen 2000 may transmit the stored pen data to the electronic device 3000.

The operation of the electronic apparatus 3000 that has received pen data will be described below.

1. Real-time rendering of notes written by a user #1

The electronic device 3000 may reproduce notes being written by the user on the medium 1000 in real time and display the reproduced notes on a display of the electronic device 3000 based on pen data received from the electronic pen 2000. A method of reproducing a note being written by a user and displaying the reproduced note on a display in real time will be described below.

In the following description, it is assumed that the electronic pen 2000 and the electronic device 3000 are communicably connected to each other, and that the electronic pen 2000 transmits the generated pen data to the electronic device 3000 in real time, as described above.

The electronic device 3000 may acquire information about the target medium 1000 on which the user is writing notes based on the medium identification information included in the pen data received from the electronic pen 2000.

For example, the electronic device 3000 may check whether the target medium 1000 on which the user is writing notes is a medium having a specific format for a specific use or a medium constituting part of a product provided in the form of a book or notebook.

Further, when the target medium 1000 is a medium constituting a part of a product provided in the form of a book or notebook, the electronic apparatus 3000 may check which page of the book or notebook the target medium 1000 is located on.

The electronic device 3000 may display a pre-stored template of the object medium 1000 or a note written on the object medium 1000 by the user through the display based on the acquired medium identification information.

In this case, the pre-stored template of the target medium 1000 includes text, images, shapes, symbols, etc. printed in advance in the target medium 1000, and a code pattern of the target medium. For example, when the target medium 1000 constitutes a part of a notebook, a plurality of lines may be printed on the target medium 1000 to facilitate writing by a user, and information indicating "a plurality of lines are printed" may be included in the information on the pre-stored template of the target medium 1000. As another example, when the target medium 1000 constitutes a part of a notebook such as a diary, a specific date such as 2015 year 2 month 1 day may be printed on the target medium 1000, and information indicating that specific information is printed on the target medium 1000 may also be included in the predetermined template information. Hereinafter, for convenience of description, an image displayed on the target medium 1000 configured to include the above-described template information may be referred to as a medium image.

Subsequently, the electronic device 3000 may generate stroke data corresponding to a writing trajectory of the user to be displayed through the display based on the position information and the time information included in the received pen data. To generate stroke data, at least one piece of unit stroke data may be used.

When generating stroke data, electronic device 3000 may reproduce a trajectory of a user writing a note on target medium 1000 at a position corresponding to a predetermined template displayed on a display according to the generated stroke data. Fig. 10 shows an example of reproducing a note written by a user on the target medium 1000b on the display of the electronic device 3000 in real time.

When a note written by the user is reproduced through the electronic device 3000, a trajectory of the note written by the user, a position of the note written by the user with respect to the medium 1000, and the like may be accurately reproduced based on the position information included in the pen data information.

Hereinafter, for convenience of description, the operation of the electronic device 3000 according to an embodiment of the present invention may be referred to as an operation in a "normal reproduction mode" in which a note written by a user is reproduced in real time.

2. Real-time rendering of notes written by a user #2

Next, another embodiment of reproducing notes written by the user on the target medium 1000 in real time will be described.

When a user is using the electronic device 3000 according to an embodiment of the present invention, the user may desire that the entire surface of the target medium 1000 on which the user is currently writing notes be displayed by the electronic device 3000, or in some cases, the user may desire that only a portion of the surface of the target medium 1000 be displayed by the electronic device 3000.

Hereinafter, at least a partial region of the media image corresponding to the target medium 1000 displayed by the electronic device 3000 is referred to as a display region. Also, the partial region (or display region) of the target medium being displayed may refer to a partial region of the target medium in the entire region of the medium image displayed on the display as the display region.

Fig. 11 shows an example of displaying only a partial region of the target medium 1000 by the electronic device 3000.

In this case, when the user writes notes in another area other than the partial area displayed by the electronic device 3000, a real-time reproduction of the notes written by the user may not be displayed by the display, as described above by the electronic device 3000. For example, as shown in fig. 12, when the user further writes "EFG" on the right side of "ABCD" and the partial area set as shown in fig. 11 continues to be displayed by the electronic device 3000, a note corresponding to "E" additionally written by the user cannot be viewed through the electronic device 3000.

Another embodiment of reproducing notes written by a user in real time according to the present invention is proposed to solve the problem.

The electronic device 3000 can confirm the position of a note being written by the user on the target medium 1000 (hereinafter referred to as a writing position) based on the received pen data.

Subsequently, the electronic device 3000 may determine whether the confirmed writing position satisfies a predetermined condition for the current display area.

For example, electronic device 3000 may determine whether the writing position corresponds to a position outside the display area. In this case, when it is determined that the writing position is outside the display area, the electronic apparatus 3000 may determine that the predetermined condition is satisfied.

As another example, the electronic device 3000 may determine whether the confirmed writing position corresponds to a boundary area of the current display area. That is, even when the writing position is included in the current display area, the electronic device 3000 may determine whether the writing position corresponds to a boundary area of the display area. In this case, when determining that the writing position corresponds to the boundary area of the display area, the electronic apparatus 3000 may determine that the predetermined condition is satisfied.

When it is determined that the writing position does not satisfy the predetermined condition (e.g., when it is determined that the writing position is included in the display area), the electronic device 3000 may reproduce "E" corresponding to a note written by the user through the electronic device 3000, and may display the "E" on the display without moving the display area, as shown in fig. 12.

On the other hand, when it is determined that the writing position satisfies the predetermined condition (for example, when it is determined that the writing position is outside the display area), the electronic device 3000 may move the position of the display area to display a position corresponding to the writing position on the display of the electronic device 3000.

For example, when the user additionally writes "F" while writing "ABCDE", as shown in fig. 12, the electronic device 3000 may move the display area (as shown in fig. 13), and may reproduce the note written by the user in the moved display area.

Various methods may be applied as a method in which the electronic apparatus 3000 moves the position of the display area.

For example, the electronic device 3000 may determine a direction in which the position of the display area is to be moved in consideration of the writing direction of a note that has been written by the user.

Fig. 13 shows an example in which the position of the display area is moved in the same direction as the writing direction of "ABCDE", which corresponds to a note that has been written by the user.

As another example, the electronic device 3000 may move the position of the display area such that a note written outside the original display area is displayed in the center of the moved display area.

Fig. 14 shows an example in which the display area is moved so that "F" written outside the position of the original display area can be placed at the center of the moved display area.

As yet another example, the electronic device 3000 may move the position of the display area in consideration of the distance and direction between a note that has been previously written and a note that was written outside the original display area.

As yet another example, the electronic device 3000 may consider an edge of the media image when moving the position of the display area. For example, the position of the display area may be moved so that the edge of the media image is not outside the predetermined area of the display screen. That is, the position of the display area may be moved so that the edge of the media image is not outside the border area of the display screen. In other words, the position of the display area may be moved so that the edge of the medium image is not displayed to be spaced apart from the boundary area of the display screen by a predetermined distance or more.

In order to determine whether the writing position satisfies a predetermined condition, the electronic device 3000 considers pen data received from the electronic pen 2000. In this case, various methods may be applied as a method in which the electronic device 3000 samples pen data for recognizing a writing position.

For example, the electronic device 3000 may determine whether the writing position for each piece of unit pen data satisfies a predetermined condition with respect to the partial area (e.g., whether the writing position is outside the display area). When the electronic device 3000 operates in this manner, the electronic device 3000 can reproduce the user's writing most similarly to real-time reproduction through the electronic device 3000, but the amount of calculation increases.

As another example, the electronic device 3000 may determine whether the writing position of at least part of the pen data, which may be grouped in stroke units, satisfies a predetermined condition for the display area.

A stroke may refer to a line drawn by a pen on a writing surface until the pen contacts the writing surface and then separates from the writing surface. That is, "A" generally consists of three strokes, "B" generally consists of two strokes, "C" generally consists of one stroke, "D" generally consists of two strokes, and "E" generally consists of three strokes. For convenience of description, an operation in which the user brings the electronic pen 2000 into contact with the target medium 1000 in order to write one stroke is referred to as "pen-down", and an operation in which the user separates the electronic pen 2000 from the target medium 1000 after writing one stroke is referred to as "pen-up".

In this case, since the electronic pen 2000 may generate pen data including the pen pressure sensed by the pressure sensor and transmit the generated pen data to the electronic device 3000, the electronic device 3000 may determine "pen down" or "pen up" by using a value of the pen pressure included in the pen data. That is, when pen data is received, the electronic device 3000 may determine that "pen down" occurs at a point in time when the value of pen pressure suddenly increases from zero to a specified value. For convenience, this point in time is defined as "pen down time". Also, when pen data is received, the electronic device 3000 may determine that "pen up" occurs at a point in time when the value of pen pressure suddenly becomes zero. For convenience, this point in time is defined as "pen-up time".

The electronic device 3000 may determine whether the writing position satisfies a predetermined condition for the display area by using position information included in pen data corresponding to the pen-down time among pen data grouped in stroke units (hereinafter, this is referred to as a pen-down-based movement manner). In this case, in order to determine whether the writing position satisfies the predetermined condition for the display area, the electronic device 3000 may use only the writing position of the pen down time without using the position information included in the remaining pen data among the plurality of pen data groups grouped in stroke units.

The electronic device 3000 may determine whether the writing position satisfies a predetermined condition for the display area by using position information included in pen data corresponding to the pen-up time among pen data grouped in stroke units (hereinafter, this is referred to as a pen-up-based movement manner). In this case, in order to determine whether the writing position satisfies the predetermined condition for the display area, the electronic device 3000 may use only the writing position of the pen-up time without using the position information included in the remaining pen data in the plurality of pen data groups grouped in stroke units.

Alternatively, the electronic device 3000 may determine whether the writing position satisfies the predetermined condition for the display area by using all position information included in the pen data corresponding to the pen-up time and the pen-down time among the pen data grouped in stroke units (hereinafter, this is referred to as a hybrid moving manner). In this case, the electronic apparatus 3000 may use only writing positions of the pen-down time and the pen-up time without using other pen data.

For a pen down based movement pattern, the display area can only be moved at pen down time. Thus, when the start point of a stroke input by the user is included in the original display area and the end point of the stroke is excluded from the original display area, the movement according to the present invention does not occur. That is, as shown in fig. 15, even if the end point EP of the stroke S is excluded from the original display area, since the start point SP of the stroke S is included in the original display area, even after the user completes the stroke S, the screen as shown in (b) of fig. 15 may be displayed through the display of the electronic device 3000 even at the pen-up time Tup.

For the pen-up based movement mode, the display area can be moved only at the pen-up time. Thus, when the user inputs one stroke for a long time (e.g., when one stroke is input for about five seconds), the electronic device 3000 may not perform the moving operation until the user's stroke input is completed.

For the hybrid movement mode, both advantages of the movement mode based on pen-down and the movement mode based on pen-up can be achieved.

The method of determining a writing position using a portion of a plurality of pieces of pen data that may be grouped in stroke units and performing a moving operation according to the determination may include a method of selectively using pen data at a pen-down time, pen data at a pen-up time, and pen data at the pen-down time and the pen-up time, and various other methods. For example, there may be a method of determining a writing position using pen data selected at predetermined time intervals, and the like. That is, there may be a method of determining a writing position by selectively using pen data at pen-on time, pen data at pen-off time 0.05 seconds, pen data at pen-off time 0.10 seconds, pen data at pen-off time 0.15 seconds, and the like (when the predetermined time interval is 0.05 seconds).

The electronic device 3000 may determine whether at least one move operation is performed on one stroke. That is, the electronic device 3000 may determine whether the predetermined condition is satisfied using at least one of the plurality of pieces of pen data included in each stroke. In other words, the electronic device 3000 may determine whether the predetermined condition is satisfied using at least one of pen data (first pen data) corresponding to a pen-down time, pen data (second pen data) corresponding to a pen-up time, and all pen data located between the first pen data and the second pen data, which constitute one stroke.

As described above, according to another embodiment in which notes written by the user on the target medium 1000 are reproduced in real time, the screen of the electronic apparatus 3000 is enlarged. Thus, even when the display area of the target medium 1000 is displayed only by the display, the note written by the user can be reproduced and displayed in real time by the display of the electronic apparatus 3000.

Hereinafter, for convenience of description, the operation of the electronic device 3000 according to another embodiment of the present invention may be referred to as an operation in an "enlarged reproduction mode" in which notes written by a user are reproduced in real time.

3. Real-time rendering of notes written by a user #3

Hereinafter, for convenience of description, a mode in which only a partial region of the target medium 1000 is displayed by the electronic device 3000 may be referred to as an "enlargement mode", and a mode in which the entire region of the target medium 1000 is displayed by the electronic device 3000 may be referred to as a "normal mode".

In order to reproduce notes written by the user in real time according to the present invention, the electronic apparatus 3000 may operate in the above-described "normal reproduction mode" or "enlarged reproduction mode".

For example, the electronic device 3000 may determine whether a current display mode of the electronic device 3000 is a "normal mode" or an "enlargement mode", and may select a mode for reproducing notes written by a user in real time and operate in the selected mode according to a result of the determination. That is, when the display mode of the electronic apparatus 3000 is the "normal mode", the electronic apparatus 3000 may reproduce the note written by the user in real time according to the above-described "normal reproduction mode", and when the display mode of the electronic apparatus 3000 is the "enlargement mode", the electronic apparatus 3000 may reproduce the note written by the user in real time according to the above-described "enlargement reproduction mode".

In this way, when the reproduction mode is selectively set and the electronic apparatus 3000 operates in the set reproduction mode, it is possible to apply the "enlarged reproduction mode" having a relatively large calculation amount only when the screen is enlarged and displayed, and operate in the "normal reproduction mode" having a relatively small calculation amount without the screen being enlarged and displayed, thereby effectively and efficiently reproducing the note written by the user in real time.

4. Playback of written notes #1

The electronic device 3000 may later play back the note that the user is writing on the medium 1000 and display the played back note on a display of the electronic device 3000 based on pen data received from the electronic pen 2000. A method of playing back and displaying the note being written by the user on the display in real time will be described below.

In the following description, it is assumed that the electronic pen 2000 and the electronic device 3000 receive and store pen data regarding a note written by a user on the target medium 1000 using the electronic pen 2000.

The electronic device 3000 may detect whether a predetermined event for replaying a note written by a user has occurred.

For example, the predetermined event may be a request from a user. That is, the electronic device 3000 may provide a User Interface (UI) for requesting playback of a note written by the user through the electronic device 3000. In this case, the user may request the written note to be played back through the UI, and thus the electronic device 3000 may determine that a predetermined event has occurred and may perform a playback operation for the written note according to the present invention.

As another example, the predetermined event may be a request from an Operating System (OS) of the electronic device 3000 or another application installed in the electronic device 3000.

As yet another example, the predetermined event may coincide with a predetermined condition. That is, when a predetermined condition of "reaching a specific time" is set, the electronic device 3000 may monitor the current time, determine whether the current time reaches the specific time of the predetermined condition, determine whether a predetermined event has occurred when the current time reaches the specific time, and perform a playback operation on the user's writing according to the present invention. The predetermined condition is not limited to the "time" condition as described above, and may be set in various forms.

When a predetermined event has occurred, the electronic device 3000 may determine a target note to be subjected to a playback operation.

For example, the electronic device 3000 may have all pen data stored therein regarding notes written by the user on the medium 1000 using the electronic pen 2000, and may determine a target note to be subjected to a playback operation from among the written notes. That is, when the target medium 1000 is provided in the form of a notebook, the electronic device 3000 may have entire notes written by the user on a plurality of pages of the notebook stored in the form of pen data (or stroke data). In this case, the electronic device 3000 may determine which of the plurality of pages (one or more pages) to perform the playback operation, or may determine which specific region of the same page to perform the playback operation.

The electronic device 3000 may consider a predetermined event to determine a playback target.

For example, when the predetermined event is a request from a user, the electronic device 3000 may determine the playback target according to the user's request. As another example, when the predetermined event is a request from an operating system or another application, the electronic device 3000 may determine the playback target in consideration of the request.

When the playback target is determined, the playback section may be determined. For example, when the electronic device 3000 determines a specific page of the notebook as a playback target as the target section, a plurality of strokes written in the determined specific page are determined as playback targets, and the playback section is defined by the plurality of playback target strokes and the temporal relationship between the playback target strokes. The playback section will be described in detail below.

Fig. 16 is a diagram illustrating a playback section according to the present invention. Referring to fig. 16, the playback section RS according to the present invention includes a stroke section SS and a gap section GS. In particular, fig. 16 shows that the playback section RS includes first to fourth stroke sections SS1, SS2, SS3, and SS4, and first to third sections GS1, GS2, and GS 3.

In the present invention, for convenience of description, a playback section is defined as a period of time required for a user to write a playback target.

That is, the replay section is defined as a time section from a point of time when the user starts writing the first stroke of the target note requested to be replayed to a point of time when the user finishes writing the last stroke of the target note. For example, as shown in fig. 16, when the playback section is determined, the length of the playback section RS is t8-t 1.

A stroke section is defined as the period of time required for a user to enter a stroke while the user is writing a note. That is, as described above, the stroke section is defined as a period from the pen-down time of the target medium 1000 to the pen-up time of the target medium 1000 in order to input one stroke. For example, as shown in FIG. 16, first stroke section SS1 has a length t2-t1, second stroke section SS2 has a length t4-t3, third stroke section SS3 has a length t6-t5, and fourth stroke section SS4 has a length t8-t 7.

The gap section GS is defined as a period of time from the time when one stroke is input to the time when another stroke after the input stroke is input. That is, the time from the pen-up time of the input stroke to the pen-down time of the input subsequent stroke is defined as the gap section GS. For example, as shown in fig. 16, the first gap section GS1 has a length of t3-t2, the second gap section GS2 has a length of t5-t6, and the third gap section GS3 has a length of t7-t 6.

It has been described that the gap section GS is defined by the relationship between two strokes. In this case, the two strokes used to define the gap section are determined only by the relationship between the strokes included in the note determined as the playback target, without considering the strokes not included in the playback target. For example, assume that one stroke (stroke a) is recorded in a first area (e.g., page x of notebook a), the next stroke (stroke B) is recorded in a second area (e.g., page y of notebook a), and then the next stroke (stroke C) is recorded in the first area. In this case, when only page x is selected as the playback target and page y is not selected, stroke B is not taken into consideration to define the above gap section GS because stroke B is not included in the playback target. That is, in this case, the stroke section defined by the stroke B does not exist in the playback section. Also, a gap section between stroke a and stroke B and a gap section between stroke B and stroke C are not defined, and only a gap section between stroke a and stroke C may be defined.

Subsequently, the electronic device 3000 may acquire pen data about the strokes included in the playback target from the memory module 3400, and may generate playback data based on the acquired pen data. The replay data is data generated in order to perform a replay operation for replaying a note written by a user according to the present invention, and may be data generated by sampling pen data regarding strokes. Also, the playback data may be interpreted as a very comprehensive meaning including the position of the strokes displayed on the display when performing the playback operation, the order in which the strokes are displayed, timing information for displaying the strokes, and the like.

Subsequently, the electronic device 3000 performs a playback operation according to the present invention based on the playback data.

The playback operation will be described in detail below.

In the playback operation for a note written by a user according to an embodiment of the present invention, the reproduction time for playback is different from the time required for the user to write a playback target.

For example, a specific page may be selected as a playback target, a part of notes included in the specific page selected as the playback target may be written at 10 am, and another part included in the specific page may be written at 8 pm. In this case, the time required for the user to write the replay target is 10 hours, but according to an embodiment of the present invention, the reproduction time required when the written note is replayed may be several minutes. That is, the playback reproduction time for the playback section may be shorter than the writing time of the playback section (the length of the playback section).

As another example, a specific word may be selected as a playback target, and the actual time required for a user to write the specific word selected as the playback target may be two seconds, but the playback reproduction time of the word may be two seconds or more. That is, the playback reproduction time for the playback section may be longer than the writing time of the playback section (the length of the playback section).

(1) Determination of playback reproduction time for stroke sections

The electronic device 3000 may determine a playback reproduction time for strokes included in the playback target.

< relationship between playback reproduction time and Stroke section Length >

According to some embodiments of the present invention, the electronic device 3000 may determine a playback reproduction time for playing back the target stroke in consideration of the actual time required to write the stroke.

Fig. 17 is a diagram illustrating a playback reproduction time for the playback section RS shown in fig. 16, which is determined according to an embodiment of the present invention.

Fig. 17 illustrates that the first to fourth stroke playback reproduction times SR1, SR2, SR3, and SR4 of the first to fourth stroke sections SS1, SS2, SS3, and SS4 are determined in proportion to the lengths of the first to fourth stroke sections SS1, SS2, SS3, and SS 4. In this case, the electronic device 3000 may determine the playback reproduction time for playing back the target stroke in consideration of the actual time required to write the playback target stroke.

FIG. 17 illustrates an example of determining a playback reproduction time for a stroke in proportion to the length of the stroke section. However, the playback reproduction time for the strokes does not necessarily have to be determined to scale and may be determined on a segment basis. For example, the playback reproduction time for a stroke may be determined by corresponding the actual time required to write each stroke to each segment. In this case, different playback reproduction times correspond to different sections. That is, when two strokes having slightly different stroke section lengths are classified into the same section, the playback reproduction times for the two strokes may be determined to be equal to each other. Also, FIG. 17 shows an example of determining a playback reproduction time for a stroke considering only the length of the stroke section. However, the playback reproduction time may be determined in consideration of the length of the stroke and the length of the stroke section. A method of determining a playback reproduction time considering the length of strokes will be described below.

Unlike this, the electronic device 3000 may determine the same playback reproduction time for the playback target strokes.

For example, the electronic device 3000 may determine a playback reproduction time for the playback target stroke without any consideration of the actual time required to write the stroke.

In this case, the electronic device 3000 may determine the predetermined playback reproduction time as the playback reproduction time for playing back the target stroke, regardless of the actual time required to write the playback target stroke. That is, the playback reproduction times determined for all strokes included in the playback target may all be the same.

Fig. 18 and 19 are diagrams showing playback reproduction times for the playback section RS shown in fig. 16, which are determined according to another embodiment of the present invention.

Fig. 18 shows that the first to fourth stroke playback reproduction times SR1, SR2, SR3, and SR4 of the first to fourth stroke sections SS1, SS2, SS3, and SS4 are determined to be equal to each other. In this case, the electronic device 3000 may determine the playback reproduction time for the playback target stroke without considering the actual time required to write the playback target stroke at all.

However, it has been described with reference to fig. 18 that, when determining the playback reproduction time, the playback reproduction times determined for the strokes are equal to each other regardless of the length of the stroke section. In this case, the playback reproduction time may be determined in consideration of the length of the stroke. When the playback reproduction time is determined in consideration of the length of the strokes, the playback reproduction times for the strokes may be different from each other. A method of determining a playback reproduction time considering the length of strokes will be described below.

However, fig. 18 shows that the same playback reproduction time is one duration. However, the playback reproduction time may be determined as one time point.

For example, as shown in fig. 19, when the strokes of the first to fourth stroke sections SS1, SS2, SS3, and SS4 are played back, the electronic device 3000 may display the stroke corresponding to the first stroke section SS1 at one stroke at a first time point, display the stroke corresponding to the second stroke section SS2 at one stroke at a second time point, display the stroke corresponding to the third stroke section SS3 at one stroke at a third time point, and display the stroke corresponding to the fourth stroke section SS4 at one stroke at a fourth time point. Here, it is understood that displaying a stroke at once means that all points constituting the stroke are displayed at once at the corresponding time.

That is, fig. 18 relates to an embodiment in which the points constituting each stroke are sequentially displayed when the determined replay is performed, and fig. 19 relates to an embodiment in which all the points constituting each stroke are immediately displayed on the display.

The relationship between the length of the stroke section and the playback reproduction time determined by the length has been described above. The relationship between the length of the stroke (the actual physical length of the stroke written by the user) and the playback reproduction time determined by the length will be described below.

< relationship between playback reproduction time and Stroke Length >

According to some embodiments of the present invention, the electronic device 3000 may determine a playback reproduction time for playing back the target stroke in consideration of the length of the stroke.

20-24 are diagrams that illustrate examples of determining a playback reproduction time for playing back strokes written by a user, according to some embodiments of the invention.

Fig. 20 shows that the first to fourth strokes S1, S2, S3, and S4 are written on the target medium 1000.

For convenience of description, it is assumed that the first stroke S1 is written by the user during the first stroke section SS1 shown in fig. 16, the second stroke S2 is written by the user during the second stroke section SS2 shown in fig. 16, the third stroke S3 is written by the user during the third stroke section SS3 shown in fig. 16, and the fourth stroke S4 is written by the user during the fourth stroke section SS4 shown in fig. 16.

For convenience of description, it is also assumed that the lengths of the first, second, third and fourth strokes S1, S2, S3 and S4 are SL1, SL2, SL3 and SL4, which have the relationship SL4> SL1> SL3> SL2, similar to that shown in fig. 20.

Here, it is understood that the length of a stroke refers to the physical length of the stroke that the user actually writes on target medium 1000. However, in this case, the length may be understood as an absolute length or a relative length between strokes.

In this case, the electronic device 3000 may determine the playback reproduction time for the stroke included in the playback target in consideration of the length of the stroke.

FIG. 21 illustrates playback reproduction times for strokes S1, S2, S3, and S4 included in the playback target illustrated in FIG. 20, which are determined according to some embodiments of the invention.

Fig. 21 illustrates that the first to fourth stroke playback times SR1, SR2, SR3, and SR4 of the first to fourth strokes S1, S2, S3, and S4 are determined in proportion to the lengths of the first to fourth strokes S1, S2, S3, and S4. In this case, the electronic device 3000 may determine the playback reproduction time for the playback target stroke in consideration of the actual time required to write the playback target stroke.

FIG. 21 illustrates an example of determining a playback reproduction time for a stroke in proportion to the length of the stroke. However, the playback reproduction time for the strokes does not necessarily have to be determined to scale and may be determined on a segment basis. For example, a playback reproduction time for a stroke may be determined by corresponding the length of each stroke to each segment. In this case, different playback reproduction times correspond to different sections. That is, when two strokes having slightly different lengths are classified into the same section, the playback reproduction times for the two strokes may be determined to be equal to each other. Also, fig. 21 shows an example in which the playback reproduction time of a stroke is determined in consideration of only the length of the stroke. However, the playback reproduction time may be determined in consideration of the length of the stroke section (the actual time required to write the stroke) and the length of the stroke.

Unlike this, the electronic device 3000 may determine the same playback reproduction time for the playback target stroke regardless of the length of the stroke. That is, the electronic device 3000 may determine a playback reproduction time for the playback target stroke, without considering the length of the stroke at all.

Fig. 22 and 23 are diagrams illustrating playback reproduction times for the playback target strokes illustrated in fig. 20, which are determined according to another embodiment of the present invention.

Fig. 22 shows that the first to fourth stroke playback reproduction times SR1, SR2, SR3, and SR4 of the first to fourth strokes S1, S2, S3, and S4 are determined to be equal to each other. In this case, the electronic device 3000 may determine the playback reproduction time for the playback target stroke, without considering the length of the playback target stroke at all.

However, fig. 22 shows that the same playback reproduction time is one duration. However, the playback reproduction time may be determined as one time point.

For example, as shown in fig. 23, when the first to fourth strokes S1, S2, S3, and S4 are played back, the electronic device 3000 may display the playback corresponding to the first stroke S1 at once at a first time point, display the playback corresponding to the second stroke S2 at once at a second time point, display the playback corresponding to the third stroke S3 at once at a third time point, and display the playback corresponding to the fourth stroke S4 at once at a fourth time point.

(2) Determination of playback reproduction time of gap section

The electronic device 3000 may determine a playback reproduction time for a gap section existing between strokes included in the playback target.

According to some embodiments of the present invention, the electronic device 3000 may determine the playback reproduction time of the gap segment taking into account the actual time interval between strokes when the user actually writes the playback target strokes. According to some embodiments, the method of determining a playback reproduction time for a gap section is almost similar to the above-described method of determining a playback reproduction time for a stroke section in consideration of the length of the stroke section or in consideration of the length of the stroke section, and thus will be briefly described.

This will be described with reference to fig. 16 to 19.

Fig. 17 shows that the first to third gap section playback reproduction times GR1, GR2, and GR3 of the first to third gap sections GS1, GS2, and GS3 are determined in proportion to the lengths of the first to third gap sections GS1, GS2, and GS 3. In this case, when the playback target is reproduced, the electronic device 3000 may determine the playback reproduction time for the gap section between the strokes in consideration of the time difference between the strokes occurring in the actual writing.

However, the playback reproduction time determined for the gap section does not necessarily have to be determined in proportion to the length of the gap section, and may be determined on a section basis.

Unlike this, the electronic device 3000 may determine the same playback reproduction time for the playback target gap section, as shown in fig. 18. For example, the electronic device 3000 may determine a playback reproduction time for the playback target stroke without any consideration of the actual time required to write the stroke.

(3) Determination of playback reproduction speed and playback reproduction time based on playback reproduction speed

According to some embodiments of the present invention, a playback reproduction speed for the user's writing may be determined, and according to some embodiments of the present invention, the electronic device 3000 may perform a playback operation on the user's writing according to the determined playback reproduction speed.

The electronic device 3000 may determine the playback reproduction speed in consideration of the number of strokes included in the selected playback section.

For example, electronic device 3000 may have first and second predetermined conditions stored therein, and may confirm the number of strokes included in the selected playback section. The electronic device 3000 may determine a first playback reproduction speed corresponding to the first predetermined condition as the playback reproduction speed for the selected playback section when the confirmed number of strokes is equal to or less than the first predetermined condition, or may determine a second playback reproduction speed corresponding to the second predetermined condition as the playback reproduction speed for the selected playback section when the confirmed number of strokes is greater than the first predetermined condition and equal to or less than the second predetermined condition. The electronic device 3000 may have more predetermined conditions stored therein in addition to the first predetermined condition and the second predetermined condition. The predetermined playback reproduction speed may correspond to a predetermined condition.

When the first predetermined condition is less than the second predetermined condition, a first playback reproduction speed corresponding to the first predetermined condition may be slower than a second playback reproduction speed corresponding to the second predetermined condition. For example, when the first predetermined condition is 100 and the second predetermined condition is 200, the first predetermined playback reproduction speed may be slower than the second predetermined playback reproduction speed.

The electronic device 3000 may determine the playback reproduction time for the stroke section and the playback reproduction time for the gap section in consideration of the determined playback reproduction speed.

When the first playback reproduction speed is faster than the second playback reproduction speed, the stroke section playback reproduction time determined when the determined playback reproduction speed is the first playback reproduction speed may be shorter than the stroke section playback reproduction time determined when the determined playback reproduction speed is the second playback reproduction speed.

Similarly, when the first playback reproduction speed is faster than the second playback reproduction speed, the gap section playback reproduction time determined when the determined playback reproduction speed is the first playback reproduction speed may be shorter than the gap section playback reproduction time determined when the determined playback reproduction speed is the second playback reproduction speed.

The process of determining the stroke section playback reproduction time based on the reproduction speed and the process of determining the gap section playback reproduction time based on the reproduction speed operate independently of each other. When the reproduction speed increases, one or both of the stroke section reproduction time and the gap section reproduction time may be determined to be shorter than before.

(4) Determination of the total playback reproduction time, and playback operation

Subsequently, the electronic device 3000 may determine a reproduction time for the strokes included in the selected playback target and for the gap sections between the strokes based on at least one of the length of the stroke sections, the length of the gap sections, the length of the strokes, and the reproduction speed, which have been described above, to determine an entire reproduction time for the selected playback target.

Subsequently, the electronic device 3000 performs a playback operation on the selected playback target according to the determined playback reproduction time.

Fig. 24 is a diagram illustrating a screen on which a note written by a user is played back through a display according to an embodiment of the present invention.

The electronic device 3000 may sequentially replay strokes based on replay reproduction times determined for stroke sections and gap sections determined from notes written by the user according to the above description, as shown in fig. 20.

When the playback reproduction time for the stroke section and the playback reproduction time for the gap section are determined to be "duration" as described above with reference to fig. 21 or 22, the electronic device 3000 may sequentially display the points constituting the first stroke S1 during the determined first stroke reproduction time SR 1. As a result, the screen shown in (a) of fig. 24 can be displayed by the display of the electronic apparatus 3000. Subsequently, the electronic device 3000 may maintain the screen shown in (a) of fig. 24 during the determined first gap section playback time GR1 and sequentially display the points constituting the second stroke S2 during the determined second stroke reproduction time SR 2. As a result, the screen shown in (b) of fig. 24 can be displayed by the display of the electronic apparatus 3000. Similarly, a playback operation may be performed on the third and fourth strokes.

As described with reference to fig. 23, when the playback reproduction time for the stroke section is determined as the "point in time", the electronic device 3000 may display the screen shown in (a) of fig. 24 at a first point in time through the display, then maintain the screen shown in (a) of fig. 24 during the determined first gap section playback reproduction time GR1, and may display the screen shown in (b) of fig. 24 at a second point in time through the display, then maintain the screen shown in (b) of fig. 24 during the determined second gap section playback reproduction time GR 2. Similarly, a playback operation may be performed on the third and fourth strokes.

The playback operation for the user's writing according to some embodiments of the present invention has been described above. In the above-described embodiment, fig. 17 shows that when the stroke playback reproduction time is determined in proportion to the length of the stroke section at the time of determining the playback reproduction time of the stroke section, the gap section playback reproduction time is also determined in proportion to the length of the gap section, and fig. 18 shows that when the stroke playback reproduction times are determined to be equal to each other regardless of the length of the stroke section, the gap section playback reproduction times are also determined to be equal to each other regardless of the length of the gap section. However, the operation of determining the playback reproduction time for the stroke section and the operation of determining the playback reproduction time for the gap section are independent of each other, and are not necessarily limited to the operations shown in fig. 17 and 18. Even when the playback reproduction time for the stroke section is determined depending on the length of the stroke section, the playback reproduction time for the gap section can be determined regardless of the length of the gap section, and vice versa.

5. Playback of written notes #2

The electronic device 3000 may later play back the note being written by the user on the medium 1000 and display the played back note on a display of the electronic device 3000 based on pen data received from the electronic pen 2000. Another method of replaying and displaying notes being written by a user on a display in real time will be described below.

The electronic device 3000 may detect whether a predetermined event for replaying a note written by a user occurs. The predetermined event has been described above, and thus a detailed description thereof will be omitted.

Subsequently, the electronic device 3000 can determine a playback target intended for playback from the note written by the user. The playback target and the determination of the playback target have been described above, and thus a detailed description thereof will be omitted.

The electronic device 3000 may determine the reproduction time of the playback operation after the playback target is determined. In this case, the electronic device 3000 may confirm the number of strokes included in the playback target, and may determine the reproduction time of the playback operation based on the confirmed number of strokes.

The electronic device 3000 may have a number of strokes to be reproduced per unit time stored therein in advance so as to determine a reproduction time of a playback operation based on the confirmed number of strokes. For example, the electronic device 3000 may have four strokes to be played back and reproduced within 1 second previously stored therein. As another example, the electronic device 3000 may have two strokes to be played back and reproduced within 1 second pre-stored therein. As another example, the electronic device 3000 may have eight strokes to be played back and reproduced within 1 second pre-stored therein.

The electronic device 3000 may determine the reproduction time of the playback operation based on the confirmed number of strokes and the number of strokes to be played back and reproduced per unit time. For example, when the total of 100 strokes are included in the note determined as the playback target and the number of strokes to be played back and reproduced within 1 second is four, the reproduction time of the playback operation may be determined to be 25(═ 100/4) seconds.

That is, the reproduction time of the playback operation determined by the electronic device 3000 may be determined in proportion to the number of strokes included in the playback target. For example, when the number of strokes is 100 and the predetermined number of strokes to be played back per second is 4, the reproduction time may be determined to be 25 seconds. However, when the stroke number is determined to be 200, the reproduction time may be determined to be 50 seconds.

In other words, according to some embodiments of the present invention, the number of strokes included in writing of a user to be performed a playback operation may be defined as N, and the predetermined number of strokes reproduced in a predetermined unit time may be defined as M. In this case, the reproduction time of the playback operation determined by the electronic device 3000 may be determined to be N/M (predetermined unit time).

In the above description, only the example in which the number of strokes included in the playback target is N times the number of strokes played back per predetermined unit time is described. However, an embodiment of determining the playback reproduction time when the number of strokes is not a multiple of the number of strokes played back per predetermined unit time will be described below.

According to some embodiments, the electronic device 3000 may determine the playback reproduction time by the same method as described above. That is, when the number of strokes included in the playback target is 102 and the number of strokes played back per predetermined unit time is 4, the playback reproduction time may be determined to be 25.5(═ 102/4) seconds.

According to other embodiments, when the number of strokes included in the writing of the user to be performed with the playback operation is defined as N, the predetermined number of strokes reproduced for 1 second is defined as M, a quotient obtained by dividing N by M is defined as Q, and a remainder obtained by dividing N by M is defined as R, the electronic device 300 may determine the playback reproduction time as Q +1 second. That is, when the number of strokes included in the playback target is 102 and the number of strokes played back per predetermined unit time is 4, the quotient obtained by dividing 102 by 4 is 25. Therefore, the playback reproduction time can be determined to be 26(═ 25+1) seconds.

In summary, it is assumed that the number of strokes included in writing of a user to be subjected to a playback operation is defined as N, a predetermined number of strokes reproduced for 1 second is defined as M, a quotient obtained by dividing N by M is defined as Q, and a remainder obtained by dividing N by M is defined as R. In this case, the electronic device 3000 may determine the playback reproduction time as Q when R is zero, and may determine the playback reproduction time as Q +1 when R is not zero (i.e., R is an integer less than M).

However, in this case, the reproduction time of the playback may be determined in proportion to the number of strokes, but cannot be determined in full proportion. That is, the playback reproduction time when the number of strokes is 100 and the playback reproduction time when the number of strokes is 200 may have a difference of two times. However, the playback reproduction time when the number of strokes is 101, the playback reproduction time when the number of strokes is 102, and the playback reproduction time when the number of strokes is 103 may be determined to be equal to each other. FIG. 25 is a diagram illustrating a method of determining a playback reproduction time based on a stroke number according to some embodiments of the invention. Fig. 25 schematically illustrates a relationship between a playback reproduction time and a number of strokes when determining the playback reproduction time according to some embodiments of the present invention when the number of strokes played back per unit time is 4.

The electronic device 3000 may store therein the number of strokes played back in each of the plurality of unit times instead of only one unit time, and may determine the number of strokes played back in each of the appropriate plurality of unit times depending on a predetermined event.

Also, the electronic apparatus 3000 may determine the reproduction speed of the playback operation. The electronic device 3000 may determine the reproduction speed of the playback operation in various ways.

For example, the electronic device 3000 may determine a reproduction speed of a playback operation according to a predetermined event. The user can request the reproduction speed of the playback operation, respectively. Thus, the electronic apparatus 3000 can determine the playback reproduction speed from the reproduction speed requested by the user. Alternatively, the operating system of the electronic device 3000 or another application installed in the electronic device 3000 may request the reproduction speed of the playback operation, respectively. Thus, the electronic device 3000 can determine the playback reproduction speed from the reproduction speed requested by the operating system or the application.

As another example, the electronic device 3000 may determine the reproduction speed of the playback operation based on the number of strokes included in the playback target.

FIG. 26 is a diagram illustrating an example of determining a playback reproduction speed from the number of strokes included in a playback target according to some embodiments of the invention.

Fig. 26 shows that a plurality of sections are divided according to the number of strokes, and a predetermined reproduction speed corresponds to each section. The electronic device 3000 may previously store the table shown in fig. 26 in the memory module 3400, confirm the total number of strokes included in the playback target, and determine the playback reproduction speed with reference to the previously stored table in accordance with the confirmed number of strokes. For example, assume that a section is determined in units of 500 strokes. When the number of strokes to be played back is equal to or greater than 1 and less than 500, the playback reproduction speed may be determined to be x 1. When the number of strokes to be played back is equal to or greater than 500 or less than 1000, the playback reproduction speed may be determined to be x 2. Similarly, the playback reproduction speed may be determined for other sections.

In this case, the determined playback reproduction speed may be determined as a default reproduction speed of the playback operation, and may be recommended to the user. That is, when the number of strokes included in the playback target is 300, x1 may be displayed to the user as the recommended playback speed, and when the number of strokes included in the playback target is 700, x2 may be displayed to the user as the recommended playback speed.

After determining the playback reproduction speed in this manner, the electronic device 3000 may determine the number of strokes played back per unit time based on the determined playback reproduction speed.

Fig. 27 shows an example in which the number of strokes played back per unit time corresponds to each reproduction speed. The electronic device 3000 may store the table shown in fig. 27 in the memory module 3400 in advance, and may determine the number of strokes played back per unit time with reference to the pre-stored table in accordance with the determined playback reproduction speed. For example, when the reproduction speed is x1, the number of strokes played back per unit time (the number of strokes played back per second) may correspond to four, and when the reproduction speed is x2, the number of strokes played back per unit time (the number of strokes played back per second) may correspond to eight.

In this way, the electronic apparatus 3000 can determine the number of strokes played back per unit time from the determined playback reproduction speed.

As described above, after determining the playback target, the electronic device 3000 may determine the reproduction time of the playback operation based on a request of the user, a request from the operating system, a request from the application, the number of strokes included in the playback target, and the like.

Subsequently, the electronic device 3000 performs the playback operation according to the determined reproduction time of the playback operation.

When performing the playback operation, the electronic device 3000 may perform the playback operation so that one stroke is displayed on the display at a stroke.

In particular, fig. 28 shows the timing of displaying the playback target stroke on the display when the number of strokes played back per second is four. It can be seen that, referring to fig. 28, one stroke is displayed on the display at intervals of 0.25 seconds, and the first stroke is displayed at 0 seconds, instead of being displayed sequentially from 0 seconds to 0.25 seconds. In other words, according to some embodiments of the present invention, when the number of strokes played back per unit time is determined, the display intervals between the strokes may be determined to be equal to each other when the playback operation is performed. That is, when performing the playback operation, the electronic device 3000 may perform the playback operation so that one stroke is displayed on the display at a stroke.

Alternatively, all strokes to be displayed on the display may be displayed at a much shorter time interval than the display interval between strokes (e.g., 0.25 seconds in the above example) rather than at one particular point in time.

However, in some embodiments, two or more strokes may be replayed such that the strokes are displayed on the display at one stroke at the same time.

As described above, the electronic device 3000 can change the playback reproduction speed according to another predetermined event while performing the playback operation based on the determined playback reproduction speed and the determined number of strokes played back per unit time.

For example, the user may request to change the playback reproduction speed that is currently being reproduced, and the operating system or another application may request to change the playback reproduction speed that is currently being reproduced.

In this case, the electronic device 3000 may change the playback reproduction speed according to other predetermined events, change the number of strokes played back per unit time based on the changed playback reproduction speed, and perform the remaining playback operations according to the changed number of strokes played back per unit time.

As described above, according to some embodiments of the present invention, when a note written by a user is replayed, a timing of writing by the user and a timing of replaying reproduction are not synchronized with each other, but the note written by the user can be reproduced and replayed in a faster time.

6. Capture of written notes

The electronic device 3000 may capture notes written with the electronic pen 2000. Here, the electronic device 3000 may provide a user interface for selecting a specific portion intended to be captured from the entire note, and the specific portion may be selectively captured from the entire note through the user interface. Also, the electronic device 3000 may provide a user interface for confirming a selected portion from the entire note to capture the portion to the user.

The electronic device 3000 can determine a portion intended to be captured (hereinafter referred to as a target note) from the entire note.

Fig. 29 is a diagram showing an entire note and a target note at the time of a capture operation of the electronic apparatus according to the present invention.

Here, the entire note may refer to any note that can be continuously reproduced through playback. For example, when the electronic device 3000 reproduces notes in units of pages of the medium 1000, the entire note a may include all notes written on the corresponding page, as shown in fig. 29. When a note written on a single page is divided and reproduced as needed, only a portion of the note on the page may correspond to the entire note described above. When notes written on a plurality of pages are continuously reproduced at one stroke, all notes written on the plurality of pages may correspond to the entire note.

Here, the target note may include the entire note or a portion thereof. The target note may include one or more portions of the entire note that are made up of a continuous series of scripts. For example, the first note T1 may be selected as the target note, as shown in FIG. 29. As another example, the first note T1 and the second note T2 may be selected as target notes, as shown in FIG. 29.

The electronic device 3000 may set a target section from a reproduction section of the entire note, and may determine a portion corresponding to the target section set from the entire note as the target note.

Specifically, the electronic device 3000 may receive at least one point of time of a reproduction section of the entire note from the user, and may set the target section based on the received point of time.

For example, the electronic device 3000 may receive a start time point and an end time point from a user, and may set a section from the start time point to the end time point in a reproduction section of the entire note as a target section.

As another example, the electronic device 3000 may receive any one of the start time point and the end time point from the user. In this case, the electronic apparatus 3000 may set a section from the start time point to the end of the reproduction section as the target section, or may set a section from the start of the reproduction section to the end time point as the target section. That is, when only the start time point is received, the end of the entire reproduction section is the end time point. On the other hand, when only the end time point is received, the start of the entire reproduction section is the start time point.

In contrast, the electronic apparatus 3000 may receive a start time point and an end time point of an excluded section from a user, set the excluded section based on the received start time point and end time point, and set the entire reproduction section excluding the excluded section as a target section.

For example, the electronic apparatus 3000 may receive a start time point and an end time point from a user, set a section from the start time point to the end time point among the entire reproduction sections as an exclusion section, and set other sections as target sections.

The electronic apparatus 3000 may set the target sections to be separated from each other. For example, the electronic device 3000 may receive a first start time point, a first end time point, a second start time point, and a second end time point from a user, set a section from the first start time point to the first end time point as a first target section, and set a section from the second start time point to the second end time point as a second target section.

In order to receive at least one of the above-described points in time, the electronic device 3000 may provide various user interfaces.

According to an example, the electronic device 3000 may display a progress bar in addition to the entire note, and may receive a user input to the progress bar and set the target section based on the user input.

Here, the progress bar may refer to an indicator associated with a reproduction progress when the pen note is reproduced. As shown in fig. 30, the progress bar P is generally provided in the form of a bar at the bottom of the screen, but is not necessarily limited to the form shown in fig. 30.

According to the present invention, the total reproduction time represented by the progress bar may correspond to a time required to reproduce a note by replay, and may be determined based on a writing time, a stroke number, an interval between adjacent writing times, and the like included in the pen data.

For example, the total reproduction time according to the progress bar may be determined to correspond to a time interval from the writing time of the pen data for the first stroke of the note to the writing time of the pen data for the last stroke. In this case, the actual time required for the user to write a note on the medium 1000 using the electronic pen 2000 may be set to be equal to the total playback time according to the progress bar.

As another example, when a time interval between adjacent strokes in writing order is greater than a predetermined reference interval, the time interval may be decreased after a total reproduction time according to the progress bar may be set to correspond to a time interval from a writing time of pen data for a first stroke of a note to a writing time of pen data for a last stroke. That is, when the time interval between the writing time of executing a specific stroke and the writing time of a stroke subsequent to executing the specific stroke is greater than the reference interval set to 20 seconds, the time interval may be reduced to a time interval of 5 seconds. In this case, the playback time may be shorter than the actual time required for the user to write notes on the medium 1000 using the electronic pen 2000, according to the total weight of the progress bar.

As still another example, the total reproduction time according to the progress bar may be determined according to the total number of strokes of the corresponding note. That is, when three strokes correspond to each second of playback and the note is composed of a total of 300 strokes, the playback time of the progress bar may be determined to be 100 seconds. In this case, the playback time may be different from the actual writing time according to the total weight of the progress bar.

The electronic device 3000 may receive a user input for setting a target section from a user through a user interface displaying such a progress bar. For example, the user may select two points from the progress bar, as shown in fig. 30, and thus the electronic apparatus 3000 may determine the previous point as a start time point, determine the next point as an end time point, and set the target section according to the determination. In this case, a start indicator I-1 for indicating a start time point and an end indicator I-2 for indicating an end time point may be displayed on the progress bar. Also, the target section R may be displayed on the progress bar P so as to be recognizable with another section. The user can select a desired point from the progress bar as shown in fig. 30 by touching the point or performing a mouse click on the point. Alternatively, the user may drag from one point to another on the progress bar, as shown in FIG. 31. In this case, the electronic apparatus 3000 may set the dragged section as the target section R. That is, the head of the dragged segment is the starting point I-1 and the tail of the dragged segment is the ending point I-2. As shown in fig. 30 and 31, a target note corresponding to the target section R may be determined from such input. Also, the user can set two or more target sections R-1 and R-2 separated from each other on the progress bar P, as shown in FIG. 32, and determine a plurality of target notes T-1 and T-2.

According to another example, the electronic device 3000 may display the entire note and may set the target section according to a user input for selecting a stroke from the note. Specifically, when two strokes are selected, the electronic device 3000 may select a note composed of strokes between the previous stroke and the next stroke as the target note. In other words, the electronic device 3000 may set the target note by setting a reproduction time point of the selected stroke as a start time point or an end time point of the target section.

Referring to FIG. 33, the user selects two strokes S-1 and S-2 from the entire note, so the target note T may be determined by the strokes between the two strokes.

Unlike this, similar to that shown in FIG. 34, the user may select two strokes S-1 and S-2 from the entire note by dragging from the starting stroke S-1 to the ending stroke S-2. Even in this case, the electronic device 3000 may determine the target note T in the same manner as selecting the two strokes S-1 and S-2 separately.

This method may be used to set multiple target sections to select multiple target notes, as shown in FIG. 35.

As described above, according to the present invention, user input for selecting a stroke from a note may accurately select the displayed stroke. However, unlike this, even when a specific stroke is not accurately selected, the electronic device 3000 may determine that the stroke closest to the point at which the user input is received is selected.

Fig. 36 and 37 show yet another example of a user interface for receiving a target section according to the present invention.

According to yet another example, the electronic device 3000 may reproduce the entire note in the form of a video through the display, receive a user input during the reproduction, and acquire a time point at which the user input has occurred as a start time point or an end time point. Specifically, referring to FIG. 36, when a note has been rendered up to section "In the revealing" U-1, the user may enter a point In time as the starting point In time. Referring to FIG. 37, when the note has been reproduced up to the section "secure" U-2, the user may input a point in time as an ending point in time. Thus, the section from section "In the evolving" U-1 to section "secure" U-2 may be set as the target section, and the target note T may be determined. As shown in fig. 36 and 37, the determination may be made by selecting a time point setting button B of a control panel C provided through a user interface or by performing a predetermined click, touch, or the like. Here, it should be appreciated that multiple target sections or multiple notes may be selected. In these figures, it is shown that a time point setting button B is provided. The first selected time may be a start time point and the second selected time may be an end time point in the selection order of the buttons. However, unlike this, the time point setting buttons may include a start time point setting button and an end time point setting button, and each time point may be determined by selecting the corresponding button.

When a user input for inputting a start time point or an end time point has occurred during reproduction, the electronic apparatus 3000 may temporarily stop reproduction or continuously perform reproduction.

As described above, the electronic device 3000 may set the target section by the user input and determine the target note according to the target section. Also, the electronic device 3000 may enable the user to easily select a target note, and also enable the user to check what the selected target note is.

Fig. 38 is a diagram illustrating a screen to output an entire note to receive a target section according to the present invention. Referring to fig. 38, the electronic device 3000 may display the entire note before receiving user input or before the determination of the target note is completed. Thus, the user can determine which portion will be the target note to be captured while confirming the entire note.

Fig. 39 and 40 are diagrams illustrating a screen for recognizing a target note according to the present invention. Referring to fig. 39, when the target note is determined, the electronic device 3000 may display the target note such that the target note is distinct from other notes. In other words, the target note and other notes may be displayed with different display attributes (e.g., brightness, color, font type, font shape, and font size). Also, referring to fig. 40, when the target note is determined, the electronic device 3000 may display the target note without displaying other notes on the screen.

Also, the electronic device 3000 may display a target section that is distinct from other sections on the progress bar, as shown in fig. 39 and 40, and may display indicators indicating start and end time points of the target section on the progress bar.

Fig. 41 is a diagram showing an image captured according to the present invention.

The electronic device 3000 may capture the target note determined in this manner. The capture of the target note may include capture in the form of pen data and capture in the form of an image. Capturing in the form of pen data may refer to storing the target note as separate pen data and enabling the electronic device 3000 or another electronic device 3000 to confirm the target note using the pen data.

When captured in the form of an image, electronic device 3000 may generate an image that includes the target note. The image may include only the target note or a media image that is the background of the target note. In some cases, the target note and other notes may be included in the captured image and may be displayed such that the target note is distinct from the other notes. The image capture may be generated by writing the target note on the media image using pen data of the target note, or may be implemented by capturing an image displayed on a screen.

Instead of or in addition to capturing the target note, the electronic apparatus 3000 may store the target note and may transmit the target note to an external device, respectively. For example, the electronic apparatus 3000 may transmit an image obtained by capturing the target note to a designated email.

Specifically, as described above, according to some embodiments of the present invention, when a note written by a user is played back, the timing of writing by the user and the timing of playback reproduction are not synchronized with each other, but it is possible to reproduce and play back the note written by the user in a faster time.

< Structure of electronic Pen >

The structure of the electronic pen 2000 according to an embodiment of the present invention will be described below.

Fig. 42 is a schematic perspective view illustrating an electronic pen having a cap part and a body part separated from each other according to an embodiment of the present invention, and fig. 43 is a schematic perspective view illustrating an electronic pen in which a cap part is inserted into an opposite side of a body part according to an embodiment of the present invention.

Referring to fig. 42 and 43, an electronic pen 2000 according to an embodiment of the present invention may include: the electronic pen 2000 includes a lead portion 2010 capable of writing notes on a medium, a main body portion 2020 configured to fix the lead portion 2010, and a cap portion 2030 configured to protect the lead portion 2010 when the electronic pen 2000 is in an unused state.

Lead portion 2010 is an element capable of writing notes on a medium and may include a ball and ink. In this case, the ink may include various forms such as oil, water, and neutral inks.

The cap portion 2030 may have a predetermined inner space S1 into which one side of the body portion 2020 may be inserted in an unused state to protect the lead portion 2010. When the cap portion 2030 is inserted into the other side of the body portion 2020, the lead portion 2010 may be partially exposed to the outside, enabling a note to be written.

Each of the body portion 2020 and the cap portion 2030 may have a cross-section formed in a circular or polygonal shape. However, an example in which the cross section is formed in a polygonal shape will be described below.

Fig. 44 is a schematic perspective view showing an internal configuration of a cap portion and a body portion of an electronic pen according to an embodiment of the present invention, fig. 45 is a schematic perspective view taken along line a-a of fig. 44, and fig. 46 is a schematic perspective view of a modified example of the cap portion of the electronic pen according to the embodiment of the present invention.

Also, fig. 47 is a schematic front view showing a main body portion of an electronic pen according to an embodiment of the present invention.

Referring to fig. 44 to 47, the electronic pen 2000 according to an embodiment of the present invention may include a pen cap portion 2030 and a body portion 2020 configured to fix a lead portion 2010. The above elements will be described in detail below.

The cap part 2030 may include a receiving part having a predetermined inner space S1 for receiving one side of the body part 2020 and a fixing part 2034 formed to protrude from a bottom surface inside the receiving part 2032.

Here, the fixing portion 2034 need not necessarily be formed to protrude from the bottom surface inside the accommodating portion 2032. For example, the fixing portion 2034 may be formed to protrude from one side inside the accommodating portion 2032.

The cap part 2030 may further include a rigid reinforcement part 2036 configured to connect an outer surface of the fixing part 2034 and a bottom surface of the receiving part 2032 such that the fixing part 2034 is stably placed in the inner space S2.

The rigidity reinforcing part 2036 may reinforce the rigidity of the fixing part 2034 and may perform a reinforcing rib function.

When the other side of the body portion 2020 is inserted, the receiving portion 2032 may receive the other side of the body portion 2020. In this case, the fixing portion 2034 may be placed in the internal space S1 to contact the other side of the body portion 2020 and specify a restriction of insertion of the other side of the body portion 2020.

In other words, when the other side of the body portion 2020 is inserted toward the inner space S1, the other side end of the body portion 2020 may contact the fixing portion 2034 to prevent the insertion. Accordingly, the other side of the body portion 2020 can be guided to be inserted into an appropriate position of the internal space S1.

Here, a light emitting unit 2028 (see fig. 43) for displaying the state of the electronic pen 2000 to the outside may be included at the other side of the main body part 2020. It is possible to provide user convenience by adjusting the degree to which the other side of the body part 2020 is inserted into the internal space S1 through the fixing part 2034 to expose the light emitting unit 2028 to the outside through the through hole part 2038 (see fig. 2 and 3) formed through the receiving part 2032.

The mounting space S2 may be a space provided by the hook portion 2023 of the main body portion 2020. The body portion 2020 may include a body portion 2024 capable of being gripped by a user and a hook portion 2023 protruding from one side of the body portion 2024 to provide a mounting space S2 so that the fixing portion 2034 is mounted to fix the cap portion 2030 in an unused state.

Specifically, the hook portion 2023 may include a first hook portion 2021 protruding from one side of the body portion 2024 toward the center of the body portion 2024 and a second hook portion 2022 separated from the first hook portion 2021 and protruding toward the center of the body portion 2024. A predetermined mounting space S2 may be formed between the first and

second hook portions

2021 and 2022.

The mounting space S2 may be a space in which the mounting portion 2033 of the fixing portion 2034 is mounted. The mounting portion 2033 may be an element formed to expand from an end of the protrusion portion 2031 (protruding from the center of one side of the receiving portion 2032 toward the inner space S1).

Here, each of the accommodating part 2032 and the body part 2024 may have a cross section formed in a circular or polygonal shape. For example, the cross section may be formed in the shape of a polygon having k sides (here, k is a natural number of 3 or more) and may be formed in a triangle as shown in the drawing.

When the body portion 2024 is inserted into the receiving portion 2032, the mounting portion 2033 may be formed to radially expand from an end of the protrusion portion 2031 toward an apex of a cross section of the receiving portion 2032 such that the mounting portion 2033 is mounted in the mounting space S2 regardless of a change in the position of the mounting space S2.

Herein, a vertex generally refers to an intersection point where line segments of a polygon intersect. However, according to the present invention, it should be noted that the apex has a concept including a position having a minimum radius of curvature.

According to the present invention, convenience can be increased by mounting the mounting portion 2033 in the mounting space S2 formed between the first and

second hook portions

2021 and 2022 when the body portion 2024 is inserted into the receiving portion 2032 regardless of the mounting method because the mounting portion 2033 is formed to be radially expanded from the end of the protrusion portion 2031.

As shown in fig. 46, the mounting portion 2033a may be formed to radially expand circumferentially from an end of the protrusion 2031 a.

The lead portion 2010 may be eccentrically placed in a direction from the center of the cross section of the body portion 2024 toward the apex of the cross section of the body portion 2024. The first and

second hook portions

2021 and 2022 may be formed between the lead portion 2010 and the center line of the cross section of the body portion 2024.

Therefore, the mounting space S2 in which the mounting portion 2033 is mounted may also be formed between the center lines of the cross sections of the lead portion 2010 and the body portion 2024. Thus, damage to the first and

second hook portions

2021 and 2022 by external factors when a user writes notes on a medium using the lead portion 2010 may be minimized, thereby preventing functional degradation of the mounting space S2.

Fig. 48 and 49 are schematic side views illustrating a process in which a cap part is inserted into one side of a main body part of an electronic pen according to an embodiment of the present invention.

First, referring to fig. 48, when the electronic pen 2000 according to the embodiment of the present invention is held in an unused state, one side of the body portion 2020 is inserted into the cap portion 2030.

In this case, the first hook portion 2021 of the main body portion 2020 may be inserted into the receiving portion 2032 of the cap portion 2030. The first hook portion 2021 may include an insertion inclined surface 2026, and the inclined surface 2026 is inclined downward in the insertion direction so that the mounting portion 2033 may contact and then enter the mounting space S2 when the first hook portion 2021 is inserted into the receiving portion 2032.

When the body portion 2024 is inserted into the accommodating portion 2032, the insertion slant surface 2026 may contact the mounting portion 2033 to serve as a first stopper wall for restricting the insertion of the body portion 2024 into the accommodating portion 2032.

Here, as shown in fig. 49, when the user continuously applies an external force to the inner space S1 of the accommodating part 2032 by pushing the body part 2024, the mounting part 2033 is finally mounted in the mounting space S2 by inserting the inclined surface 2026.

When the mounting part 2033 is placed in the mounting space S2 by a continuous external force of a user, the body 2024 may not be inserted into the inner space S1 of the accommodating part 2032 although the continuous external force is applied. This may be achieved by the second hook portion 2022 for providing the function of the second blocking wall.

When the body portion 2024 is inserted into the accommodating portion 2032, the second hook portion 2022 may be in surface contact with the mounting portion 2033 to prevent the mounting portion 2033 from being excessively inserted into the mounting space S2. Further, the second hook portion 2022 may include an excessive insertion prevention surface 2025 perpendicular to the insertion direction.

As a result, even when a continuous external force is applied from the user, the mounting portion 2033 can be stably mounted in the mounting space S2 because the mounting portion 2033 is in contact with the excessive insertion prevention surface 2025 and is prevented from being inserted into the mounting space S2.

Here, the end of the second hook portion 2022 may be formed closer to the center of the body portion 2024 than the end of the first hook portion 2021. This is because the first hook portion 2021 is an element that serves as a first blocking wall through which the mounting portion 2033 should pass, and the second hook portion 2022 should serve as a second blocking wall through which the mounting portion 2033 should not pass.

When the mounting portion 2033 is mounted in the mounting space S2, one surface 2028 of the body portion 2024 (which defines the mounting space S2) between the first and

second hook portions

2021 and 2022 may be in surface contact with the mounting portion 2033. Specifically, the surface 2028 may be formed as a surface parallel to the direction in which the body 2024 is inserted into the accommodating portion 2032.

Since the mounting portion 2033 and the body portion 2024 are in surface contact with each other in the mounting space S2, it is possible to increase the engagement force between the cap portion 2030 and the body portion 2020 and prevent the cap portion 2030 and the body portion 2020 from being separated from each other even in the case where an abnormal external force is applied.

When the user desires the electronic pen 2000 to change from a state in which the mounting portion 2033 is mounted in the mounting space S2 (i.e., an unused state of the electronic pen 2000) to a used state of the electronic pen 2000 to write notes, the user applies an external force to separate the cap portion 2030 from the body portion 2020.

To this end, the first hook portion 2021 may include a separating inclined surface 2027 inclined downward in the separating direction such that the mounting portion 2033 mounted in the mounting space S2 is retracted from the mounting space S2 when the body portion 2024 inserted into the accommodating portion 2032 is separated from the accommodating portion 2032.

The dividing inclined surface 2027 may be formed symmetrically to the above-described insertion inclined surface 2026, but is not limited thereto. The inclination angles of the dividing inclined surface 2027 and the insertion inclined surface 2026 may be slightly different.

The structure of the electronic pen 2000 will be described in more detail below.

Fig. 50 and 51 are schematic exploded perspective views illustrating a main body part of an electronic pen according to an embodiment of the present invention, and fig. 52 to 54 are schematic perspective views illustrating a substrate part, a camera module, and an illumination module fixed in the main body part of the electronic pen according to an embodiment of the present invention.

First, it should be noted that the electronic pen 2000 according to the present invention may include the cap part 2030 as described with reference to fig. 42 to 49, but the cap part 2030 may not necessarily be an essential part required to implement the electronic pen 2000.

Referring to fig. 50 to 54, the electronic pen 2000 according to an embodiment of the present invention may include a camera module 2320 and a lighting module 2340 configured to recognize a note written by a user using a lead part 2010, a substrate part connected with the camera module 2320 and the lighting module 2340, a body 2120 having a hollow S5 (which is configured to define a substrate part 2110 having a received substrate part C1 received in the hollow S5 and an exposed substrate part C2 exposed to the outside of the hollow S5 and is made of a first material for blocking electromagnetic waves generated during wireless communication), a battery part 2160 configured to drive the camera module 2320 and the lighting module 2340, and a wireless communication module 2440 mounted in the exposed substrate part C2 so as to wirelessly communicate with other electronic devices.

The camera module 2320, the lighting module 2340 and the wireless communication module 2440 have been described in detail above, and thus a detailed description thereof will be omitted.

The body 2120 may be an element providing an appearance of an electronic pen according to an embodiment of the present invention. The main body 2120 may constitute the main body 2020 together with the substrate fixing portion 2030 and the substrate protecting portion 2140.

In the electronic pen 2000 according to the present invention, the substrate portion 2110 may be implemented with a single substrate, but may be implemented with a plurality of substrates. For example, as shown, substrate portion 2110 may be implemented with a first substrate 2112, a second substrate 2114, and a third substrate 2116.

For convenience of description, an example in which the substrate portion 2110 includes three substrates will be described below.

Specifically, the substrate portion 2110 may include a first substrate 2112 disposed near a first end of the body 2120 (from which the lead portion 2010 is exposed) and a second substrate 2114 disposed near a second end opposite the first end, and may further include a third substrate 2116 disposed between the first substrate 2112 and the second substrate 2114.

Here, the first, second, and

third substrates

2112, 2114, and 2116 may be electrically connected to each other through the flexible printed circuit board FC, and may be defined as a received substrate portion C1 or an exposed substrate portion C2 depending on whether the substrates are received in the hollow S5 of the body 2120. The specific substrate may include all of the accommodated substrate section C1 and the exposed substrate section C2.

Conceptually, the accommodated substrate section C1 and the exposed substrate section C2 refer to a substrate region virtually divided into a portion accommodated in the hollow S5 of the body 2120 and a portion exposed to the outside of the hollow S5. Both the accommodated substrate section C1 and the exposed substrate section C2 may exist in one substrate.

For example, as shown, since the first and

third base plates

2112 and 2116 are accommodated in the hollow S5 of the body 2120, the first and

third base plates

2112 and 2116 may integrally form an accommodated base plate portion C1.

On the other hand, since the second substrate 2114 is accommodated only partially in the hollow S5, the accommodated portion may form an accommodated substrate section C1, and another portion may form an exposed substrate section C2.

That is, in the electronic pen 2000 according to the present invention, the second substrate 2114 may include at least a portion of the accommodated substrate portion C1 and the exposed substrate portion C2.

Here, the body portion 2020 may include a substrate fixing portion 2030 connected with the body 2120 such that the accommodated substrate portion C1 is mounted and fixed at a predetermined position in the hollow S5, and the body 2120. The exposed substrate portion C2 may be mounted in the substrate fixing portion 2030 and fixed at a predetermined position.

The substrate fixing portion 2030 may be one structure for fixing the first substrate 2112, the second substrate 2114, and the third substrate 2116. The first substrate 2112, the second substrate 2114, and the third substrate 2116 may be mounted in the substrate fixing portion 2030 through an additional member 2150.

The substrate fixing portion 2030 may be composed of a single member, but is not limited thereto. The substrate fixing portion 2030 may be implemented by connecting a plurality of components to each other.

The exposed substrate portion C2, which is part of the second substrate 2114, may be exposed from a second end opposite the first end of the body 2120 from which the lead portion 2010 is exposed. Thus, the wireless communication module may be disposed proximate to the second end.

Here, the camera module 2320 and the lighting module 2340 may be mounted in the first substrate 2112 and may be electrically connected to the first substrate 2112 through the flexible printed circuit board FC.

Also, a battery portion 2160 configured to provide driving force to the camera module 2320 and the lighting module 2340 may be disposed between the first substrate 2112 and the second substrate 2114.

The battery part 2160 may be mounted in the substrate mounting part 2132 providing the mounting space S6.

Since the battery portion 2160 may be disposed between the first and

second substrates

2112, 2114, the battery portion 2160 may be disposed proximate to the second end of the body 2120. However, to increase the convenience of the user's writing, the battery part 2160 may be formed near the first end of the body 2120 such that the center of gravity of the electronic pen 2000 is biased to the first end of the body 2120.

A wireless communication module 2440 capable of wireless communication with other electronic devices may be mounted in the second substrate 2114. Specifically, the wireless communication module 2440 may be mounted in the exposed substrate portion C2 of the second substrate 2114.

The wireless communication module 2440 may be exposed to the outside of the body 2120 made of the first material. Regardless of the first material, the wireless communication module is not prevented from wirelessly communicating with other electronic devices.

Here, the first material of which the body 2120 is made may include aluminum (Al) blocking electromagnetic waves generated during wireless communication of the wireless communication module 2440, but is not limited thereto. Any material may be applied as long as the material can block electromagnetic waves.

As described above, the wireless communication module 2440 for implementing wireless communication with other electronic devices may be mounted in the exposed substrate portion C2 exposed to the outside of the body 2120 such that the wireless communication module 2440 is disposed outside the body 2120 made of the first material capable of blocking electromagnetic waves. Accordingly, the wireless communication module 2440 can perform wireless communication without any influence from the main body 2120. Thus, the recognizability of the user's writing can be maximized.

Also, since the main body 2120 is made of the first material, the electronic pen 2000 according to the present invention may prevent electromagnetic waves generated during wireless communication of the wireless communication module 2440 from being discharged to the outside through the main body 2120. Thus, the harmful effects of electromagnetic waves generated during wireless communication can be minimized.

Also, since the body 2120 made of the first material has a higher heat transfer coefficient than the substrate fixing part 2030 made of a synthetic resin (such as plastic), heat generated by operating the lighting module 2340 or the like can be effectively released to the other side. Thus, the durability of the electronic pen 2000 can be enhanced.

The body portion 2020 provided in the electronic pen 2000 according to the present invention may include a substrate protecting portion 2140 connected to the body to protect the exposed substrate portion C2 by surrounding the exposed substrate portion C2 together with the body 2120 and the substrate fixing portion 2030.

The substrate protecting portion 2140 may be a kind of pen cap connected to the second end of the body 2120 to cover the exposed substrate portion C2. As described above, the exposed substrate portion C2 may be part of the second substrate 2114.

The substrate protection part 2140 may be made of a second material that passes electromagnetic waves generated during wireless communication of the wireless communication module 2440 mounted in the exposed substrate part C2. Therefore, the wireless communication module is not prevented from wirelessly communicating with other electronic devices regardless of the substrate protecting portion 2140.

Here, the second material may be a synthetic resin (including plastic), and may have a lower heat transfer coefficient than the first material.

Fig. 55 to 57 are schematic perspective views illustrating a process in which a lead wire is inserted into a main body part of an electronic pen according to an embodiment of the present invention.

First, referring to fig. 55, an electronic pen 2000 according to the present invention may have a lead part 2010 which is freely replaceable. The lead portion 2010 may be inserted into a lead portion insertion hole S7 formed in the substrate fixing portion 2030 of the main body portion 2020.

Referring to fig. 56, the substrate fixing part 2030 may include: a substrate mounting portion 2132 in which the accommodated substrate portion C1 is mounted, a lead portion insertion guide portion 2134 configured to provide a path through which the lead portion 2010 is inserted, and a lead portion fixing portion 2136 configured to fix the lead portion 2010 by shrink-fitting the lead portion 2010 inserted into the lead portion insertion guide portion 2134.

The substrate mounting portion 2132 may be a kind of frame constituting the substrate fixing portion 2030, and the lead portion insertion guide portions 2134 may be formed such that the lead portion insertion guide portions 2134 become paths communicating with the lead portion insertion holes S7 and may be formed in an open state.

However, the lead portion insertion guide portion 2134 does not necessarily need to be formed in the open state and may be formed as a groove or a hole.

Referring to fig. 55 and 56, the lead part insertion guide part 213

A first lead portion insertion guide portion 2135 through which the lead portion 2010 passes and a second lead portion insertion guide portion 2137 in which the lead portion fixing portion 2136 is mounted may be included. The lead fixing portion 2136 may be slidably mounted in the second lead insertion guide portion 2137.

The lead fixing part 2136 may be removably mounted in the second lead insertion guide part 2137 and may include a protrusion part 2139 formed to extend outward so that the lead fixing part can slide in the second lead insertion guide part 2137 without falling off when the lead fixing part 2136 is mounted in the second lead insertion guide part 2137.

Also, the second lead portion insertion guide portion 2137 may include a protrusion corresponding portion 2131 formed to protrude to form a space in which the protrusion portion 2139 is mounted.

When the lead portion fixing portion 2136 is mounted in the second lead portion insertion guide portion 2137, the protruding portion 2139 can be caught by the protrusion corresponding portion 2131, thereby preventing falling off.

Since the lead part fixing part 2136 is slidably mounted in the lead part insertion guide part 2134, the electronic pen 2000 according to the present invention can make a user feel comfortable when he or she writes notes.

In the second lead insertion guide portion 2137, a suction force acts between the lead fixing portion 2136 and the board mounting portion 2132. The magnetic forces of the first magnet 2172 and the second magnet 2174 may be used to generate the attractive force.

The first magnet 2172 may be mounted in the lead portion fixing portion 2136, and the second magnet 2174 may be mounted in the substrate mounting portion 2132 to maintain the attractive force between the lead portion fixing portion 2136 and the substrate mounting portion 2132.

Resilient elastomer 2176 may be disposed between first magnet 2172 and second magnet 2174 and may provide smoothness when a user writes a note on the medium. Thus, user convenience can be enhanced.

< method for controlling electronic apparatus >

1. Method #1 for real-time reproduction of notes written by a user

A first embodiment of a real-time reproduction method for notes written by a user according to the present invention will be described below.

FIG. 58 is a flowchart of a first embodiment of a method for real-time rendering of notes written by a user in accordance with the present invention. Referring to fig. 58, the real-time reproducing method for a note written by a user includes acquiring pen data (S110), acquiring information about a target medium based on the acquired pen data (S120), displaying a medium image corresponding to the target medium (S130), and reproducing the user' S writing on the medium image based on the acquired pen data (S140).

The reproducing of the user' S writing (S140) may include reproducing notes written by the user at corresponding locations on the media image in real time based on stroke data generated from the stroke data.

Accordingly, the electronic device 3000 may reproduce a note written by the user in real time based on pen data received from the electronic pen 2000 and display the reproduced note through the electronic device 3000.

Each step has been described in detail in the above description of the operation method of the electronic device, and thus a detailed description thereof will be omitted.

2. Method #2 for real-time reproduction of notes written by a user

A second embodiment of a real-time reproduction method for notes written by a user according to the present invention will be described below.

FIG. 59 is a flowchart of a second embodiment of a method for real-time rendering of notes written by a user in accordance with the present invention. Referring to fig. 59, the real-time reproducing method for a note written by a user includes acquiring pen data (S210), determining whether a predetermined condition is satisfied for a first area, which is at least a portion of a media image being displayed, based on the acquired pen data (S220), moving the portion of the media image being displayed through a display from the first area to a second area and then displaying the moved portion in the second area when the predetermined condition is satisfied (S230), and reproducing the note written by the user in the second area (S240).

In this case, the predetermined condition is whether the writing position of the user is outside the first area. That is, when the position of the user's writing is outside the first area displayed through the display, it may be determined that the predetermined condition is satisfied.

When the electronic device 3000 reproduces a note written by a user in real time based on pen data received from the electronic pen 2000, the target medium may be enlarged and displayed, and thus only a portion of the target medium, not the entire target medium, may be displayed through the display. In this case, when the user's writing is input to an area that is not yet displayed through the display, the electronic device 3000 may move the display area to a corresponding area and then reproduce the user's writing.

3. Playback of written notes #1

A first embodiment of a playback method for a written note according to the present invention will be described below.

FIG. 60 is a flowchart of a first embodiment of a playback method for written notes in accordance with the present invention. Referring to fig. 60, the playback method includes receiving information on user 'S writing from an electronic pen (S310), storing the received information on the user' S writing in a memory (S320), and performing a playback operation for sequentially playing back notes written by the user on a display, wherein the notes include at least a first stroke and a second stroke following the first stroke (S330).

In this case, regardless of the time required for the user to write the first stroke and the second stroke, each of the first stroke and the second stroke may be displayed on the display once at a stroke, the first stroke may be displayed on the display at a first time point and the second stroke may be displayed on the display at a second time point later than the first time point.

Accordingly, when the electronic device 3000 replays the user's writing based on the pen data received from the electronic pen 2000, the user's writing can be replayed in a time shorter than the actual time required for the user's writing.

4. Playback of written notes #2

A second embodiment of a playback method for a written note according to the present invention will be described below.

FIG. 61 is a flow chart of a second embodiment of a playback method for written notes in accordance with the present invention. Referring to fig. 61, the playback method for a written note includes receiving pen data, which is information on writing of a user, through a communication unit (S410), storing the received pen data in a memory (S420), and performing a playback operation for sequentially playing back notes written by the user on a display based on the stored pen data (S430).

In this case, the reproduction time of the playback operation may be determined based on the number of strokes included in the writing of the user to be performed with the playback operation.

Accordingly, when the electronic device 3000 plays back the user's writing based on the pen data received from the electronic pen 2000, the user's writing can be played back by determining the reproduction time of the playback operation based on the number of strokes in a time shorter than the actual time required for the user's writing.

5. Capture of written notes

The writing capture method according to the present invention will be described below.

FIG. 62 is a flow diagram of a writing capture method according to an embodiment of the present invention. Referring to fig. 62, the writing capturing method may include receiving pen data from the electronic pen 2000 (S510), acquiring a written note based on the pen data (S520), receiving a user input corresponding to at least one point of time of a reproduction section of the written note (S530), setting a target section based on the user input (S540), determining a target note corresponding to the target section from the written note (S550), and capturing an image including the target note (S560).

First, the electronic device 3000 may receive pen data regarding writing from the electronic pen 2000 (S510). Data regarding writing time points and coordinate data written on the medium 1000 using the electronic pen 2000 may be included in the pen data.

The electronic device 3000 may acquire the written note using the pen data (S520). The written note may be obtained by connecting the coordinate data in the order of writing time points to generate strokes.

The electronic device 3000 may receive a user input corresponding to at least one time point in the reproduction part (S530). To this end, the electronic device 3000 may display a progress bar to receive user input to the progress bar, display a written note to receive a specific stroke in the written note, or receive user input while reproducing the written note. The electronic device 3000 may acquire the start time point and the end time point of the reproduction section according to such user input. When only one point of time is acquired, other points of time may be autonomously set by the electronic apparatus 3000.

Also, to receive such user input, the electronic device 3000 may display an entire note on the screen to assist the user in entering the input.

The electronic apparatus 3000 may set the target zone based on the user input (S540). The electronic device 3000 may determine a start time point and an end time point of the target zone according to a user input, and may set the target zone according to the determined start time point and end time point.

When the target section is set, the electronic device 3000 may determine a target note from the entire note (S550). One or more target notes may be determined. Also, the electronic device 3000 may display the target note such that the target note is distinct from other notes and may provide feedback to the user as to which note is determined to be the target note.

The electronic device 3000 may capture the selected target section (S560). The capturing may be performed in an image capturing manner or a pen data manner. Also, the captured image may be transmitted to an external device or the outside via the internet, or may be separately stored.

In the control method of the electronic device according to the present invention described above, the steps of each embodiment are not necessary, and therefore each embodiment may selectively include the steps described above. The steps of each embodiment need not be performed in the order described. The steps described later may be performed before the steps described earlier.

The control method of the electronic device according to the present invention may be implemented in the form of a code or program that executes the control method. Such a code or program may be stored in a computer-readable recording medium.

It will be apparent to those skilled in the art that various substitutions, modifications and changes may be made thereto without departing from the scope and spirit of the invention. The present invention is not limited by the above-described embodiments and drawings. However, the embodiments described herein may be applied without limitation, and various modifications may be made to the embodiments by selectively combining all or some of the embodiments. Further, the steps of each embodiment may be used separately or in combination with the steps of another embodiment.

Claims

1. An electronic device for reproducing notes written by a user on a medium, the electronic device comprising:

a communication unit configured to communicate with an electronic pen that detects information about writing of a user;

a display configured to display a first portion of a media image corresponding to a media, reproduce a note written by a user on the media image, and visually represent the reproduced note; and

a control unit configured to:

pen data including pressure values is obtained from the electronic pen,

wherein the pen data includes first data corresponding to the pen-down time point and second data corresponding to a predetermined time point after the pen-down time point, an

Wherein the pressure value is above a predetermined value between the pen-down time point and the predetermined time point,

selecting a second portion of the media image as a display image based on the first data corresponding to the pen down time point, wherein

Changing the display image from the first portion to the second portion;

displaying the selected second portion as a display image in which the user's writing is reproduced through the display;

selecting a third portion of the media image as a display image based on a second stroke of data corresponding to the predetermined point in time, wherein the display image changes from the second portion to the third portion; and

displaying the selected third portion as a display image in which the user's writing is reproduced through the display.

2. A control method of an electronic device for reproducing notes written by a user on a medium, the control method comprising:

displaying a first portion of a media image corresponding to a media;

acquiring pen data including a pressure value from an electronic pen that detects information about writing of a user;

selecting a second portion of the media image as a display image based on the first data corresponding to the pen down time point, wherein the display image changes from the first portion to the second portion;

displaying the selected second portion as a display image in which the user's writing is reproduced;

the selected third portion is displayed as a display image in which the user's writing is reproduced.

3. The control method of claim 2, wherein the second portion of the media image comprises a location of a user's writing on the media at a predetermined point in time.

4. The control method of claim 2, wherein the second stroke data corresponds to a stroke-up time point.

5. The control method of claim 2, wherein the second portion of the media image is determined such that a reproduction of a user-written note corresponding to the first data is placed in a center portion of the second portion of the media image, and

the third portion of the media image is determined such that a rendition of the user-written note corresponding to the second stroke of data is positioned in a center portion of the third portion of the media image.

6. The control method according to claim 2, wherein

The pen data includes third pen data corresponding to the pen-up time point,

the control method further includes determining a stroke input of the user on the medium based on the pen down time point and the pen up time point.

7. The control method of claim 2, wherein the selected second portion includes a reproduction of a user's writing based on the first data before being displayed.

8. The control method of claim 2, wherein the selected second portion includes a reproduction of the user's writing based on the first data after being displayed.

9. A control method of an electronic device for reproducing notes written by a user on a medium, the method comprising:

displaying a first portion of a media image corresponding to a media;

wherein the pen data includes first data corresponding to the pen-down time point, second data corresponding to a predetermined time point after the pen-down time point, and third data corresponding to the pen-up time point, and

wherein the pressure value is above a predetermined value between the pen-down time point and the pen-up time point,

holding the selected second portion as a display image based on a second stroke of data corresponding to the predetermined point of time;

displaying a second portion in which the user's writing is reproduced;

selecting a third portion of the media image as a display image based on a third stroke of data corresponding to the stroke-raising time point, wherein the display image changes from the second portion to the third portion; and