JP2012190303A - Comment sharing system, method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a system or the like for achieving dense communication through a computer system by effectively using handwritten information by a user.SOLUTION: A comment sharing system includes: a content server for storing and managing an electronic content; a plurality of content display devices that are connected to the content server through a network and capable of downloading the electronic content; and a comment information management server for storing and managing comment information that is added to a page of the electronic content displayed on the content display devices. The content display device transmits the comment information that is added to the electronic content to the comment information management server as handwritten data expressed as a secular change of a coordinate, and the comment information management server stores the handwritten data.

Description

  The present invention relates to a system for widely sharing comment information, and more particularly to a system that can share handwritten data written with a pen or the like on an input device such as a tablet among a plurality of people.

  There are a number of application software that can exchange and share information among multiple users over a computer network. For example, e-mail is a communication tool that can attach an image or a sound as needed, with text as the center. E-mail is widely used in that users can receive and reply to messages at a convenient time without constraining time between users.

  On the other hand, there are systems that share electronic books and electronic contents among a plurality of users, accept comments from a large number of specific users or a large number of unspecified users for these shared contents, and make them viewable from a plurality of users.

  For example, a system that allows users to add and share memo information as if they were pasting memos on a book for information that can be shared on a network called an ebook. (Patent Document 1).

  That is, Patent Document 1 discloses an electronic book server that stores and manages electronic book content, a plurality of client terminals connected to the electronic book server via a network, and a user of the client terminal that stores the electronic book content. A method for sharing electronic book content and memo information in a system having a memo server for managing memo information added to a page, the electronic book content managed by the electronic book server being downloaded via the network at the client terminal A process of displaying, and when a memo is added to the page of the electronic book content viewed on the client terminal and user information sharing the memo is input, transmitting the information to the memo server; To the client terminal In response to an operation on the electronic book content, the memo server searches for a memo on the page of the electronic book content and includes a user information included in the shared user information, and displays the memo. An electronic book content and memo information sharing method characterized by the above is disclosed.

  Further, there is the following (Patent Document 2) as the one that can collect the latest information and useful information widely and enhance the electronic book data by the user-participating electronic book publishing system (Patent Document 2).

  That is, Patent Document 2 discloses information collection means for collecting provided information added to a container storing electronic book data by a user operation in a storage device on the host computer side, and all or part of the collected provided information. An electronic book publishing system comprising: editing means for editing and adding to the electronic book data; and distribution means for distributing electronic book data with the edited provision information added thereto is disclosed. .

  In addition, there is the following as a comment distribution system that allows users to share the same video and use a comment to communicate (Patent Document 3).

  That is, Patent Document 3 includes a comment distribution system that includes a video distribution server that distributes video data, a comment distribution server that distributes comments on the video, and a plurality of terminal devices. A first comment information storage unit that stores, as comment addition time, a moving image playback time based on a moving image playback start time at the time when comment content is added, and stores the comment information as comment information; A comment information storage unit that reads the comment information stored in the comment information storage unit and distributes the comment information to the terminal device, and the terminal device receives the video data distributed from the video distribution server. The comment reproduction server and the comment information input to the movie to be reproduced from the comment distribution server A comment information receiving unit to be transmitted, a second comment information storage unit for storing the comment information received by the comment information receiving unit, and a moving image reproduced by the moving image reproducing unit, and the second comment information A first display unit that reads out the comment information associated with the video playback time of the video to be played back from the comment information out of the comment information stored in the storage unit, and displays the read comment content together with the video A second display unit that displays a list of comment information stored in the second comment information storage unit; and a comment information selection among a list of comment information displayed by the second display unit. A selection unit that accepts input, and comment information that has been accepted by the selection unit as input for selection. A reproduction control unit that reproduces the moving image from the reproduction time of the movie corresponding to the comment adding time of the comment information read out from the storage unit and displays the read comment information on the first display unit; A transmission unit that accepts input of comment content for the video displayed by the display unit, and transmits the video playback time at the time when the comment content is input to the comment distribution server as comment information together with comment content; The first comment information storage unit further stores real time information indicating the time when the comment content is input in the comment information, and the second display unit stores the comment The comment distribution server has a function of displaying a list according to the order of the real-time information. Further includes a comment erasure request information receiving unit that receives comment erasure request information including a comment ID for identifying a comment to be erased from the terminal device, and the comment erasure request information receiving unit receives the comment erasure request information receiving unit. Information indicating that the comment is not displayed is added to the comment specified by the comment ID included in the comment deletion request information and transmitted to the terminal device, and the terminal device further receives the comment information from the comment distribution server. A comment distribution system comprising: a display control unit configured to hide the received comment on the first display unit according to information indicating that the comment attached to the comment information is not displayed when received. It is disclosed.

JP 2001-52025 A JP 2002-132774 A Japanese Patent No. 4263218

  However, comment information (memo information) from a user managed in the above-described electronic book server or comment distribution system is registered in a record as text data or managed as scanning information (image data) of handwritten memos. There is room for improvement in the effective use of user handwritten data.

  In addition, even if the position in the electronic book where the user's memo is written, the place (latitude, longitude) where the memo was written, and the memo creation time are recorded, the contents of the memo are recorded in any order. In particular, there was room for improvement in the effective use of user handwritten data.

  The present invention provides a content server for storing and managing electronic content, a plurality of content display devices connected to the content server via a network and capable of downloading the electronic content, and the electronic content displayed on the content display device A comment sharing system including a comment information management server for storing and managing comment information assigned to the page, wherein the content display device represents the comment information assigned to the electronic content as a change in coordinates over time The handwritten data is transmitted to the comment information management server, and the comment information management server stores the handwritten data.

  The comment information management server manages the handwritten data for each stroke.

  The comment information management server manages the handwritten data together with writing pressure data.

  Further, the content display device displays a predetermined area for inputting comment information given to the electronic content on the electronic content.

  The comment information management server manages the handwritten data as a different layer for each of the plurality of content display devices.

  With the comment sharing system and the like according to the present invention, it is possible to effectively utilize the user's handwritten information and realize higher-density communication via the computer system.

It is explanatory drawing explaining the structure of the comment sharing system in one Embodiment of this invention. It is explanatory drawing explaining the structure of the comment sharing system in other embodiment of this invention. It is explanatory drawing explaining the external appearance structure of the content display apparatus in the system concerning one Embodiment of this invention. It is explanatory drawing explaining the functional block of the content display apparatus in the system concerning one Embodiment of this invention. It is explanatory drawing explaining the structure of the comment information table used in the system concerning one Embodiment of this invention. It is a flowchart explaining the operation | movement flow of the comment sharing system in one Embodiment of this invention. It is a flowchart explaining the operation | movement flow of the comment sharing system in one Embodiment of this invention. It is explanatory drawing explaining the example of a display of the content display apparatus in the system concerning one Embodiment of this invention. It is explanatory drawing explaining the example of a display of the content display apparatus in the system concerning one Embodiment of this invention. It is explanatory drawing explaining the example of a display of the content display apparatus in the system concerning one Embodiment of this invention. It is explanatory drawing explaining the example of a display of the content display apparatus in the system concerning one Embodiment of this invention. It is explanatory drawing explaining the example of a display of the content display apparatus in the system concerning other embodiment of this invention. It is explanatory drawing explaining the example of a display of the content display apparatus in the system concerning other embodiment of this invention. It is explanatory drawing explaining the operation example of the content display apparatus in the system concerning other embodiment of this invention. It is a flowchart explaining the operation | movement flow of the system concerning other embodiment of this invention. It is a flowchart explaining the operation | movement flow of the system concerning other embodiment of this invention. It is a flowchart explaining the operation | movement flow of the system concerning other embodiment of this invention. It is a flowchart explaining the operation | movement flow of the system concerning other embodiment of this invention. It is a flowchart explaining the operation | movement flow of the system concerning other embodiment of this invention.

  Hereinafter, embodiments for implementing a comment sharing system and the like according to the present invention will be described in detail with reference to the drawings.

  FIG. 1 shows a configuration of a comment sharing system according to an embodiment of the present invention. It is a structural example of a typical B2C system. The comment sharing system 100 includes an API server 101, a handwritten information management server (hereinafter referred to as an ink server or INK server) 102, a database server (hereinafter referred to as DB server) 103, a content management system (hereinafter referred to as CMS) 104, a content The server 105 and the content display device 106 are connected via a wired or wireless network. Each server / system, including those not shown, is configured to be communicable with an appropriate communication protocol via a network.

  The API server 101 is a server system having an application program interface for providing electronic content such as electronic book content and magazine data to the user, and is provided to the platform (OS) of the content display device 106 carried by the user. It has a set of possible instructions and functions.

  The INK server 102 is a server system that manages handwritten information, for example, as coordinate data or stroke data. The INK server 102 cooperates with the API server as necessary to provide magazine data or paper provided to the user (the content display device 106 is carried). A process of associating the user's handwritten information with the data is performed.

  The DB server 103 stores a database necessary for operating the comment sharing system 100. For example, user information, XML format for distribution, metadata, and the like are stored. Although not essential to the present system, a charging information management database can be built in the DB server 103 as needed.

The CMS 104 is a system for managing content, and can manage all types of electronic content including text, images, audio, moving images, and the like. Content can be registered to the CMS 104 from a remote or other application.
The CMS 104 can also perform operations such as content creation, editing, and search. These operations can be performed remotely from another server or computer terminal, for example, by providing a browser-based user interface. .
Further, in the CMS 104, it is possible to disassemble individual contents constituting an electronic book or an electronic magazine and manage assets as a content organization unit.

  The content server 105 is a server for distributing content managed by the CMS 104 to the terminal 106. Distribution forms include streaming distribution of moving images in addition to still image distribution, and content distribution combining these with electronic books and electronic magazines is possible.

In addition, the content server 105 in FIG. 1 can be configured as a distributed network in which a plurality of cache servers each having a copy of the original content are arranged at geographical locations in order to accommodate access from a large number of users. . In this case, it is possible to reduce the load on the entire server by distributing user requests to appropriate cache servers. Further, the content server can be configured to adopt a plurality of delivery methods such as unicast delivery, unicast delayed delivery, and multicast delivery method so as to switch efficiently as the user request increases.
For example, when a large number of users request content in a short period of time, unicast delayed delivery is employed to delay the delivery timing. If the delayed delivery cannot be handled due to an increase in user requests, it can be switched to multicast delivery.

  The content display device 106 is a terminal for a user to view content, and is configured as a terminal capable of inputting a comment from the user. The comment includes text input, handwriting input, and voice input. If it is a terminal equipped with a camera, it can be configured so that still images and moving images can be posted. Typically, notepads and PDAs (personal digital assistants) are used, but a detailed configuration example will be described later.

  FIG. 2 shows the configuration of a comment sharing system according to another embodiment of the present invention. It is a structural example of a typical B2B system. The comment sharing system 200 includes an API server 201, an INK server 102, a DB server 203, a CMS 204, a content server 205, and a content display device 106, which are connected by a wired or wireless network. . A characteristic difference from the comment sharing system 100 shown in FIG. 1 is that an authentication protocol is employed in communication between the API server 201 and the DB server 203, and this will be described in detail.

  Various protocols for user authentication can be adopted as an authentication protocol used in communication between the API server 201 and the DB server 203, for example, CHAP (Challenge Handshake Authentication Protocol) or PAP (Password Authentication). There are user authentication security protocols such as Protocol. As described above, in the comment sharing system 200, communication between the company that provides the content and the company that provides the service to the general user while using the content is possible. It is suitable for implementation as an accompanying electronic commerce system.

  FIG. 3 shows an external appearance of a content display device that constitutes a system according to an embodiment of the present invention. As an example, the content display device 106 includes a housing unit 1061, a display 1062, and hardware buttons 1063 a and 1063 b provided on the left and right sides of the housing 106. The housing portion 1061 has a horizontally long shape, but may have a vertically long shape (see FIGS. 8 to 13). In addition, the home button is replaced with the hardware buttons 1063a and 1063b or together with the hardware buttons 1063a and 1063b. 890 (see FIGS. 8-13) can also be implemented.

  The display 1062 includes a liquid crystal display (LCD) or the like, and can display various information such as characters and images as will be described later. Further, a menu button or a software keyboard is displayed on the display 1062, and an instruction (command) to the information display device can be obtained by touching the button with a finger or a touch pen (not shown). In this respect as well, the hardware buttons 1063a and 1063b are not essential components, but will be described as buttons having a certain function for the convenience of explanation of the present invention. Of course, these hardware buttons 1063a and 1063b can be replaced with menu buttons displayed on a part of the display 1062.

  Further, the display 1062 includes a multi-touch input panel, and touch input position coordinates on the touch input panel are transmitted to a processing system (CPU) of the content display device 106 via an input device interface (not shown). And processed. And this multi-touch input panel is comprised so that the several contact point with respect to a panel can be sensed simultaneously. This detection (sensor) can be realized by various methods, and is not necessarily limited to the contact sensor. For example, an indication point for the panel can be extracted using an optical sensor. Furthermore, in addition to a contact sensor or an optical sensor, a capacitive sensor that senses contact with human skin can be used as the sensor.

  The multi-touch input panel may be a pen pressure sensing type liquid crystal pen tablet in which a liquid crystal display is integrated with the pen tablet. In this case, the level 512 to 2048 is typically adopted as the writing pressure level. However, it will be obvious to those skilled in the art that higher resolutions can be achieved with improvements in pen tablet manufacturing technology.

  Although not shown in FIG. 3, the content display device 106 can include a microphone and a speaker as provided in many portable information terminals. In this case, it is possible to determine the voice of the user picked up from the microphone and use it as an input command.

  FIG. 4 illustrates a block diagram of hardware constituting the content display device 106 according to the present invention. The operation of the content display device 106 is realized by individual operations of hardware described below and their cooperative operations.

  The content display device 106 is roughly divided into an input unit 401 including hardware buttons 1063a and 1063b, a multi-touch input panel provided on the display 1062, a microphone, and a hard disk for storing programs, data, and the like. A storage unit 402 composed of RAM and / or ROM, a central processing unit 403 composed of a CPU that performs various numerical calculations and logical operations by a program, a display unit 404 composed of a display 1062, etc. A control unit 405 for controlling the chip and the electrical system, a slot for accessing the Internet, a port for performing optical communication, and a communication interface unit 406 including a communication interface, a speaker, a vibration, etc. Output unit 407 and time etc. It consists of a timing unit 408 for measuring time, a sensor unit 409 for measuring temperature, humidity, device posture, light quantity, etc. as necessary, and a power source unit 410 for supplying power to each module in the apparatus. These modules are appropriately connected by a communication bus or a power supply line (connection 411 in FIG. 4) as necessary.

  Further, the program or software necessary for implementing the present invention is normally installed or stored in a hard disk or the like in the storage unit 402, and when executing the program or software, all of the program or software is stored in the memory in the storage unit 402 as necessary. Alternatively, it is read out as a part of software module and is executed in the central processing unit 403.

  It should be noted that the execution of the calculation is not necessarily performed by the central processing unit 403 such as a CPU, and an auxiliary arithmetic unit such as a digital signal processor (DSP) (not shown) can be used.

  Note that the sensor unit 409 includes a writing pressure detection sensor when the above-described writing pressure detection type liquid crystal pen tablet is employed. As the writing pressure level, 512 level to 2048 level is typically adopted, and the strength of the writing pressure can be changed. At this time, the maximum level is set in consideration of the strength of the tablet.

  FIG. 5 shows the structure of the comment information table used in the system according to the embodiment of the present invention. The comment information table 500 is stored in the INK server 102 as one embodiment. As shown in FIG. 5, the comment information table 500 includes a “comment ID” item 501, a “content ID” item 502, a “page” item 503, a “position coordinate” item 504, and “ A comment giver item 505 and a comment content item 506 are included.

  The “comment ID” item 501 stores an ID uniquely assigned to a comment posted by the user regarding an electronic book or magazine content. An arbitrary code can be adopted as the comment ID, and it functions as an identification code for uniquely identifying a comment managed in the comment sharing system.

  The “content ID” item 502 stores an ID uniquely assigned to the electronic book or magazine content itself. Thereby, it can be determined to which content the corresponding comment ID is attached.

  The “page” item 503 stores numerical data for indicating on which page of the electronic book or magazine content the comment is attached. As a result, it is possible to determine to which page of which content the corresponding comment ID is assigned.

  The “position coordinate” item 504 stores the position coordinate information of the comment attached to the content page. Typically, the start coordinates of the comment area with respect to the page coordinate system (for example, the upper left is the origin) are registered.

  The “comment grantor” item 505 stores user information that posted a comment. Typically, the user ID is stored, but the present invention is not limited to this, and instead of the user ID or in addition to the user ID, the user profile information (birth date, birth date) (Location, occupation, blood type, height, weight, special skill, hobby, career, etc.) may be registered.

  The “comment content” item 506 stores the comment content or a pointer address to the comment content. As a data format of comment contents, user handwritten data is stored in addition to text data. Alternatively, a pointer address to still image data or moving image data may be stored, and actual data may be stored in a storage area starting from another address.

  In addition, when storing the user's handwritten data in the comment content, the pen coordinates for every predetermined time (for example, 10 milliseconds) can be stored as a list. In this case, the writing pressure at the pen coordinates may be a numerical value corresponding to the resolution level and stored together with the pen coordinates. Specifically, assuming that the sampling interval is 10 milliseconds, pen coordinates and writing pressure data for 0.1 seconds can be stored as follows.

Here, the first column (left end) is the stroke ID. As the stroke ID, one stroke ID can be assigned to each stroke constituting the character. The second column shows the time interval every 10 milliseconds. The third column is the pen coordinates, where the upper left corner of the page is the origin (0, 0), the right direction is the positive direction of the X coordinate, and the lower direction is the positive direction of the Y coordinate. The fourth column is writing pressure. In Table 1, the larger the numerical value, the stronger the writing pressure.

  Then, in Table 1, an almost straight line segment is drawn in the horizontal direction in 0.1 seconds starting from the coordinates (100, 50), and the writing pressure increases toward the right end. You can see that it is recorded.

  As described above, the present invention is characterized in that handwritten information input as comment information is stored in a data format that is expressed as a change with time of pen coordinates. As a format that can be expressed as a change in pen coordinates over time, there is a method of storing a change amount of a vector of pen coordinate positions in addition to storing a data series of pen coordinates at regular intervals.

Furthermore, by registering other strokes such as ST1 and ST2 in association with the stroke of ST1 and the list structure, it is possible to manage several strokes as a group of characters.
In this case, in order to manage as one character, character IDs that combine several strokes starting from ST0 into one may be hierarchically configured. For example, since the capital letter “G” consists of three strokes, the following data structure can be adopted.
In this case, the entity of the character ID “CH0” is an uppercase “G”, and three configured strokes are configured from ST0 to ST2 with a linear pointer list or the like (and “NULL” is followed by ST2). Is stored). Each stroke data has a data structure as shown in Table 1.

  FIG. 6 shows an operation flow of the comment sharing system in one embodiment of the present invention. In step S601, the content display apparatus 106 is turned on, and a software application such as an electronic book viewer is activated. In S <b> 602, a request for downloading content such as an electronic book is transmitted from the content display device 106 to the content server 105 (or 205).

  In step S603, the content server 105 (or 205) determines whether the content requested by the content display device 106 is permitted to be downloaded. For example, when a download request is made (S602), the content display device 106 transmits the content purchase authentication ID and password together with the content server 105 (or 205) to check whether download is possible. be able to. If it is determined in S603 that the download is possible, the process proceeds to S604. If it is determined that the download is not possible, the process is terminated by returning a message indicating that the download is not permitted, as necessary, to the content display device 106. (S614).

  In S604, the content permitted to be downloaded from the content server 105 (or 205) to the content display device 106 is downloaded according to a predetermined transmission protocol. At this time, the downloaded content is encrypted as necessary. The contents include electronic books, electronic magazines, electronic information such as videos and TV programs, articles and advertisements.

  In step S605, the user operates the content display device 106 to start browsing the content. At this time, the user can, for example, turn pages of an electronic book or an electronic magazine, jump to a specific page, play a moving image, or the like via a user interface (not shown) of the content display device 106. Hereinafter, specific examples of browsing will be exemplified.

  When browsing is started, the process proceeds to S606, where the user displays content (page) via a user interface (not shown) and waits for an event to occur via the user interface (S607). Occurrence of an event includes touch (pressing) of a button on the screen with a pen or a finger, voice input by a user's utterance, etc., posture input such as changing the direction of the content display device, and acceleration input.

  When an event occurs (Yes in S607), the content display device 106 analyzes the details of the event and performs processing according to the details. 6 includes, for example, page turning (S608), comment search (S609), comment assignment (S610), item selection (S611), reservation (or purchase) (S612), and application termination (S613). .

  Page turning (S608) is image processing for turning the page of the content screen displayed on the display of the content display device 106. By this process, the user can refer to, for example, an electronic book downloaded to the device 106 and in a browsing state with the same usability as referring to an actual book. For example, when the user wants to refer to the next page, the user can trace the information display area on the display from left to right with a pen or index finger as if turning a page of a paper book. The display screen is updated to the contents of the next page with a turning image.

  The comment search (S609) is a process of searching for a comment added or added to the electronic information by the user. The comment group given by the user, for example, directly or indirectly with the content server 105 (or 205) so that it can be understood by whom to which item on which page of which content is added or added. It can be stored in the linked DB server 103 (or 203) (necessary tags and items for the index are added sequentially). Thus, comments stored in the database can be searched according to a user instruction by inputting a keyword or the like, and can be displayed on the display of the content display device 106 as necessary.

  Comment addition (S610) is processing for adding or adding a comment by the user to electronic information. A comment group added or added by the user is, for example, a DB connected to the content server 105 (or 205) so that it can be understood who added or added to which item of which page of which content. Accumulated in the server 103 (or 203). As described above, these accumulated information can be searched not only by the user who posted the comment but also by other users.

  The item selection (S611) is performed when an item (text or image, or a product or service indicated by such text or image) included in the content displayed on the display of the content display device 106 is selected by the user. The process of recording which item on which page of which content is selected by whom. In the item selection (S611), it is possible to perform a cooperative operation for other processing (for example, comment search, comment addition, reservation, purchase, etc.) based on this record. These cooperative operations are performed in the content server 105 (or 205).

  In the reservation (or purchase) (S612), the selected item is reserved or purchased by electronic commerce. It is possible to record which items are reserved or purchased by whom, and to perform necessary settlement processing using a settlement system (not shown). Since this payment system is not necessarily essential in the system according to the present invention, a detailed description is omitted. That is, in the system according to the present invention, when the user instructs reservation or purchase for any item or service, the role as the system according to the present invention is transmitted by transmitting a settlement processing request to another settlement system as necessary. Is sufficient.

  When the above-described processes (for example, S608 to S612 described above) are completed, the process returns to S606 to display necessary content (page display) as appropriate and waits for further events by the user (S607).

  Then, when an application termination (S613) is instructed, content browsing as an application program is terminated (S614).

  FIG. 7 shows an operation flow of the comment sharing system according to the embodiment of the present invention. In the flow shown in FIG. 7, for a content such as an electronic book displayed on the content display device 106, a comment or memo handwritten by a user with a pen or the like as electronic data is sent via the API server 101 (or 201) to the INK server 102. The flow to send to is explained.

  When the process starts in S701, the process proceeds to S702, and it is detected that a handwritten stroke is started on the display on the content display device 106. Once the handwriting stroke is started, the process proceeds to S703, and it is determined whether or not the stroke is continued every certain time (for example, 10 milliseconds). The determination of continuation of the stroke can be made based on whether or not touch coordinates with the pen exist. If it is determined in S703 that the stroke is continued, the process proceeds to S704, and pen coordinates for each predetermined time are transmitted to the INK server 102 via the API server 101 (or 201). At this time, it is good also as transmitting the pen pressure level of a pen as needed. If it is determined in S703 that the stroke is not continued, the process proceeds to S705.

  In S705, the pen coordinate series transmitted so far is registered as one stroke. In the above example, a series of pen coordinates (and writing pressure levels) grouped as “ST0” in Table 1 is handled as one stroke.

  Next, it progresses to S706 and it is judged whether there exists any further stroke. For example, it is determined that the second stroke has started when a new stroke is started within a predetermined time within the treatment area (such as a rectangular area in the display screen). If it is determined in S706 that there is a new stroke, the process returns to S703 to determine whether or not the second stroke is continued. In S706, when it is determined that there is no new stroke (for example, no stroke is generated within a predetermined time), the process proceeds to S707, and one or more strokes (groups) up to that point are registered as characters or symbols, Proceeding to S708, the series of processing ends.

  In addition, although it demonstrated as what transmits a coordinate for every fixed time in S704, this invention is not limited to this, The pen coordinate data sampled for every fixed time (and writing pressure level data if necessary) Can be stored in the content display device 106 for a certain period of time, and when a certain amount of data has been stored, it can be transmitted collectively to the INK server 102.

Next, how the present invention is implemented while the user uses the content display device 106 will be described based on more specific examples shown in FIGS. In the example described later, two terminal users appear. One is user A who should be a student, and the other is user B who should be a teacher. User A uses the content display device 106a (see FIG. 2) to download electronic book content containing English practice questions from the content server 205. Next, an English exercise displayed on the electronic book content is answered using a pen or the like. The answered data is uploaded to the INK server 102 via the API server 201 as handwritten data. Next, the user B who is a teacher calls the exercise problem that has been answered by the user A from the API server 201 using the content display device 106b (see FIG. 2). Then, the correction to the answer of the user A is performed. This correction work is also performed by handwriting with a pen or the like, and checks, comments, etc., which are correction strokes, are uploaded to the INK server 102 via the API server 201 as handwritten data.
A series of these states will be described with reference to FIGS.

FIG. 8 shows a state in which user A downloads and displays English practice question content on the content display device 106a. In FIG. 8, an English exercise 801 is displayed on the display 1062 of the apparatus 106 a, and the user A is in a state where the pen tip is placed on the question 3 -B- (2) using the pen 899. Reference numeral 890 denotes a home button.
The question sentences shown in FIG. 8 are extracted as follows.
“Question 3: Answer the following questions (A) and (B).”
“(A) Choose one of the words / phrases that is most appropriate for the next sentence, and answer with a sign.”
“(B) Read the following dialogue between A and B, choose the word or phrase that is most appropriate for the empty space, and answer with a symbol.”
(B) Problem sentence of (3) “A: Do you mind if I use your computer for a while?”
(B) Option (3) “A. Go ahead. I. Please don't. C. It's nice of you. D. I'm sorry I can't.”

  In FIG. 9, the user A (student) has already finished answering the exercise. User A answers each question directly to the liquid crystal tablet by handwriting. According to FIG. 9, the answer of user A to question 3 (A) (1) is “A” (901), and the answer of user B to question 3 (A) (2) is “I”. (902).

  Here, the handwritten data answered by the user A is associated with the content ID, the page ID, and the like of the exercise problem, and is transmitted to the INK server 102 as one data structure.

In FIG. 10, the user B (teacher) downloads and corrects the exercises that have been answered by the user A to the content display device 106b. In FIG. 10, first, electronic data about an exercise is downloaded, a layer having handwritten answer data by the user is downloaded and overwritten, and correction by the user B is performed by handwriting. That is, the correction by the user B is transmitted to the INK server 102 by forming a layer different from the layer by the user A. The layer having the correction data by the user B is also managed by the INK server 102 in association with the content ID and page ID of the exercise problem. In FIG. 10, question 3 (A) (1) has a correct answer and a circle (1001) is handwritten, and question 3 (A) (1) has an incorrect answer and a check mark (1002). Is handwritten, and a check mark is also placed on the correct branch (1003). Since questions 3 (A) and (4) are also incorrect answers, a check mark (1004) is entered at the correct position “c”. And as a total score, it is written by hand that it is 7 points out of 10 points (1005).
As for the check mark for correction, it should be noted that in Japan, the correct answer is a circle and the incorrect answer is a check mark, but in Europe and the United States, the correct answer is a check mark.

  In FIG. 11, the user A reads the correct answer data and overwrites it without receiving correction by the user B. The correct answer data also forms one independent layer, and is managed in the INK server 102 in association with the content ID and page ID of the exercise problem. And it is downloaded as correct data according to a user's request. In FIG. 11, the correct answer 1101 in question 3 (A) (1), the correct answer 1102 in question 3 (A) (2), the correct answer 1103 in question 3 (A) (3), etc. are overwritten and displayed. Thereby, since the user can confirm the correct answer of each question instantly with his / her answer, he / she can easily confirm his / her performance.

  The above-described handwritten data and correct data for each layer can be displayed in different colors by registering and managing together with color attributes (not shown).

  As in the present invention, if the INK server 102 is configured to manage handwritten data in time series together with writing pressure, in addition to the above-mentioned answer and correction of English teaching materials, Practice writing, correcting and testing kanji writing (including writing order). Also, managing the pen pressure is useful for teaching calligraphy over the network.

  Further, FIG. 12 shows a display example of a content display device constituting a system according to another embodiment of the present invention.

  In FIG. 12, a handwritten area in the answer sheet 802 displayed on the display 1062 is allocated in advance. For example, the name is entered in a rectangle in the name field 1201, and a corresponding number is assigned to the answer to each question, and is designated as 1202 or 1203, for example. In FIG. 12, a plurality of numbers are entered in the answer column 1204, but such mathematical formulas and symbol strings can also be transmitted to the INK server 102 as handwritten information. Further, it may be configured such that characters, numbers, mathematical formulas, symbols, etc. entered in the answer column are recognized and transmitted to the INK server 102.

  FIG. 13 shows a display example of a content display device constituting a system according to another embodiment of the present invention. In the figure, an article 803 relating to a car is displayed on the display 1062, and the user can underline the part of interest (1301) or enter a comment (1302). These underlines and comments are transmitted to the INK server 102 as handwritten data together with pages and position coordinates in the magazine. Also in this case, the handwritten comment can be transmitted to the INK server 102 after character recognition in advance.

  Next, an example of the operation of a system according to another embodiment of the present invention will be described with reference to FIGS. In this operation example, the handwriting (sign) input by the user by handwriting on the content display device is managed in the system, and thereafter, the user's signature input can be recognized.

  FIG. 14 exemplifies two handwritten strokes (starting from A and starting from B) that are even input by an input device such as a pen on the display 1062 of the content display device 106. In the system according to the present invention, as described later, these handwritten strokes are learned and managed, and authentication is performed.

(Management Variable) First, in the system according to the present invention, the following variables are typically used for storing, managing, and authenticating handwritten data. These variables do not limit the gist of the present invention at all, and are merely illustrated to satisfy the requirements for enabling the invention.
X: abscissa Y: ordinate n: number of times of sampling (0 to 5)
t: Time (0.01 second unit)
tc: writing time P: writing pressure CH: handwritten data CF0: first time (0th) sampling data (stroke, time, pen coordinates, writing pressure)
CFn: nth sampling data (stroke, time, pen coordinates, writing pressure)
CF'n: Data obtained by enlarging and converting the nth sampling data (stroke, time, converted pen coordinates, writing pressure)
CF ″: authentication source data <average of 5 sampling data (stroke, time, pen coordinates, writing pressure)>
STn: n stroke th data ST _MAX: Maximum number of strokes XD: the horizontal length of the sampling data YD: vertical length of the sampling data Tn: Duration painted stroke b: Number of time obtained by equally dividing the stroke ( 0-d)

(Handwritten data storage processing)
In FIG. 15, when the handwritten data storage process starts (S1501), first, each of the variables is initialized (S1502). In particular, here, the variables CH, ST, t, and tc are initialized. In step S1503, the position of the pen on the tablet and the writing pressure are detected, and it is determined whether the writing pressure P is greater than Pw (S1503). If the writing pressure P is less than or equal to Pw (No in S1503), the writing time is counted up (S1504).
If the writing pressure P> Pw, the process proceeds to S1505.

  In S1501, it is determined whether or not the brush release time tc is 0. If Yes, the process proceeds to S1506, and the pen is still touching the tablet. , Time t, pen coordinates (X, Y), and writing pressure P are stored, and the process proceeds to S1507 to count up t. The unit of the count up time is typically 0.01 to 0.1 seconds. Thereafter, the process returns to S1503.

  If it is determined in step S1505 that the rewriting time is not tc = 0, the process proceeds to step S1508, and in order to determine whether the stroke has been rewritten with the same character or the character has been written, a rewriting time tc is determined in advance. When the threshold value tn is compared and tc is smaller than tn (Yes in S1508), it is assumed that the stroke has been repeated with the same character, the stroke ST is counted up (S1509), and t and tc are cleared (S1510). , CH stores the stroke number ST, time t, pen coordinates (X, Y), and writing pressure P (S1506), and counts up t (S1507). Then, the process returns to S1503.

  Next, in S1508, the rewriting time tc is compared with the threshold value tn. If tc is equal to or greater than tn (Yes), it is determined that one stroke has been written, and a NULL value is entered at the end of the CH. (S1511), the process ends (S1512).

  As described above, the handwritten data input to the content display device 106 is stored in the storage unit of the system.

(Sampling process)
Next, the sampling process performed with respect to the stored handwritten data will be described. In FIG. 16A, when the sampling process starts (S1601), first, the sampling count n is initialized (S1602). Next, handwritten data is stored as CH by the above-described processing (S1603). In step S1604, the handwritten data CH is copied as the nth sampling data CFn.

In S1605, it is determined whether or not sampling is the first time. If the first time of sampling (No in S1605), the process proceeds to S1606, it is compared with the maximum number of strokes ST _MAX of the maximum number of strokes ST _MAX and n-th sampling CFn initial sampling CF0, stroke, i.e. the number of strokes are different In the case (No in S1606), an error is displayed (S1607), and the process is terminated (S1608).

If it is the first sampling (Yes in S1605), or if the number of first and nth strokes is the same (Yes in S1606), the process proceeds to S1609 and the maximum X coordinate among the points of CFn is set to X _MAX , The maximum Y coordinate is extracted as Y _MAX , the minimum X coordinate is extracted as X _min , and the minimum Y coordinate is extracted as X _min (S1609), and the horizontal length XD and the vertical length YD of the sampling data CFn are calculated (S1610).

Next, proceeding to S1611, the magnitude relationship between the horizontal length XD and the vertical length YD is determined, and if XD is greater than YD or the vertical and horizontal lengths are equal (Yes in S1611), In step S1612, the set of points of the sampling data CFn is enlarged and converted to the full width while maintaining the same aspect ratio, and stored in CF′n. On the other hand, if YD is larger than XD (No in S1611), the process proceeds to S1613, and the set of points of the sampling data CFn is enlarged and converted to the full length while maintaining the same aspect ratio, and CF′n To store.
When the conversion processing is completed, n is counted up (S1614). If n is smaller than 5 (Yes in S1615), the nth sampling processing is repeated (S1603 to S1614).

  When sampling is completed five times (0 to 4) (Yes in S1615), the process proceeds to S1616, where n and ST are initialized, and the time Tn required to draw each stroke ST of CF'n is divided into d equal parts. The data of the points equally divided in time is re-extracted as appropriate under the adjustable loop condition (S1617 to S1619), and the re-extracted time of each stroke ST of CF′0 to CF′4 An average CF ″ of the coordinates of the points at the time of (b · Tn / d) is calculated, and when averaging is performed, points that are more than a certain distance are excluded from the average of the five points, and the average is re-acquired (S1620 to S1623) ). Note that S1620 to S1623 are processed based on an adjustable loop condition.

The data structure of the reference source CF ″ is exemplified below.

(Authentication process)
Next, a process for authenticating a signature or the like by collating with handwritten data input separately based on the sampling data will be described in detail below.

  In FIG. 17A, when the authentication process starts (S1701), handwritten data is stored as CH based on the above-described process (S1702).

Then, the process proceeds to S1703, compares the maximum number of strokes ST _MAX handwritten data CH entered the maximum number of strokes ST _MAX and S1702 of the original data CF '' are compared, in the case where the number of strokes is different from the determination (S1703 No) goes to S1721, and ends the process as authentication NG (S1722).
On the other hand, if it is determined in S1703 that the number of strokes is the same, the process proceeds to S1706, and the handwritten data CH is copied as the comparison destination data CT.

Next, the process proceeds to S1707, and among the CT points, the maximum X coordinate is extracted as X _MAX , the maximum Y coordinate is extracted as Y _MAX , the minimum X coordinate is extracted as X _min , and the minimum Y coordinate is extracted as X _min , and the process proceeds to S1708. The horizontal length XD and the vertical length YD of the comparison target data CT are calculated.

Next, the processing proceeds to S1709, where the magnitude relationship between the horizontal length XD and the vertical length YD is determined, and when the horizontal length XD is greater or the vertical and horizontal lengths are determined to be equal (S1709). In step S1710, the set of points of the comparison target data CT is enlarged and converted to the full width while maintaining the same aspect ratio, and stored in CT ′.
If it is determined that the vertical length YD is greater than the horizontal length XD (No in S1709), the process proceeds to S1711, and the set of points of the comparison target data CT maintains the same aspect ratio while maintaining the same aspect ratio. Fully enlarged and converted and stored in CT ′.

When the above conversion processing is completed, the process proceeds to S1712, where ST is initialized, the time Tn required to draw each stroke ST of CT ′ is equally divided into d, and under an adjustable loop condition, If necessary, re-extract the data of the points equally divided in time (S1713 to S1715), compare the comparison source data CF ″ with the converted comparison destination data CT ′, calculate the difference S, and add up (S1716 to S1718). Note that S1716 to S1718 are processed based on an adjustable loop condition.
In S1717, S = S + 0 is set up to a certain distance L, and S = S + lx ² when the distance L is more than that. If the difference in writing pressure difference P is too large, S = S + 10 can be processed. .

  Next, proceeding to S1719, the degree of difference S is compared with a threshold SI for determining similarity / dissimilarity. If S is larger than SI (No in S1719), processing is appropriately performed as authentication NG (S1721), the processing is terminated (S1722), and if S is smaller than SI (Yes in S1719), processing is appropriately performed as authentication OK. (S1720), and the process ends (S1722).

  The program modules constituting the applications and software implemented in the present invention can be executed by a plurality of computers connected to the network in all or part of the execution steps, and cooperate among a plurality of computers via the network. It operates as a feature.

  In addition, various types of removable media that can store various data such as content data can be used as recording media used in the present invention. For example, various optical disks such as DVD-R, DVD-RW, DVD-RAM, CD-R, CD-RW, and magneto-optical disk, magnetic disks such as flexible disk and hard disk, and various semiconductor memories. As the recording medium, for example, a recording medium with a copyright management function that restricts copying and reproduction of content data using an encryption key or the like can be used.

  The computer of the present invention described above is not limited to pure hardware such as a CPU, and may include firmware, an OS, or peripheral devices.

  As described above, each configuration of the present invention may be implemented as software or hardware at a level that can be implemented by those skilled in the art at the time of filing.

100 Comment sharing system 101 API server 102 Handwritten information management server (INK server)
103 DB server 104 Content management system 105 Content server 106a-106b Content display apparatus

Claims (15)

A content server for storing and managing electronic content, a plurality of content display devices connected to the content server via a network and capable of downloading the electronic content, and an electronic content page displayed on the content display device A comment sharing system including a comment information management server for storing and managing the comment information,
The content display device transmits the comment information given to the electronic content to the comment information management server as handwritten data expressed as a change in coordinates over time,
The comment information management server stores the handwritten data.   The system according to claim 1, wherein the comment information management server manages the handwritten data for each stroke.   The system according to claim 1, wherein the comment information management server manages the handwritten data together with writing pressure data.   The system according to claim 1, wherein the content display device displays a predetermined area for inputting comment information given to the electronic content on the electronic content.   The system according to claim 1, wherein the comment information management server manages the handwritten data as a different layer for each of the plurality of content display devices. A content server for storing and managing electronic content, a plurality of content display devices connected to the content server via a network and capable of downloading the electronic content, and an electronic content page displayed on the content display device A comment sharing system in a comment sharing system including a comment information management server that stores and manages the comment information.
Sending the comment information given to the electronic content from the content display device to the comment information management server as handwritten data expressed as a change in coordinates over time;
Storing the handwritten data in the comment information management server.   The method according to claim 6, wherein the comment information management server further comprises managing the handwritten data for each stroke.   The method of claim 6, wherein the comment information management server further comprises managing the handwritten data together with writing pressure data.   The method according to claim 6, wherein the content display device further includes a step of displaying, on the electronic content, a predetermined area for inputting comment information given to the electronic content.   The system according to claim 6, wherein the comment information management server further includes a step of managing the handwritten data as a different layer for each of the plurality of content display devices. A content server for storing and managing electronic content, a plurality of content display devices connected to the content server via a network and capable of downloading the electronic content, and an electronic content page displayed on the content display device A computer program executed in a comment sharing system including a comment information management server that stores and manages the comment information,
From the content display device, transmitting comment information given to the electronic content to the comment information management server as handwritten data expressed as a change in coordinates over time;
The comment information management server includes a step of storing the handwritten data.   The program according to claim 11, wherein the comment information management server further includes a step of managing the handwritten data for each stroke.   The program according to claim 11, wherein the comment information management server further includes a step of managing the handwritten data together with writing pressure data.   The program according to claim 11, wherein the content display device further includes a step of displaying, on the electronic content, a predetermined area for inputting comment information given to the electronic content.   The program according to claim 11, wherein the comment information management server further includes a step of managing the handwritten data as a different layer for each of the plurality of content display devices.