JP2011253307A - Information processing system and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an information processing system capable of synchronizing handwritten strokes between PC devices and displaying the handwritten strokes without impairing operability, and a program for the system.SOLUTION: The information processing system includes a PC device, an input device electrically connected to the PC device, and a server device connected to the PC device through a communication network. First drawing means of the PC device performs drawing of a stroke on display means on the basis of entry information transmitted from the input device, and second drawing means of the PC device performs drawing on the display means on the basis of object information received from the server device by object information reception means. When the first drawing means performs drawing of a handwritten stroke on the basis of the entry information, the second drawing means delays drawing on the basis of the object information until drawing of the handwritten stroke is completed.

Description

The present invention relates to a technique for sharing handwritten stroke information between remote PC devices.
In recent years, electronic pens that digitize written information have been developed, and “Anot pen” developed by Swedish company Anoto is known as a representative one. Anotopen is used with dedicated paper on which a predetermined dot pattern is printed. In addition to the usual ink cartridge for writing characters, etc., the anotopen has a small camera for capturing the dot pattern printed on the dedicated paper and the position coordinates on the dedicated paper from the captured dot pattern. A processor for calculating and a data communication unit for transmitting the calculated position coordinates and the like to an external device are mounted. When the user writes characters on the special paper with an anotopen or puts a check mark on the image designed on the special paper, the small camera captures the dot pattern printed on the special paper as the pen moves. The entry information such as characters and images written by the user is recognized from the continuous position coordinates calculated by the processor. Then, this entry information is transmitted from the Anotopen by the data communication unit to a nearby terminal device such as a personal computer or a mobile phone (for example, see Patent Document 1).

  Patent Document 2 discloses a paper-based conference service management tool for a conference support function. Specifically, the paper-based tool includes a digital pen that can be used to generate a stroke, which is transmitted from the digital pen to a service that provides a conference support function. The paper-based tool further includes digital paper having an image pattern in the lower layer. The digital pen recognizes the underlying dot pattern and sends its position information to the computer via a Bluetooth connection to determine its position on the digital paper. The digital paper has an image pattern in the lower layer, and includes a free-form drawing pad, a meeting place matching portion, and a user matching portion. The digital pen recognizes the underlying dot pattern and sends its position information to the computer via the Bluetooth connection to determine its position on the paper. The pen location and time stamped pen travel path is then mapped to a command corresponding to a particular conference service. In addition, the digital pen has a pressure activated switching function for mode transition between pointer mode and visible stroke mode. Patent Document 3 discloses a technique for realizing calibration by obtaining a coordinate conversion function for converting a coordinate system. Further, Patent Document 4 describes a laminated structure of transparent sheets on which a dot pattern readable by a digital pen for printing is printed.

Japanese translation of PCT publication No. 2003-511761 JP 2008-217782 A Registered Utility Model No. 3151886 JP 2008-269545 A

  In a remote conference or the like, it may be convenient for a user to share handwritten stroke information between PC devices. Specifically, a server device is provided on the network so that handwritten stroke information can be shared smoothly between PC devices, handwritten stroke information is exchanged via the server device, and handwritten stroke display is synchronized. An information processing system to be used is considered. In this information processing system, the PC device may receive information on the handwritten strokes of the users of other PC devices during the entry of the handwritten strokes based on the connected input device. At this time, if the display is changed based on the handwritten stroke information received from another PC device, there is a problem that the handwritten stroke being written is erased by the input device or the display is flickering while writing by the input device. Therefore, an object of the present invention is to provide an information processing system capable of displaying handwritten strokes in synchronization between PC apparatuses without impairing operability, and a program provided for the system.

  An information processing system according to the present invention includes a PC device, an input device that is electrically connected to the PC device and generates entry information related to a handwritten stroke, a server device that is connected to the PC device via a communication network, An information processing system comprising: display means; entry information receiving means for receiving the entry information from the input device; and first drawing means for drawing on the display means based on the entry information. And object information receiving means for receiving object information, which is information related to drawing of each handwritten stroke from the server device, and drawing on the display means based on the object information received by the object information receiving means. Drawing means, wherein the second drawing means is based on the entry information. If the product has been drawn handwritten strokes, delaying the rendering based on the object information to the drawing of the handwritten stroke is completed.

  According to the information processing system, the first drawing unit of the PC device draws a stroke on the display unit based on the entry information transmitted from the input device, and the second drawing unit of the PC device is operated by the object information receiving unit. Drawing is performed on the display means based on the object information relating to the handwritten stroke received from the server device. Here, when the first drawing means is drawing the handwritten stroke based on the entry information, the second drawing means delays the drawing based on the object information until the drawing of the handwritten stroke is completed. By doing in this way, the PC device prevents the display from being updated based on the object information received from the server device while the user is filling in with the input device, and the display flickering or handwriting stroke while writing the handwritten stroke. Can be suppressed. Therefore, according to the information processing system described above, it is possible to display handwritten strokes in synchronization between the PC devices without impairing operability.

  In one aspect of the information processing system, the PC device transmits object information to the server device and object information generation means for generating object information for each stroke of the handwritten stroke based on the entry information. Object information sending means, and object information inquiry means for making an inquiry about the presence or absence of object information sent from another PC device at predetermined intervals to the server device, wherein the object information receiving means When the object information transmitted from the PC device exists in the server device, the object information is received from the server device. In this aspect, each PC device in the information processing system can smoothly exchange object information from the server device, and can display handwritten strokes synchronously.

  In another aspect of the information processing system, the display unit displays a plurality of pages, the object information includes a page number specified from the entry information, and the second drawing unit includes the object When the page number included in the information matches the page number of the page being displayed by the display means, the object information is drawn based on the object information. In this aspect, when there are a plurality of pages displayed by the display unit, the PC device accurately determines whether or not the handwritten stroke corresponding to the object information should be drawn based on the page number included in the object information. Can do.

  In another aspect of the information processing system, the object information includes line type information specified from the entry information, and the second drawing unit is configured to perform the object information on the object information based on the line type information. Draw based on. Here, “line type information” corresponds to the line width, line color, etc. of the handwritten stroke drawn on the display means of the PC device. Thus, by including the line type information in the object information, it is possible to synchronize the display of each PC device including the line type of the handwritten stroke.

  In another aspect of the information processing system, the input device is an electronic pen that reads a coded pattern from an input sheet on which the coded pattern is formed, and the second drawing unit includes the entry information receiving unit. When the received entry information indicates pen-up, it is determined that the drawing of one stroke corresponding to the electronic pen that transmitted the entry information has been completed. By doing in this way, when the input device is an electronic pen, the PC device can accurately grasp the timing of drawing a handwritten stroke based on the received object information, and can suppress a decrease in operability. it can.

  In another aspect of the information processing system described above, when the input device is a mouse and the second drawing unit determines that the left click of the mouse has been completed, the second drawing unit performs one stroke corresponding to the mouse. It is determined that the drawing of the handwritten stroke has been completed. By doing in this way, when the input device is a mouse, the PC device can accurately grasp the timing of drawing a handwritten stroke based on the received object information, and can suppress a decrease in operability. .

  In another aspect of the information processing system, the object information generation unit generates a smooth curve based on coordinate points indicated by each entry information corresponding to one stroke of the handwritten stroke, and the object information based on the curve Is generated. In this way, the PC device can draw a handwritten stroke with a smooth curve based on the received object information.

  In another aspect of the information processing system, the object information is a coordinate system based on a page on which the handwritten stroke is displayed by the display unit from a coordinate system of a coordinate point indicated by each entry information. A set of coordinate points constituting the curve converted into. In this way, the PC device converts each coordinate point constituting the curve into a coordinate system based on the page on which the handwritten stroke is to be displayed as a reference, so that each coordinate point constituting the curve is relative to the page. Specific positions can be defined. Therefore, in this aspect, even when the PC device generates object information by another PC device having a different screen resolution of the display means, the handwriting stroke is appropriately performed based on the object information without causing misalignment or the like. Can be displayed.

  In another aspect of the information processing system, the second drawing unit converts the coordinate point corresponding to the handwritten stroke included in the object information into a smooth curve, and then draws the curve on the display unit. Do. Also according to this aspect, the PC device can draw a handwritten stroke with a smooth curve based on the received object information.

  In another aspect of the information processing system, the PC device further includes a mode switching unit that switches a setting as to whether or not the remote conference mode is set based on an external input, and the object information receiving unit is configured to switch the remote conference mode. Only when the object information is received.

  The program according to the present invention is a program that is electrically connected to an input device that generates entry information related to handwritten strokes, is connected to a server device via a communication network, and is installed and executed in a PC device that includes display means. Completion information receiving means for receiving the entry information from the input device, first drawing means for drawing on the display means based on the entry information, and information relating to drawing of each handwritten stroke from the server device The PC apparatus functions as an object information receiving means for receiving object information, and a second drawing means for drawing on the display means based on the object information received by the object information receiving means, wherein the second drawing means The first drawing means draws a handwritten stroke based on the entry information. If it is, delaying the rendering based on the object information to the drawing of the handwritten stroke is completed.

  In one aspect of the program, object information generating means for generating object information for each stroke of the handwritten stroke based on the entry information, object information transmitting means for transmitting the object information to the server apparatus, and the server apparatus The PC device further functions as an object information inquiry means for inquiring the presence / absence of object information transmitted from another PC device at predetermined intervals, and the object information receiving means transmits from the other PC device. When the object information that has been set exists in the server device, the object information is received from the server device.

  In another aspect of the program, the display unit displays a plurality of pages, the object information includes a page number specified from the entry information, and the second drawing unit includes the object information. When the included page number matches the page number of the page being displayed by the display means, an image based on the drawing object information is performed on the object information.

  In another aspect of the program, the object information includes line type information specified from the entry information, and the second drawing unit is based on the object information based on the line type information for the object information. Draw.

  In another aspect of the program, the input device is an electronic pen that reads an encoded pattern from an input sheet on which the encoded pattern is formed, and the second drawing unit is received by the entry information receiving unit. When the entry information indicates pen-up, it is determined that the drawing of one stroke corresponding to the electronic pen that transmitted the entry information has been completed.

  In another aspect of the above program, when the input device is a mouse, and the second drawing means determines that the left click of the mouse has been pressed, handwriting for one stroke corresponding to the mouse is performed. It is determined that stroke drawing has been completed.

  In another aspect of the program, the object information generation unit generates a smooth curve based on coordinate points indicated by each entry information corresponding to one stroke of the handwritten stroke, and generates object information based on the curve. To do.

  In another aspect of the program, the object information is converted from a coordinate system of coordinate points indicated by the entry information into a coordinate system based on a page on which the handwritten stroke is displayed by the display unit. A set of coordinate points constituting the curved line.

  In another aspect of the above program, the second drawing unit converts the coordinate point corresponding to the handwritten stroke included in the object information into a smooth curve, and then draws the curve on the display unit.

  In another aspect of the program, the PC device is further functioned as a mode switching unit that switches a setting as to whether or not the remote conference mode is set based on an external input, and the object information receiving unit is configured to operate in the remote conference mode. Only receive object information.

  The printing system according to the present invention can be configured by installing these programs in a PC device and causing them to function.

  According to the present invention, the PC device exchanges handwritten stroke object information from the server device on the network so that the handwritten stroke information can be smoothly shared between the PC devices. The handwritten stroke display can be synchronized with the PC device. Further, when the handwritten stroke is drawn based on the entry information transmitted from the input device, the PC device delays the drawing based on the object information until the drawing of the handwritten stroke is completed. Can be prevented from being updated based on the object information received from the server device. Therefore, according to the information processing system described above, even when the object information generated by another PC device is received during the entry of the handwritten stroke, the display during the entry of the handwritten stroke flickers or is being written. It is possible to prevent the handwritten stroke from being erased, and to suppress a decrease in operability. Therefore, according to the information processing system described above, it is possible to display handwritten strokes in synchronization between the PC devices without impairing operability.

It is a system configuration figure of the information processing system in a 1st embodiment. It is explanatory drawing which shows the relationship between the arrangement | positioning of the dot in a dot pattern, and the value converted. (A) shows a dot pattern typically, and (b) is a figure showing an example of information corresponding to it. It is the schematic which shows the structure of an electronic pen. It is a figure which shows typically the data structure of the entry information transmitted to a PC apparatus from an electronic pen. It is the table | surface explaining the symbol used in FIG. It is a schematic block diagram of PC apparatus and a server apparatus. It is a figure which shows the data structure of object information typically. It is a figure which shows the data structure of coordinate definition information typically, Comprising: (a) is the coordinate definition information memorize | stored by PC apparatus 2A, (b) illustrates the coordinate definition information memorize | stored by PC apparatus 2B. FIG. (A) is a database defining the relationship between page numbers and background image information, and (b) is a database defining line width information corresponding to the line type change instruction area. This is a database in which an input device, a pen ID, an entry state, a start coordinate, and an end coordinate are associated. It is a database in which object information, a transmission source of object information, and an object ID are associated with each other. It is an example of a display of the display application displayed on the display of PC apparatus. 5 is a flowchart illustrating a processing procedure on the object information transmission side in the first embodiment. 4 is a flowchart illustrating a procedure of real-time drawing processing in the first embodiment. 4 is a flowchart illustrating a procedure of object information generation processing and storage processing in the first embodiment. 5 is a flowchart illustrating a procedure of drawing update processing in the first embodiment. 5 is a flowchart illustrating a procedure of display update processing in the first embodiment. 5 is a flowchart illustrating a procedure of object information inquiry and reception processing in the first embodiment. It is an example of a display of a drawing object line. It is an example of a display of a drawing object line. It is a conceptual diagram of the curve produced | generated based on the coordinate point. It is a display example of a curve drawn based on object information. It is an example of a display of the display area in a display application. It is an example of a display of the display area in a display application. It is a flowchart which shows the procedure of the production | generation process and storage process of object information which concern on the modification 1 of 1st Embodiment. It is a flowchart which shows the procedure of the drawing update process which concerns on the modification 1 of 1st Embodiment. It is a flowchart which shows the procedure of the display update process which concerns on the modification 1 of 1st Embodiment. It is a system configuration figure of the information processing system in a 2nd embodiment. The example of a display of the screen in 2nd Embodiment is shown. In 2nd Embodiment, it is a flowchart which shows the procedure of the inquiry and reception process of object information. It is a system configuration figure of an information processing system in a 3rd embodiment. The example of a display of the display application on the display of PC apparatus in 4th Embodiment is shown. In a 4th embodiment, it is a flow chart which shows a procedure of inquiry and reception processing of object information.

  Hereinafter, first to fifth embodiments, which are embodiments of the present invention, will be described in order with reference to the drawings.

<First Embodiment>
First, a first embodiment according to the present invention will be described.

[Configuration of information processing system]
FIG. 1 shows a configuration of an information processing system 10 according to the first embodiment. As shown in FIG. 1, the information processing system 10 according to the first embodiment includes an electronic pen 1 (1A, 1B, 1C) used by a user and an input paper 4 (4A, 4B) on which a dot pattern is printed. And an instruction sheet 5 (5A, 5B), a PC (Personal Computer) device 2 (2A, 2B) that receives and processes the entry information from the electronic pen 1, and the PC device 2 via a communication network such as the Internet. And a server device 3 that performs communication.

  In FIG. 1, such an information processing system 10 is used for a remote conference system. Then, the entry contents of the electronic pen 1 are synchronized between the PC device 2A and the PC device 2B via the server device 3. Hereinafter, each component will be described.

(Input paper)
First, the input paper (electronic pen paper) 4 will be described. As a meeting material, one or a plurality of input sheets 4 are created. An input area 401 is set on the input sheet 4 and is defined by a printer (not shown) with a dot pattern (coding pattern) indicating position coordinates to be described later and a later-described FIG. The background image corresponding to the page number to be printed is printed. The dot pattern is printed with an ink containing carbon that absorbs infrared rays so that the electronic pen 1 can read the dot pattern. Further, in the input area 401, for each page number of the input paper 4, a dot pattern assigned to the page number is printed. The background image is printed on the printed dot pattern with ink that does not absorb in the infrared region so as not to hinder reading of the dot pattern by the electronic pen 1. A dot pattern may be printed on the background image.

(Instruction sheet)
Similar to the input paper 4, a dot pattern is also printed on the instruction paper 5 on substantially the entire surface thereof. In the instruction sheet 5, line type change instruction areas (line type instruction areas) 501 and 502 are set, and the electronic pen 1 instructs the line type of a stroke drawn by the PC device 2. The line type change instruction areas 501 and 502 indicate the line type of the writing content on the input paper 4 drawn by the PC device 2 for the electronic pen 1 tapping the area in accordance with the specified content of FIG. This is an instruction area for changing. In the line type change instruction area 501, a dot pattern associated with the line width “thin” is printed, and in the line type change instruction area 502, a dot pattern associated with the line width “thick” is printed. . Furthermore, on the instruction sheet 5, characters representing a frame line and a line type that divide the instruction area are printed with ink that does not absorb in the infrared region.

(Dot pattern)
Next, an Anoto dot pattern (coded pattern) printed on the input paper 4 and the instruction paper 5 will be described with reference to FIGS. FIG. 2 is a diagram for explaining the relationship between dots of a dot pattern and values to which the dots are converted. As shown in FIG. 2, each dot of the dot pattern is associated with a predetermined value depending on its position. That is, each dot is associated with a value of 0 to 3 depending on which direction the top, bottom, left, or right is shifted from the reference position of the virtual grid (the intersection of the vertical and horizontal lines). The value of each dot can be further converted into a first bit value for the X coordinate and a second bit value for the Y coordinate. The position coordinates of the input sheet 4 and the instruction sheet 5 are determined by the combination of the information thus associated. A dot pattern is formed on the input paper 4 and the instruction paper 5 so that the coordinate areas do not overlap.

  FIG. 3A shows an arrangement of a certain dot pattern. As shown in FIG. 3A, 6 × 6 dots within a range of about 2 mm in length and width form a unique pattern no matter what part of the input paper 4 or instruction paper 5 is taken. Are arranged as follows. The dot pattern formed by these 36 dots holds position coordinates (for example, where the dot pattern is on the input paper 4 or the instruction paper 5). FIG. 3B is a diagram in which each dot shown in FIG. 3A is converted into a value associated with the regularity shown in FIG. 2 according to the shift direction from the reference position of the lattice. This conversion is performed by the electronic pen 1 that captures an image of a dot pattern.

(Electronic pen)
Next, the electronic pen 1 will be described with reference to FIG. FIG. 4 is a schematic diagram showing the structure of the electronic pen 1. As shown in FIG. 4, the electronic pen 1 includes a pen unit 104, an LED 105, a CMOS camera 106, a pressure sensor 107, a processor 108 including a CPU, a memory 109 such as a ROM and a RAM, A communication unit 111 including a real time clock 110, an antenna, and the like, and a battery 112 are provided. The tip of the pen unit 104 is a pen point unit 103, and the user touches the pen point unit 103 of the electronic pen 1 on the input sheet 4 or the instruction sheet 5 and enters a stroke (handwritten stroke). Or tapping (tapping the instruction sheet 5 with the pen tip 103). Here, the first contact of the pen tip portion 103 of the electronic pen 1 with the input paper 4 or the like is referred to as “pen down”, and the removal of the pen tip portion 103 from the contacted (contacted) state “ Called “pen-up”. A trajectory written between the pen-down and pen-up of the electronic pen 1 is one stroke, and characters, figures, etc. are composed of one or a plurality of strokes. The pen unit 104 need not be filled with ink.

  The battery 112 is for supplying electric power to each component in the electronic pen 1. For example, the battery 112 is configured to turn on / off the electronic pen 1 itself by attaching and detaching a cap (not shown) of the electronic pen 1. May be. The real time clock 110 transmits time information indicating the current time (time stamp) and supplies the time information to the processor 108. The pressure sensor 107 detects the pressure applied through the pen unit 104 from the pen point unit 103 when the user writes or taps a character or mark on the input paper 4 or the like with the electronic pen 1, that is, the writing pressure. Is transmitted to the processor 108.

  The processor 108 switches on / off the switches of the LED 105 and the CMOS camera 106 based on the writing pressure data supplied from the pressure sensor 107. That is, when the user writes characters or the like on the input paper 4 or the like with the electronic pen 1, the pen pressure is applied to the pen tip portion 103, and when the pressure sensor 107 detects a writing pressure higher than a predetermined value, the processor 108 It is determined that the user has started entry, and the LED 105 and the CMOS camera 106 are activated. Then, the communication unit 111 detects pen down information (hereinafter referred to as “pen down information PD”) detected by the pressure sensor 107 and identification information (hereinafter referred to as “pen ID”) of the electronic pen 1 described later. Are transmitted to the PC device 2 as entry information. When the user finishes entering one stroke and moves the electronic pen 1 away from the input paper 4 or the like, the pressure sensor 107 detects a pen-up because the writing pressure exceeding a predetermined value is not detected. Then, the communication unit 111 associates the pen-up information detected by the pressure sensor 107 (hereinafter referred to as “pen-up information PU”) and the pen ID, and transmits the information as entry information to the PC device 2.

  The LED 105 and the CMOS camera 106 are attached near the pen tip portion 103 of the electronic pen 1, and an opening 102 is formed in a portion of the housing 101 that faces the LED 105 and the CMOS camera 106. The LED 105 illuminates infrared rays toward the vicinity of the pen tip portion 103 on the input paper 4. The area is slightly shifted from the position where the pen tip 103 contacts the input paper 4 or the like. The CMOS camera 106 is provided with an infrared filter that transmits infrared rays and blocks non-infrared rays. The CMOS camera 106 captures a dot pattern in an area illuminated by the LED 105 and processes image data of the dot pattern as a processor. 108. Here, since carbon absorbs infrared rays, the infrared rays irradiated by the LED 105 are absorbed by the carbon contained in the dots. Therefore, the dot portion has a small amount of infrared reflection, and the portion other than the dot has a large amount of infrared reflection. By photographing with the CMOS camera 106, a threshold value is provided based on the difference in the amount of reflected infrared light, so that a dot region containing carbon can be distinguished from other regions. Therefore, even when a background image is printed on the input paper 4, the printed ink does not absorb in the infrared region, so the processor 108 can recognize the dot pattern. In order to reduce noise and improve the accuracy of coordinate detection, an infrared transmission filter that selectively transmits infrared light may be provided on the light receiving surface side of the CMOS camera 106. Note that the shooting area by the CMOS camera 106 is a range including a size of about 2 mm × about 2 mm as shown in FIG. 3A, and the shooting by the CMOS camera 106 is performed at regular intervals of about 50 to 100 times per second. Is called.

  While the user performs the entry, the processor 108 uses the dot pattern of the image data supplied by the CMOS camera 106 to determine the X and Y coordinates (hereinafter referred to simply as “coordinates”) on the input sheet 4 of the stroke (handwriting) to be entered by the user. Data "or" coordinate information ") is calculated continuously. That is, the processor 108 converts the image data of the dot pattern as shown in FIG. 3A supplied by the CMOS camera 106 into the data array shown in FIG. The data is converted into a Y coordinate bit value, and X and Y coordinate data are calculated from the data array by a predetermined calculation method. Then, the processor 108 associates the current time (time stamp: entered time information) transmitted from the real-time clock 110, writing pressure data, and X and Y coordinate data. Hereinafter, these associated data are collectively referred to as “coordinate attribute information”. Since the 6 × 6 dot pattern on the input sheet 4 or the like does not overlap in the input sheet 4, when the user enters characters or the like with the electronic pen 1, Whether it corresponds to the position can be specified by coordinate calculation by the processor 108.

  The memory 109 stores property information such as a pen ID such as “pen01” for identifying the electronic pen 1, a pen manufacturer number, and a pen software version. Then, the communication unit 111 associates the pen ID, time information (time stamp), writing pressure data, and X and Y coordinate data, and transmits them to the PC device 2 as entry information. Transmission to the PC device 2 by the communication unit 111 is immediately and sequentially performed by wireless transmission such as Bluetooth (registered trademark). Here, one or more pieces of coordinate attribute information generated between the pen-down and pen-up of the electronic pen 1 and transmitted to the PC device 2 are stored as stroke information by the PC device 2. In other words, one stroke is composed of one or a plurality of sets of X and Y coordinates (hereinafter also referred to as “coordinate points”), and the PC device 2 uses the pen-down information PD and the pen-up information PU. One or more coordinate attribute information constituting one stroke is recognized.

  Next, the entry information transmitted from the electronic pen 1 will be described with reference to FIGS. FIG. 5 is a diagram schematically showing entry information transmitted by the electronic pen 1. FIG. 6 is a table explaining symbols used in FIG. When a user writes a stroke on the input paper 4 using the electronic pen 1, first, the electronic pen 1 is brought into contact with the input paper 4. Then, the pressure sensor 107 of the electronic pen 1 detects the writing pressure applied to the pen tip portion 103. When the processor 108 of the electronic pen 1 determines that the writing pressure equal to or higher than the predetermined value is detected by the pressure sensor 107, the pen down information PD indicating the contact of the electronic pen 1 with the input paper 4 and the identification information of the electronic pen 1 are provided. The entry information associated with the pen ID or the like is generated, and the communication unit 111 transmits the entry information to the PC device 2. After the user touches the input paper 4 of the pen tip portion 103 of the electronic pen 1, the user moves the pen tip portion 103 to draw a stroke. The processor 108 continues to obtain the coordinate information (X, Y) obtained by the calculation. , Coordinate attribute information including writing pressure data (P) detected by the pressure sensor 107, time information (T) transmitted by the real-time clock 110, and entry information in association with the pen ID, a dot pattern by the CMOS camera 106 The communication unit 111 sequentially generates the entry information according to the shooting period of time and causes the communication unit 111 to sequentially transmit the entry information to the PC device 2. When the user finishes drawing the stroke and moves the electronic pen 1 away from the input paper 4, the pressure sensor 107 does not detect the writing pressure. Therefore, the processor 108 determines that the writing pressure exceeding the predetermined value is not detected by the pressure sensor 107. Then, entry information in which the pen-up information PU indicating the removal of the electronic pen 1 to the input paper 4 and the pen ID which is the identification information of the electronic pen 1 are generated is generated, and the entry information is entered in the communication unit 111. Information is transmitted to the PC device 2. As described above, a set of coordinate attribute information generated by the electronic pen 1 when the user enters one stroke is referred to as “stroke information”.

(PC device)
Next, the PC device 2A will be described. Since the configuration of the PC device 2B is the same as that of the PC device 2A, the description thereof is omitted except for differences. The PC device 2 </ b> A includes, as hardware, an antenna device capable of data communication with the electronic pen 1, a processor such as a CPU, a memory such as ROM and RAM, a display, a personal computer including a mouse and a keyboard, and the like. . FIG. 7 is a functional block diagram of the PC device 2A. The PC device 2A functionally includes an input unit 21, such as a mouse or a keyboard, a receiving unit 22, a processing unit 24, a storage unit 25, a display unit 26, and a transmission unit 27. The PC device 2 </ b> A performs a predetermined process based on the entry information regarding the input sheet 4 received from the electronic pen 1.

  The receiving means 22 is constituted by an antenna receiving circuit or the like, receives entry information from the electronic pen 1 or receives object information from the server device 3, and transmits the received information to the processing means 24. The display means 26 is constituted by a display or the like, and displays the contents instructed by the processing means 24. The transmission unit 27 transmits a print instruction (output command) for the input sheet 4 and the instruction sheet 5 according to an instruction from the processing unit 24 to the printer. Further, the transmission unit 27 transmits the information instructed by the processing unit 24 to the server device. The data transmission method to the printer or server device 3 may be wired or wireless.

  The processing means 24 is constituted by a processor such as a CPU and controls the entire PC device 2A. Specifically, the processing unit 24 executes a display application 6 described later with reference to FIG. 13, thereby superimposing a predetermined background image on a page (in which input content in the input area 401 </ b> A by the electronic pen 1 is drawn ( Hereinafter, it is also simply referred to as “page”.) The image is displayed on the display means 26. Each page has a page number and is associated with a specific background image by a database shown in FIG. When the processing unit 24 detects a print instruction for the input sheet 4 or the instruction sheet 5, the processing unit 24 generates print information with reference to tables or maps shown in FIGS. A dot pattern indicating a different coordinate area is assigned to each sheet. Hereinafter, the coordinate area of the dot pattern assigned to the input area 401 or the coordinate space of the input paper 4 is referred to as “the coordinate system of the input area 401”. The coordinate information included in the entry information is defined by the coordinate system of the input area 401.

  Further, when the PC device 2A receives the entry information generated by reading the dot pattern when filling in the input paper 4 with the electronic pen 1, the processing unit 24 calculates the figure from the coordinate information included in the entry information. 9. Referring to the table or map shown in FIG. 10, the page number and the background image are specified, the page is displayed on the display means 26, and the stroke based on the entry information is drawn on the page and displayed on the display means 26. Let Further, when the processing unit 24 detects the storage instruction by the operation of the input unit 21 by the user, the processing unit 24 incorporates the stroke information into the page, generates a new background image, and stores it in the storage unit 25.

  When displaying the stroke, when the processing unit 24 identifies that the pen-down information PD is included in the entry information received from the electronic pen 1, the storage unit 25 stores a storage area for storing the latest stroke information. To set. And until the processing means 24 detects that the pen-up information PU is included in the received entry information or until it detects that the next pen-down information PD is included in the received entry information, The coordinate attribute information included in the received entry information is stored in the storage means 25 as a part of the latest stroke information.

  The processing means 24 also receives information necessary for drawing strokes in units of one stroke (hereinafter referred to as “object information”) based on the coordinate attribute information received from the time when the pen-down information PD is received until the pen-up information PU is detected. "). Specifically, the object information is interpolated so that each coordinate point indicated by the stroke information draws a smooth curve using a predetermined interpolation function, and the background image described later is input from the coordinate system of the input area 401. This is information including a set of X and Y coordinates converted into a coordinate system. FIG. 8 is a diagram illustrating an example of the data structure of the object information. As shown in FIG. 8, the object information is a “page number” and a set of coordinate data (hereinafter referred to as “curve coordinate data”) that constitutes a curve generated from stroke information by a predetermined interpolation function. Then, “curve coordinate data” which is a set of X and Y coordinates converted from the coordinate system of the input area 401 to the coordinate system of the background image (page) corresponding to the input area 401, “pen ID”, And “line type information” which is information for specifying the stroke line type. Here, the “background image coordinate system” is a coordinate system based on the background image (page) displayed in the display application 6, and for example, a predetermined point (for example, the upper left corner) of the background image is set as the origin. A coordinate system that defines a relative position in the background image. Further, the line type information specifically indicates information on the line width determined based on the operation of the electronic pen 1 on the line type change instruction areas 501 and 502 of the instruction sheet 5. Then, the processing unit 24 transmits the object information generated by the processing unit 24 to the server device 3 by the transmission unit 27. Further, the processing unit 24 receives the object information stored in the server device 3 from the server device 3 by the receiving unit 22. This specific description will be described in detail in the [Processing Flow] section.

  The storage unit 25 is configured by a memory such as a ROM or a RAM. The storage unit 25 stores a program such as a display application 6 and a printer driver for enabling an instruction to print electronic pen paper to the printer. In addition, the storage unit 25 stores the entry information received from the electronic pen 1 for each pen ID or the predetermined data generated by the execution of the program according to the processing instruction of the processing unit 24.

  The storage unit 25 has information regarding dot patterns in a wide range of coordinate areas, and specific coordinates of the dot patterns assigned to the input areas 401 and the line type change instruction areas 501 and 502 in accordance with instructions from the processing unit 24. Stores definition information about the area. FIG. 9A shows information including coordinate areas stored in the storage unit 25 of the PC device 2A and stored in association with each input area 401 and the line type change instruction areas 501 and 502 (hereinafter referred to as “coordinate definition information”). It is also called a data structure. FIG. 9B is a data structure showing coordinate definition information stored by the storage unit 25 of the PC device 2B. As shown in FIGS. 9A and 9B, the dot pattern address, the area type, and the page number correspond to the coordinate area of the dot pattern printed in each area set on each paper to be printed. Attached and memorized. The dot pattern coordinate area associated with each input area 401 and line type change instruction areas 501 and 502 is defined by corner position coordinates (Xn, Yn), height (Y direction), and width (X direction).

  The dot pattern may be generated by a predetermined algorithm by the processing unit 24 without being stored in advance. The coordinate definition information may be generated by the processing unit 24 according to an instruction from the display application 6 itself, or the processing unit 24 according to an instruction of a plug-in program added to extend the function of the display application 6. May be generated.

  Further, the storage means 25 corresponds to the page number corresponding to each page when one of the one or more background images printed on the input paper 4 is set as one page according to the instruction of the processing means 24. The information of the background image to be stored is stored in association with each other, and the stroke width information displayed on the display unit 26 corresponding to each of the line type change instruction areas 501 and 502 is stored. FIG. 10A is an example of a database that defines the relationship between the page number corresponding to each page and the background image information corresponding thereto. The database shown in FIG. 10A includes a “page number”, a “background image” indicating a storage name of the background image in the PC device 2, and an “image size” indicating the size of the background image. The background image corresponding to each page number is printed in the input area 401 of the input sheet 4 as a picture, a character, or an entry frame. The background image may be a blank pattern. The background image is used for, for example, a picture of one page in a conference material such as a drawing, a design drawing, a plan document, and a report. For example, 15 background images are generated from all 15 pages of conference materials, and the page number and the image size are associated with each other. FIG. 10B is an example of a database that defines line width information corresponding to the line type change instruction areas 501 and 502. The database shown in FIG. 10B includes a “line type change instruction area” indicating whether the line type change instruction is “thin” or “thick”, and the number of points of the line width set by the line type change instruction area. “Line width” shown.

  Here, a specific example of the coordinate definition information shown in FIGS. 9A and 9B will be supplementarily described. Since the line type change instruction area 501 and the line type change instruction area 502 are printed on the same instruction sheet 5, they have different coordinate areas associated with the same dot pattern address “10.30.19.1”. A dot pattern is assigned. The dot pattern of the coordinate area “(x8, y8), H8, W8” is assigned to the line type change instruction area 501 of the line width “thin”, and the line type change instruction area 502 of the line width “thick” A dot pattern “(x9, y9), H9, W9” is assigned. On the other hand, the input area 401 is basically printed on a different input sheet 4 for each page if it is a multi-page meeting material, so that the page number of the printed page can be specified. An input area 401 is required for each page. Therefore, in the input area 401, dot patterns of different coordinate areas associated with different dot pattern addresses are assigned for each page number corresponding to the page to be printed. For example, as shown in FIG. 9A, the dot area of the coordinate area “(x1, y1), H1, W1” is assigned to the input area 401 of the page number “P01”, and the input area of “P02” A dot pattern “(x2, y2), H2, W2” is assigned to 401. In some cases, two pages of background images are printed on the same input paper 4 by two-sided printing. Each page to be subjected to two-sided printing is assigned a dot pattern in a different coordinate area associated with the same dot pattern address, which is different from normal one-sided printing. In the example of FIG. 9B, the input paper 4 associated with the dot pattern address “10.30.20.1.5” has two background images corresponding to the page numbers “P01” and “P02”. Indicates that two-sided printing is being performed. In the input sheet 4 printed with two pages, the dot pattern of the coordinate area “(x10, y10), H10, W10” is assigned to the input area 401 of the page number “P01”, and the input area 401 of “P02” Is assigned a dot pattern of “(x11, y11), H11, W11”.

  Furthermore, the storage means 25 manages the entry status of input devices such as the electronic pen 1 and mouse connected to the PC device 2A. FIG. 11A shows an input device connected to the PC device 2A, a device ID that is identification information of the input device (hereinafter collectively referred to as “pen ID”), an input device entry state, start coordinates, It is a database in which end coordinates and line types are associated. The start coordinates and end coordinates will be described in the [Processing Flow] section.

  In the example shown in FIG. 11A, the electronic pen 1A whose pen ID is “Pen1A” has the entry state “writing drawing”. “Writing / drawing” means that the electronic pen 1 is filling in the input area 401 and connects the coordinate points specified by the entry information corresponding to the same stroke in accordance with the generation order of the entry information (hereinafter referred to as the entry information). , Referred to as “rendering target line”), and shows the state displayed by the display means 26. More specifically, the drawing target line is a line temporarily displayed by the display unit 26 based on the entry information before the curve is generated by the interpolation function. In the entry state corresponding to “Writing / Drawing”, the page number corresponding to the background image printed on the input sheet 4 filled with the electronic pen 1A and the background image displayed by the display application 6 are associated. The page number matches. On the other hand, for the electronic pen 1B whose pen ID is “Pen1B”, the entry state is “writing”. This indicates a state in which the electronic pen 1B is filling in the input area 401 but the drawing target line is not displayed by the display means 26. Specifically, in this case, the page number corresponding to the input sheet 4 entered with the electronic pen 1A is different from the page number displayed by the display application 6. In addition, the mouse whose pen ID is “Mouse” has a blank “-” whose entry state does not correspond to either “writing drawing” or “writing”. In this case, the mouse described above does not perform input into the page displayed on the display application 6.

  FIG. 11B is an example of a database in which an input device connected to the PC device 2B, a pen ID (device ID), an entry state, a start coordinate, and an end coordinate are associated with each other. In FIG. 11B, the electronic pen 1 </ b> C whose pen ID is “Pen1C” has the entry state “writing” and is filling in the input area 401 that is not displayed in the display area of the display application 6. It shows that it is the state of. The pen 1D whose pen ID is “Pen1D” does not correspond to either the “writing drawing” or “writing” state, and indicates a state in which the pen ID is not written in the input area 401. A mouse whose pen ID is “Mouse” indicates a state in which no input is made in the display area of the display application 6.

(Server device)
Next, the server apparatus 3 will be described again with reference to FIG. The server device 3 is configured as a hardware such as a communication device capable of data communication with the PC device 2, a processor such as a CPU, a memory such as a ROM or RAM, and the like. As shown in FIG. 7, the server device 3 functionally includes a reception unit 32, a processing unit 34, a storage unit 35, and a transmission device 37. Schematically, the server device 3 performs transmission, reception, and management of object information.

  The receiving unit 32 receives object information and the like transmitted from the PC device 2. The transmission device 37 transmits the object information to the PC device 2 according to an instruction from the processing unit 34. The processing means 34 is constituted by a processor such as a CPU, and controls the entire server device 3. Specifically, the processing unit 34 provides identification information (hereinafter referred to as “object ID”) to the object information transmitted from the PC devices 2A and 2B. As will be described later, this object ID is given a serial number so that the generated order can be grasped. Further, when the inquiry information including the object ID is transmitted from the PC devices 2A and 2B, the processing unit 34 specifies the object information to be transmitted based on the object ID, and causes the transmission unit 37 to transmit it. This specific process will be described in detail in the [Process Flow] section.

  The storage means 35 is configured by a memory such as a ROM or a RAM. The storage unit 35 stores the object information, the object ID, and the identification information (transmission source information) of the PC device 2 that is the transmission source of the object information in association with each other. FIG. 12 is an example of a database in which object information, a transmission source of object information, and an object ID are associated with each other. In FIG. 12, “Obj01” to “Obj59” are stored in the “object information” column. Each of “Obj01” to “Obj59” indicates a storage name of the object information in the server device 3. In the “transmission source information” column, “PC device 2A” or “PC device 2B” is described. Actually, the storage unit 35 stores unique identification information such as an IP address of the PC device 2 and a MAC address (Media Access Control Address) as transmission source information. In the “object ID” column, “ID001” to “ID059” are described. In this way, serial numbers are assigned to the object IDs so that the generation order can be identified. As a result, as will be described later, the server device 3 can transmit only necessary object information to the PC device 2 to reduce the amount of communication.

(Display application)
FIG. 13 shows an example of the display application 6 displayed on the display of the PC device 2. The display application 6 includes a display area 601 and operation buttons 602 to 607. The display area 601 displays the contents written by the electronic pen 1 in the input area 401 for each page number together with the background image corresponding to the input area 401. Specifically, the processing means 24 reads the background image corresponding to the page number specified by the operation buttons 602, 603, etc. with reference to the database shown in FIG. A stroke is drawn based on the entry information and the object information, and is displayed superimposed on the background image. An operation button 602 displayed as “Previous” (also referred to as “Previous button 602”) is a button that is pressed when the display content of the display area 601 is changed to a page corresponding to the page number one smaller. . An operation button 603 displayed as “next” (also referred to as “next button 603”) is a button that is pressed when the display content of the display area 601 is transitioned to a page corresponding to the next larger page number. . An operation button 604 displayed as “enlarge” (also referred to as “enlarge button 604”) is a button that is pressed when the display content of the display area 601 is enlarged and displayed by a predetermined ratio. An operation button 605 displayed as “reduction” (also referred to as “reduction button 605”) is a button that is pressed when the display content of the display area 601 is reduced by a predetermined ratio and displayed. An operation button 606 displayed as “remote” (also referred to as “remote button 606”) is a button that is pressed when turning on and off a remote conference mode to be described later. During execution of the remote conference mode, the PC device 2 transmits object information generated from the entry information received from the electronic pen 1 to the server device 3. In addition to this, during execution of the remote conference mode, the PC device 2 makes an inquiry about the presence or absence of object information that the server device 3 has received from other PC devices 2 for each predetermined time interval, and receives the object information. . An operation button 607 displayed as “END” (also referred to as “END button 607”) is a button that is pressed when the display application 6 is ended. Preferably, the display contents of the display area 601 can be enlarged or reduced, and the display range of the display area 601 is changed by providing a scroll bar, a scroll button, or the like in the screen of the display application 6. You should be able to do it.

[Processing flow]
Next, a processing flow by the information processing system 10 according to the first embodiment will be described with reference to FIGS. The PC device 2 executes the processing on the object information transmission side (see FIGS. 14 to 17) and the processing on the object information reception side (see FIGS. 19 and 17) described below in parallel and a front button 602. Or a display update process (see FIG. 18) when the next button 603 is pressed.

(Processing on the object information sending side)
FIG. 14 is a flowchart illustrating a processing procedure on the object information transmission side executed by the electronic pen 1 and the PC device 2 in the first embodiment. The processing flow shown in FIG. 14 is repeatedly executed each time entry information is generated by the electronic pen 1.

  When the user enters characters or the like on the input sheet 4 or the instruction sheet 5 with the electronic pen 1, the electronic pen 1 transmits entry information associated with the pen-down information PD and the pen ID to the PC device 2 at the time of pen-down. In addition, during the entry of the stroke, the electronic pen 1 transmits to the PC device 2 entry information in which the coordinate attribute information including the coordinate information obtained by imaging the dot pattern along the stroke and the pen ID is associated with the pen ID. . Then, at the time of pen-up, the electronic pen 1 transmits entry information associated with the pen-up information PU and the pen ID to the PC device 2 (step S101).

  In the PC device 2, when the receiving unit 22 receives the entry information transmitted from the electronic pen 1, the processing unit 24 causes the storage unit 25 to store the entry information (step S201). Next, the processing means 24 determines whether or not coordinate information has been recognized from the entry information, that is, whether or not the entry information includes coordinate information (step S202).

  If it is determined that the coordinate information has been recognized (step S202; Yes), the processing unit 24 determines whether or not the input is made in the input area 401 (step S203). Specifically, the processing unit 24 refers to the coordinate definition information (see FIG. 9) stored in the storage unit 25 and determines whether or not the recognized coordinate information is included in the coordinate area indicating the input area 401. If the processing unit 24 determines that the input area 401 has been entered (step S203; Yes), it sets the entry state of the pen ID included in the entry information to “writing and drawing” (step S204). Specifically, the processing unit 24 sets the entry state corresponding to the pen ID to “writing drawing” in the database that records the entry state of the input device shown in FIG. Then, the processing means 24 executes a real time drawing process (step S205). The real-time drawing process will be described separately using a flowchart. On the other hand, when the processing unit 24 determines that the input area 401 is not filled in (step S203; No), that is, when it is determined that the entry is made on the input sheet 4 or the instruction sheet 5 outside the input area 401, It is determined whether or not the line type change instruction areas 501 and 502 have been entered (step S206). If it is determined that the line type change instruction areas 501 and 502 have been entered (step S206; Yes), the processing unit 24 defines the line type information of the pen ID in the entered line type change instruction areas 501 and 502. The updated line type is updated (step S207). Specifically, the processing unit 24 specifies the line type information of the pen ID based on the database of FIG. 9 and stores it in the storage unit 25 in the database shown in FIG. 11 in association with the target pen ID. On the other hand, when it is determined that the entry is not in the line type change instruction areas 501 and 502 (step S206; No), the processing unit 24 executes a process of step S209 described later.

  On the other hand, when it is determined in step S202 that the coordinate information is not recognized (step S202; No), the processing unit 24 determines whether the entry information includes pen-up information PU (step S208). . When it is determined that the pen-up information PU is included (step S208: Yes), the processing unit 24 determines whether the pen ID entry state is “writing” or “writing drawing” (step). S209). Specifically, the processing unit 24 refers to FIG. 11 and determines whether the entry state of the pen ID corresponding to the entry information is “writing” or “writing drawing”. If it is determined that the pen ID entry state is “Writing” or “Writing / Drawing” (Step S209; Yes), the processing unit 24 executes the processing of the flowchart of FIG. On the other hand, when the pen-up information PU is not included in the entry information (step S208; No), or when the entry state of the pen ID is not “writing” and “writing drawing” (step S209; No). The processing unit 24 ends the process of the flowchart.

(Real-time drawing process)
FIG. 15 is a flowchart showing the procedure of the real-time drawing process in step S205. The process of the flowchart shown in FIG. 15 is executed after it is determined in step S203 that the entry is made in the input area 401 and the entry state of the pen ID is set to “writing drawing”.

  First, the processing means 24 determines whether or not the start coordinates corresponding to the target pen ID are stored in the storage means 25 (step S301) (see FIG. 11). Here, the “starting coordinate” refers to a coordinate that is a starting point of each line segment that is a drawing target line. If the start coordinate of the pen ID does not exist (step S301; No), the coordinate information recognized in step S202 is stored in the storage unit 25 as the start coordinate of the pen ID (step S309). Specifically, the processing unit 24 stores the recognized coordinate information in the database shown in FIG. 11 in the storage unit 25 as the start coordinate in association with the target pen ID. On the other hand, when the start coordinate of the pen ID exists (step S301; Yes), the processing unit 24 stores and updates the storage unit 25 using the recognized coordinate information as the end coordinate of the pen ID (step S302). Here, the “end coordinate” refers to a coordinate that is an end point of each line segment that is a drawing target line. Next, the processing means 24 converts the line segment connecting the start coordinate and the end coordinate from the coordinate system of the input area 401 to the coordinate system of the background image, and stores / updates it as a drawing target line (step S303).

  Next, the processing means 24 determines whether or not the drawing target line is included in the display area 601 (step S304). That is, in this case, the processing means 24 matches the page number corresponding to the input area 401 entered with the electronic pen 1 and the page number displayed in the display area 601 and converts it into the coordinate system of the background image. It is determined whether all or part of the drawn line to be drawn exists in a coordinate area corresponding to the display area 601. When the drawing target line is partially included in the display area 601 (step S304; Yes), the processing unit 24 reads the line type information set for the target pen ID from the database illustrated in FIG. Step S305). Next, the processing means 24 temporarily draws the drawing target line on the display screen, that is, on the display area 601 based on the read line type information (step S306). At this time, the processing unit 24 draws the drawing target line included in the display area 601 in accordance with the display range of the background image in the display area 601 and the enlargement / reduction ratio. On the other hand, when the processing unit 24 determines that the drawing target line is not included in the display area 601 (step S304; No), the processing unit 24 sets the writing state of the target pen ID to “writing” (step S308). ).

  Then, the processing means 24 stores / updates the end coordinates as the start coordinates of the target pen ID after step S306 or step S308 (step S307).

(Object information generation processing and storage processing)
FIG. 16 is a flowchart illustrating the procedure of the generation process on the object information transmission side and the storage process on the object information reception side, which is executed subsequent to step S209 in FIG. In the processing of the flowchart shown in FIG. 16, the pen-up information PU is recognized in step S208 of the flowchart shown in FIG. 14, or the coordinate information is recognized in step S202, but the input area 401 and the line type are respectively detected in steps S203 and S206. This is executed by the PC device 2 when it is determined that the change areas 501 and 502 are not filled in, and the pen ID entry state is “writing” or “writing drawing” in step S209. .

  First, the processing means 24 recognizes stroke information from the coordinate information received when the corresponding pen ID entry state is “Writing” or “Writing / Drawing” (Step S401). Here, as shown in FIG. 5, “stroke information” is associated with time information T and writing pressure data P at the time of entry with X and Y coordinate information recognized by the electronic pen 1 from pen down to pen up. A set of coordinate attribute information. If it is determined as “No” in step S206, that is, if the electronic pen 1 is entered out of the input area 401 being filled in, it is regarded as pen-up and stroke information is recognized ( Step S401). Then, the processing means 24 calculates a curve from the stroke information in the input area 401 using an interpolation function (step S402). Here, “interpolation” refers to obtaining an approximate curve that passes through the start point and end point in each section of a data string. Accordingly, the processing means 24 calculates a curve close to the actual handwriting (handwritten stroke) from the stroke information using the interpolation function. Here, the processing means 24 can apply an interpolation method using various known interpolation functions, and calculates a curve from stroke information by, for example, spline interpolation or Lagrangian interpolation. Next, the processing means 24 generates object information from the calculated curve and stores it in the storage means 25 (step S403). Specifically, the processing unit 24 generates curve coordinate data corresponding to the calculated curve. At this time, the curve coordinate data (a set of coordinate data constituting the curve) is a set having a larger number than the set of coordinate data included in the stroke information in order to draw the calculated curve with high accuracy, and each coordinate data indicates The distance between coordinate points is shortened. Then, the processing unit 24 changes the curve coordinate data from the coordinate system of the input area 401 to the coordinate system of the background image (page) corresponding to the input area 401, and specifies from the X and Y coordinate data included in the stroke information. The object number is generated and stored in association with the page type corresponding to the input area 401, the pen ID, and the line type information specified from the pen ID.

  Next, the processing means 24 clears the entry state and start coordinates of the target pen ID (step S404). That is, the processing unit 24 erases the entry state and start coordinates of the target pen ID from the database shown in FIG. Then, the processing unit 24 performs a drawing update process (step S405). The drawing update process will be separately described in detail with reference to the flowchart of FIG. Next, the processing means 24 determines whether or not it is in the remote conference mode (step S406). Note that the remote conference mode is switched on or off by a remote button 606. If the remote conference mode is not being executed (step S406; No), the processing unit 24 ends the process of the flowchart. On the other hand, when in the remote conference mode (step S406; Yes), the processing unit 24 transmits the object information stored in step S403 to the server device 3 (step S407). At this time, the processing unit 24 transmits source information such as a MAC address to the server device 3 together with the object information. On the other hand, the processing unit 34 of the server device 3 associates the source information with the object information received by the receiving unit 32 and stores it in the storage unit 35 (step S501). Further, the processing means 34 of the server device 3 assigns object IDs, which are unique IDs (identification information), to the object information in the order of serial numbers (step S502). Then, the processing unit 34 of the server apparatus 3 stores the database in the storage unit 35 in association with the object information, the transmission source information, and the object ID, as shown in FIG.

(Drawing update process)
FIG. 17 is a flowchart showing the procedure of the drawing update process in step S405. First, the processing unit 24 refers to the database of FIG. 11 and determines whether or not there is a pen ID whose entry state is “writing and drawing” (step S601). If there is no pen ID whose entry state is “writing and drawing” (step S601; No), that is, if the writing content of the electronic pen 1 is not being drawn in real time based on steps S303 to S306, the processing means 24 clears the temporarily drawn drawing target line from the display screen (step S602). Then, the processing means 24 draws and updates the object information in the display area 601 stored in the PC device 2 on the display screen (step S603). Specifically, the processing unit 24 has the same page number as the page number corresponding to the background image displayed in the display area 601, reads object information included in the display range from the storage unit 25, and displays in the display area 601. In accordance with the display range of the background image and the enlargement / reduction ratio, a curve based on the curve coordinate data is drawn / updated in the display area 601 with the line type included in the object information.

  On the other hand, when there is a pen ID whose entry state is “writing and drawing” (step S601; Yes), that is, when the writing content of the electronic pen 1 is being drawn in real time based on steps S303 to S306, the processing means 24. Ends the process of the flowchart. That is, the processing unit 24 updates the display contents based on steps S602 and S603 to prevent the drawing target line being temporarily drawn from being cleared during writing or the display screen to flicker. The display content is not updated when the pen ID “” exists.

  Note that the processing unit 24 may determine whether or not there is the electronic pen 1 that is pen-down, instead of the processing in step S601 described above. That is, the processing means 24 determines whether or not there is an electronic pen 1 that is transmitting pen-down information PD or entry information including coordinate information. If the electronic pen 1 is present, the processing unit 24 proceeds to steps S602 and S603. Do not update the display content based on it. In this case, even when there is a pen down to the input area 401 not included in the display area 601, the display content of the display area 601 is not updated.

(Display update processing)
FIG. 18 is a flowchart showing the procedure of display update processing for changing the display content of the display area 601. The PC device 2 repeatedly executes the processing of the flowchart shown in FIG. 18 according to a predetermined cycle.

  First, the processing means 24 determines whether or not the display area 601 should be changed (step S701). That is, the processing unit 24 determines whether or not the user has instructed to change the page to be displayed in the display area 601 by pressing the previous button 602, the next button 603, the enlarge button 604, or the reduce button 605 of the display application 6. To do. Then, when the processing unit 24 determines that an instruction to change the display contents of the display area 601 has not been given (step S701; No), the processing of the flowchart ends.

  On the other hand, when it is determined that the processing unit 24 has been instructed to change the display contents of the display area 601 (step S701; Yes), the temporarily drawn drawing target line is cleared from the display screen (step S702). Then, the processing unit 24 clears the drawn object information and the displayed background image from the display screen (step S703). Then, the processing unit 24 displays a background image corresponding to the changed display area 601 (step S704). That is, the processing unit 24 displays the background image corresponding to the page number of the change destination in the display area 601 within the display range. Next, the processing means 24 draws / updates the object information in the display area 601 stored in the PC device 2 on the display screen (step S705). Specifically, the processing unit 24 reads out the object information having the same page number corresponding to the background image displayed in the display area 601 and included in the display range from the storage unit 25, and displays the display area 601. The display area 601 is drawn / updated in accordance with the display range of the background image and the enlargement / reduction ratio.

(Overview of object information inquiry and reception processing)
FIG. 19 is a flowchart illustrating a processing procedure in which the PC apparatus 2A inquires the server apparatus 3 about updated object information and receives the corresponding object information in the first embodiment. The PC device 2A repeatedly executes the processing flow of FIG. 19 according to a predetermined cycle. Here, for convenience of explanation, processing for the PC device 2A will be described, but the PC device 2B also repeatedly executes processing similar to the processing flow of FIG. 19 according to a predetermined cycle.

  First, the processing unit 24 of the PC device 2A determines whether or not the remote conference mode has been started (step S801). That is, the processing means 24 determines whether or not the remote button 606 has been pressed. If the remote mode is not started (step S801; No), the processing unit 24 ends the process of the flowchart. On the other hand, when the remote conference mode is started (step S801; Yes), the processing unit 24 transmits the final object ID among the stored object IDs to the server device 3 (step S802). Here, the “final object ID” refers to the latest object ID assigned the largest serial number among the object IDs assigned by the server apparatus 3 in step S502 of FIG. In addition, the processing unit 24 transmits source information such as the MAC address of the PC device 2 </ b> A to the server device 3 together with the final object ID. The processing unit 24 temporarily stores and updates the time when the last object ID is transmitted in the storage unit 25 as reference time information.

  On the other hand, the processing means 34 of the server device 3 receives the final object ID of the PC device 2A (step S901). Then, the processing unit 34 of the server device 3 determines whether there is updated object information transmitted from other than the PC device 2A (step S902). Specifically, the processing unit 34 of the server device 3 is the object information associated with the transmission source information different from the transmission source information of the PC device 2A from the database shown in FIG. It is determined whether there is object information corresponding to the ID. Then, when the above-described object information exists (step S902; Yes), the processing unit 34 of the server device 3 transmits the object information as updated object information (step S903). On the other hand, when there is no updated object information (step S902; No), the processing unit 34 of the server device 3 ends the process of the flowchart.

  After executing step S802, the processing unit 24 of the PC device 2A determines whether or not the updated object information has been received (step S803). If the processing unit 24 determines that the updated object information has been received (step S803; Yes), the processing unit 24 stores the object information of the update unit in the storage unit 25 (step S804). Then, the processing unit 24 stores the object ID given to the updated object information (step S805). At this time, the processing means 24 stores the maximum object ID among the object IDs assigned to the updated object information as the final object ID used in step S802. Next, the processing unit 24 executes the drawing update process described with reference to FIG. 17 for the updated object information (step S806).

  Then, after the execution of step S806 or when the updated object information has not been received (step S803; No), the processing means 24 determines whether or not a predetermined time has elapsed since the execution of step S802 (reference time). Determination is made (step S807). Here, in order to reduce the performance load of the PC device 2A, for example, the predetermined time may be set to 1 second. In this case, the processing unit 24 executes the processes of steps S802 to S806 and S808 every second. When the predetermined time has not elapsed since the execution of step S802 (step S807; No), the processing unit 24 waits until the predetermined time elapses. On the other hand, when it is determined that a predetermined time has elapsed since the execution of step S802 (step S807; Yes), it is determined whether or not the remote conference mode is ended (step S808). That is, the processing unit 24 determines whether or not the remote button 606 has been pressed after the start of the remote conference mode. If the processing unit 24 determines that the remote conference mode has ended (step S808; Yes), the processing unit 24 ends the processing of the flowchart. On the other hand, if the processing means 24 determines that the remote conference mode has not ended (step S808; No), the processing proceeds to step S802.

[Specific example of curve drawing processing]
A specific example of the curve drawing process shown in the flowcharts of FIGS. 15 to 17 will be described in more detail with reference to FIGS. FIG. 20 shows a display example of a drawing target line displayed in the display area 601. In FIG. 20, coordinate points T (1) to T (6) (the numbers in parentheses indicate the generation order. Hereinafter, the same applies to FIGS. 20 to 23) are the coordinates indicated by the coordinate information recognized from the entry information. The coordinates set as the start coordinates and / or the end coordinates in steps S302, S307, and S309 in FIG. 15 are shown. The drawing target lines L (1) to L (5) indicate the drawing target lines generated in step S303. 20 and 21, the display of the background image is omitted for convenience of explanation.

  First, when the coordinate information corresponding to the coordinate point T (1) is recognized from the entry information transmitted from the electronic pen 1, the processing means 24 starts the pen ID for the target coordinate point T (1). The coordinates are stored in the storage means 25 (see step S309). Next, when the coordinate information corresponding to the coordinate point T (2) is recognized from the entry information, the processing unit 24 stores the coordinate point T (2) as the end coordinate (see step S302) and uses the start coordinate. A drawing target line L (1) is generated based on a line segment connecting a certain coordinate point T (1) and the end coordinate point T (2) (see step S303), and the line type set in the pen ID. (See step S305), the image is temporarily drawn on the display screen (see step S306). Thereafter, the processing means 24 updates and stores the coordinate point T (2) as the start coordinate (see step S307). Next, when the coordinate information corresponding to the coordinate point T (3) is recognized from the entry information, the processing means 24 updates and stores the coordinate point T (3) as the end coordinate (see step S302) and the start coordinate. A drawing target line L (2) is generated based on a line segment connecting the coordinate point T (2) as the end point and the coordinate point T (3) as the end coordinate (see step S303), and the line set in the pen ID The seed is temporarily drawn on the display screen (see step S306). Thereafter, the processing means 24 updates and stores the coordinate point T (3) as the start coordinate (see step S307). Then, the processing unit 24 performs the same processing when the coordinate information corresponding to the coordinate points T (4) to T (6) is recognized. In FIG. 20, the processing means 24 stores the coordinate point T (6) as the end coordinate, and temporarily draws the drawing target line L (5), which is a line segment connecting the coordinate point T (5) stored as the start coordinate. A display area 601 immediately before drawing is shown. FIG. 21 shows a display example of the display area 601 when the processing object 24 temporarily draws the drawing target line L (5) on the display screen with the line segment information read in step S305.

  In FIG. 22, after receiving the entry information corresponding to the coordinate point T (6) and then receiving the entry information including the pen-up information PU, the processing unit 24 performs the coordinate points T (1) to T (6). ) Shows a conceptual diagram of the curve L (1-6) generated based on the above. In this case, the processing unit 24 generates a curve L (1-6) based on the coordinate points T (1) to T (6) using a predetermined interpolation function used in spline interpolation, Lagrangian interpolation, or the like (step S1). (See S402). Note that the curve L (1-6) shown in FIG. 22 is not displayed on the display area 601 and the curve Obj (1-6) of FIG. 23 described later is displayed on the display area 601. FIG. 23 shows a curve Obj (1-6) drawn based on the object information, in which object information is generated based on the curve L (1-6) (see step S403). As shown in FIG. 23, the processing unit 24 deletes the drawing target lines L (1) to L (6) from the display area 601 (see step S602), and is designated based on the line type information included in the object information. The curve Obj is drawn with the selected line type and displayed in the display area 601 (see step S603).

[Display example]
Next, a display example of the display area 601 will be described with reference to FIGS. For convenience of explanation, the background image pattern and the like are omitted in FIGS.

  First, the case where the input area 401 is filled only with the electronic pen 1A will be described with reference to FIG. FIG. 24A shows the display contents of the display area 601 of the PC device 2A immediately before pen-up of the electronic pen 1A. In this case, the entry state of the electronic pen 1A is “Writing / Drawing”, and the processing unit 24 displays the drawing target line corresponding to the entry information transmitted from the electronic pen 1A by the real-time drawing process. The image is temporarily drawn in the area 601 (see step S306). FIG. 24B shows display contents of the display area 601 of the PC device 2A immediately after pen-up of the electronic pen 1A. In this case, the entry state of the electronic pen 1A is cleared (see step S404), and the processing unit 24 further clears the drawing target line temporarily drawn in the display area 601 by the drawing update process (see step S602). ). FIG. 24C shows the display content of the display area 601 when a curve is drawn based on the object information generated by the PC device 2A. The processing means 24 draws a curve in the display area 601 based on the object information generated and stored based on the processing in steps S401 to S403 (see step S603).

  Next, the case where entry is made in the input area 401 with the electronic pen 1A and the electronic pen 1B will be described with reference to FIG. FIG. 25A shows the display contents of the display area 601 of the PC device 2A when the electronic pen 1A is immediately before pen-up and the electronic pen 1B is pen-down. In FIG. 25, “i” indicates a drawing target line written with the electronic pen 1A, and “ii” indicates a drawing target line written with the electronic pen 1B. In FIG. 25A, the writing states of the electronic pen 1A and the electronic pen 1B are “writing drawing”, and the drawing target lines written with the electronic pen 1A and the electronic pen 1B are temporarily drawn in the display area 601. (See step S306). FIG. 25B shows display contents of the display area 601 of the PC device 2A immediately after the electronic pen 1A is pen-up and the electronic pen 1B is immediately before pen-up. In FIG. 25B, the entry state of the electronic pen 1A is blank, and the entry state of the electronic pen 1B is “writing drawing”. At this time, the processing unit 24 generates and stores object information from the drawing target line of the electronic pen 1A based on steps S401 to S403. However, since the electronic pen 1B is in writing and drawing, FIG. Since then, the drawing target line of the electronic pen 1A is temporarily drawn (see step S601; Yes). FIG. 25C shows display contents of the display area 601 of the PC device 2A when the electronic pen 1A is not pen-down after pen-up and the electronic pen 1B is immediately after pen-up. In FIG. 25C, the entry states of the electronic pens 1A and 1B are both blank (step S601; see No), and the processing unit 24 clears the drawing target line temporarily drawn in the display area 601. (See step S602). FIG. 25D shows the display area 601 of the PC device 2A when a curve is drawn in the display area 601 based on the object information corresponding to the electronic pens 1A and 1B generated and stored in steps S401 to S403. Show the contents. At this time, the processing means 24 draws each curve based on the coordinate information included in the object information with the specified line type based on the line type information included in the respective object information (see step S603).

[Operational effects of the information processing system according to the first embodiment]
According to the information processing system of the first embodiment, each PC device 2 is managed by the server device 3 managing object information, and the other PC devices 2 that perform a remote conference based on the object information transmitted from the server. The display contents can be shared. Further, for example, the information processing system according to the first embodiment displays a screen compared to a conventional information processing system that shares handwritten information by distributing the screen of one PC device 2 to the other PC device 2 as image information. Since only the object information necessary to synchronize the contents is received from the server device 3, the network load is small, and it is possible to simultaneously fill in with two or more electronic pens 1 connected to different PC devices 2. Furthermore, as another example, the movement of the cursor of both PC devices 2 is transmitted to the server device 3, and the server device 3 performs drawing and compares the image information to each PC device. The information processing system according to the first embodiment does not need to send and draw the stroke information entered by itself to the server device 3, can prevent display delay and the like, and can reduce the network load. In the information processing system of the first embodiment, even if the screen resolution of the PC device 2 is different, the stroke information is changed from the coordinate system of the input area 401 to the coordinate system of the background image corresponding to the input area 401. As a result, there is no displacement in the display.

  In the information processing system of the first embodiment, while the PC device 2 is drawing the entry information of the electronic pen 1 in real time, the display content is not updated from the drawing target line to the curve by the interpolation function. It is possible to prevent the drawing target line being temporarily drawn from being cleared in the middle of writing or flickering of the display screen by updating the display contents while the stroke of 1 is being entered.

  Further, in the information processing system according to the first embodiment, even when the object information is received from the server device 3, the received object information is not displayed while the entry information of the connected electronic pen 1 is drawn in real time. Since drawing is not performed, it is possible to prevent the drawing target line being temporarily drawn from being cleared during writing by updating the display contents while the stroke by the electronic pen 1 is being entered, or to prevent the display screen from flickering.

[Modification of First Embodiment]
Next, a modification of the first embodiment will be described. The following modifications may be applied in any combination to the first embodiment described above.

(Modification 1)
In the first embodiment, the PC device 2 generates object information after calculating a curve from the stroke information using an interpolation function, and receives object information generated by another PC device 2 based on the object information. A curve is displayed in the display area 601. Instead of this, the PC device 2 may draw the stroke by calculating the curve by the interpolation function at the time of drawing in the display area 601 without obtaining the curve by the interpolation function at the time of calculating the object information. Thereby, the PC device 2 prevents the data amount of the object information to be transmitted and received from becoming excessive, and reduces the network load. This will be described with reference to FIGS. 26 to 28 for a portion that executes processing different from that of the first embodiment described above.

  FIG. 26 is a flowchart illustrating a procedure of object information generation processing and storage processing according to the first modification. First, the processing means 24 recognizes stroke information from the coordinate information received when the entry state is “writing” or “writing drawing” (step S421). Then, the processing unit 24 generates object information from the stroke information in the input area 401 and stores it in the storage unit 25 (step S422). Specifically, the processing means 24 includes X, Y coordinate data included in the stroke information, a page number corresponding to the input area 401 specified from the X, Y coordinate data, a pen ID included in the stroke information, and a pen The line type information specified from the ID is associated and generated and stored as object information. Thereafter, the processing unit 24 performs the same processing as Steps S404 to S407 (see FIG. 16) of the first embodiment in Steps S423 to S426.

  FIG. 27 is a flowchart illustrating the procedure of the drawing update process in step S424 according to the first modification. When it is determined that there is no pen ID whose entry state is “writing and drawing” (step S621; No), the processing unit 24 clears the drawing target line temporarily drawn from the display screen (step S622). . Then, the processing means 24 calculates a curve by an interpolation function from the object information in the display area 601 stored in the PC device 2 (step S623). Specifically, the processing unit 24 has the same page number as the page number corresponding to the background image displayed in the display area 601, reads object information included in the display range from the storage unit 25, and stores the object information A curve is calculated from each coordinate point indicated by the coordinate data by an interpolation function. Then, the processing means 24 draws / updates the calculated curve on the display screen (step S624). In this case, the processing means 24 draws the curve with the corresponding line type in the display area 601 in accordance with the display range of the background image in the display area 601 and the enlargement / reduction ratio.

  FIG. 28 is a flowchart illustrating a procedure of display update processing according to the first modification. When the processing unit 24 determines that the user has instructed to change the display content of the display area 601 by pressing the previous button 602, the next button 603, the enlargement button 604, or the reduction button 605 (step S721; Yes), The drawing line to be temporarily drawn is cleared from the display screen (step S722), the drawn object information and the displayed background image are cleared from the display screen (step S723), and the changed display area 601 is displayed. A background image corresponding to is displayed (step S724). Next, the processing means 24 calculates a curve by an interpolation function from the object information in the display area 601 stored in the PC device 2 (step S725). Specifically, the processing unit 24 has the same page number as the page number corresponding to the background image displayed in the display area 601, reads object information included in the display range from the storage unit 25, and stores the object information A curve is calculated from each coordinate point indicated by the coordinate data by an interpolation function. Then, the processing means 24 draws / updates the calculated curve on the display screen (step S726). In this case, the processing means 24 draws the curve with the corresponding line type in the display area 601 in accordance with the display range of the background image in the display area 601 and the enlargement / reduction ratio.

  Note that the processing unit 24 calculates the curve from the object information using the interpolation function instead of the processing of steps S623 and S624 in FIG. 27 and steps S725 and S726 in FIG. 28, and stores the curve coordinate data in the storage unit 25. May be. Then, when the curve coordinate data corresponding to the target object information is already stored in the storage unit 25, the processing unit 24 stores the curve coordinate data without calculating the curve using the interpolation function. Read from the means 25 and draw / update on the display screen with the corresponding line type.

(Modification 2)
In the first embodiment, the line type change instruction areas 501 and 502 are printed on the instruction sheet 5 which is a sheet different from the input sheet 4 on which the input area 401 is printed. Instead, the line type change instruction areas 501 and 502 may be printed together with the input area 401 on the input paper 4. In this case, the input sheet 4 also functions as the instruction sheet 5. In the first embodiment, the information processing system 10 can designate two types of line widths using the line type change instruction areas 501 and 502. Instead of this, the information processing system 10 may be provided with three or more areas where the line width can be specified on the instruction sheet 5 and can specify three or more types of line widths. Furthermore, the information processing system 10 may provide a plurality of areas on the instruction sheet 5 in which the line color, the line style such as the solid line, the broken line, and the chain line can be designated instead of or in addition to the line width.

(Modification 3)
In addition to the configuration shown in FIG. 1, the information processing system 10 may be provided with a viewing-only PC device that receives object information from the server device 3 every predetermined time and draws the received object information on the PC device.

(Modification 4)
In the first embodiment, the coding pattern is an Anoto dot pattern. However, the coding pattern is not limited to the Anoto dot pattern as long as it is a coding pattern indicating position coordinates.

(Modification 5)
In the first embodiment, the object ID is assigned a serial number, and the PC device 2 requests the updated object information by transmitting the last object ID to the server device 3. Instead, the PC device 2 may request updated object information by transmitting all the received object IDs or information corresponding thereto to the server device 3. In this case, the object information is not necessarily a serial number. Alternatively, the PC device 2 generates the object information including the time information, and transmits the latest time information among the time information included in the received object information to the server device 3, thereby updating the updated information. Object information may be requested. The time information may be the time information included in the entry information immediately before pen-up of the corresponding stroke information, or the generation time of the object information.

Second Embodiment
Next, a second embodiment will be described. The second embodiment is different from the first embodiment in that the screen of the display unit 26 of the PC device 2 is projected by the projector 7 onto the screen 8 on which the dot pattern is printed, so that the input paper 4 is replaced with the screen 8. The point is that the electronic pen 1 is filled in. Hereinafter, the description of the same parts as those in the first embodiment will be omitted as appropriate.

  FIG. 29 shows the configuration of an information processing system 100A according to the second embodiment. In FIG. 29, for convenience of explanation, only one PC device 2C that performs a remote conference and its peripheral elements are shown, and the server device 3 and other PC devices 2 are not shown. As shown in FIG. 29, the information processing system 100A according to the second embodiment mainly includes an electronic pen 1, a PC device 2C, an instruction sheet 5, a projector 7, and a screen 8.

  In FIG. 29, such an information processing system 100A is used in the remote conference system as in the first embodiment. A projector 7 that projects the same image as the display screen of the display 201 onto the screen 8 is connected to the PC device 2C. The PC device 2 </ b> C exchanges object information with other PC devices 2 via the server device 3 as in the first embodiment.

  The screen 8 includes a magnet plate 801 made of a hard material so that writing pressure is applied to the electronic pen 1, a sheet 802, and a projection region 803. A dot pattern is printed on substantially the entire surface of the paper 802 with ink containing carbon that absorbs infrared rays so that it can be read by the electronic pen 1, and the dot pattern is protected by, for example, a transparent EB cured resin layer. . The projection area 803 is an area in which the same image as the content displayed on the display 201 of the PC device 2 </ b> C is projected by the projector 7 and a stroke is written using the electronic pen 1. The distance between the projector 7 and the screen 8, the elevation angle of the projector 7 and the like are adjusted in advance so that the projector 7 projects an image in accordance with the projection area 803. Here, even when the projection range of the image by the projector 7 does not exactly coincide with the projection region 803, it is possible to determine which position in the projected image the position coordinates of the screen strip 8 correspond to by the calibration process described later. The device 2C can grasp it. The screen 8 is an example of the “input paper” in the present invention.

  The processing unit 24 of the PC device 2C causes the projector 7 to project the calibration mark onto the screen 8 by displaying at least two calibration marks on the display 201, and the calibration mark is projected. A coordinate conversion function for converting the coordinate system related to the dot pattern on the screen 8 to the coordinate system related to the display 201 is obtained by causing the electronic pen 1 to read the dot pattern at the selected position and receiving the entry information, and store it. Store in the means 25 (calibration processing function). Further, the processing means 24 converts the coordinate system data relating to the dot pattern of the screen 8 to the coordinate system data relating to the display 201 using the coordinate transformation function obtained by the calibration process. , Strokes are drawn or predetermined processing is performed.

  FIG. 30 shows that the user is filling in an area with the electronic pen 1 in an area corresponding to the display area 601 of the display application 6 in the projection area 803 projected by the projector 7 (hereinafter referred to as “input area 401C”). The example of a display of the screen 8 in the case is shown. In this case, the processing unit 24 of the PC device 2C converts the coordinate data on the projection area 803 included in the entry information received from the electronic pen 1 into the coordinate data on the display 201 using the coordinate conversion function calculated by the calibration process. Convert. When the processing unit 24 determines that the input is made in the input area 401C from the converted coordinate data, the processing unit 24 executes a drawing process and superimposes the background image being displayed on the display area 601 of the display application 6, The stroke of the electronic pen 1 is displayed. Here, in the first embodiment, the processing unit 24 refers to the database shown in FIG. 9 and specifies the page number to be processed from the coordinate information included in the entry information. Instead, in the second embodiment, the processing unit 24 cannot distinguish the page number from the coordinate information indicated by the dot pattern, so that the entry information of the electronic pen 1 is displayed on the background image displayed by the display application 6 at the time of reception. The corresponding page number is specified as the page number to be processed.

  Next, processing portions different from the processing flows shown in FIGS. 14 to 19 described in the first embodiment will be described. FIG. 31 is a flowchart showing a processing procedure of the second embodiment corresponding to FIG. 14 of the first embodiment. As shown in FIG. 31, in the second embodiment, the process of step S223 is newly added compared to FIG. Specifically, when the receiving unit 22 receives the entry information transmitted from the electronic pen 1 in the PC device 2C, the processing unit 24 stores the entry information in the storage unit 25 (step S221). Next, when the coordinate information is recognized from the entry information (step S222; Yes), the processing unit 24 converts the coordinate system of the coordinate information using the coordinate conversion function obtained by the calibration process (step S223). ). Specifically, the processing unit 24 converts the coordinate system of the coordinate information from the coordinate system related to the dot pattern on the screen 8 to the coordinate system on the display 201 by the coordinate conversion function. Thereafter, the processing unit 24 executes the processes of steps S224 to S230 in the same manner as steps S203 to S209 (see FIG. 14).

  In the second embodiment, real-time drawing processing, object information generation processing and storage processing, drawing update processing, display update processing, object information inquiry and reception processing, modification examples, and the like are the same as in the first embodiment. .

[Operational effects of the information processing system of the second embodiment]
According to the information processing system 100A of the second embodiment, the PC device 2C can share the display content on the screen 8 with another PC device 2 that performs a remote conference based on the object information transmitted from the server. At the same time, the writing contents of the electronic pen 1 on the screen 8 can be processed in the same manner as the writing contents on the input paper 4. In the information processing system 100A of the second embodiment, the coordinate information is converted from the coordinate system related to the dot pattern on the screen 8 to the coordinate system on the display 201 using the coordinate conversion function by performing the calibration process. For this reason, no positional deviation occurs during display.

<Third Embodiment>
Next, a third embodiment will be described. The third embodiment differs from the second embodiment in that a transparent sheet that can be attached to the display of the PC device 2 is used instead of the screen 8. Hereinafter, the same reference numerals are given to the same parts in the third embodiment, the first embodiment, and the second embodiment, and the description thereof will be omitted as appropriate.

  FIG. 32 shows a schematic configuration example of an information processing system 100B according to the third embodiment. In FIG. 32, for convenience of explanation, only one PC device 2D that performs a remote conference and its peripheral elements are shown, and the server device 3 and other PC devices 2 are not shown. As shown in FIG. 32, the information processing system 100B mainly includes an electronic pen 1, a PC device 2D, an instruction sheet 5, and a transparent sheet 9. A dot pattern is printed on substantially the entire surface of the transparent sheet 9 with ink containing a material that reflects infrared rays so that it can be read by the electronic pen 1, and is attached to the display 201 of the PC device 2. The transparent sheet 9 is an example of the “input paper” in the present invention. Similarly to the second embodiment, the processing unit 24 of the PC device 2D displays at least two or more calibration marks on the display 201, and the dot pattern at the position where the calibration marks are projected is displayed on the electronic pen 1. A coordinate conversion function for converting the coordinate system related to the dot pattern of the transparent sheet 9 to the coordinate system related to the display 201 is obtained by reading and receiving the entry information (calibration processing function). Further, the processing means 24 converts the coordinate system data relating to the dot pattern of the transparent sheet 9 into the coordinate system data relating to the display 201 using the coordinate transformation function obtained by the calibration process. Then, strokes are drawn or predetermined processing is performed. In addition, the structure of the patent document 4 can be employ | adopted for the structure of the transparent sheet 9. FIG. In addition, the electronic pen 1 is preferably configured to be able to read the dot pattern printed on the transparent sheet 9 and reflecting infrared rays by inversion after binarization of the captured image.

  In the third embodiment, processing outline on the object information transmission side, real-time drawing processing, object information generation processing and storage processing, drawing update processing, display update processing, object information inquiry and reception processing, modification examples, etc. This is the same as in the second embodiment.

[Operational effects of the information processing system of the third embodiment]
According to the information processing system 100B of the third embodiment, the PC device 2D can share display contents with other PC devices 2 that conduct a remote conference based on the object information transmitted from the server. The written content of the electronic pen 1 on the display 201 via the transparent sheet 9 can be processed. Further, in the information processing system 100B of the third embodiment, by performing the calibration process, as in the second embodiment, no positional deviation occurs during display.

<Fourth embodiment>
Next, a fourth embodiment will be described. The fourth embodiment differs from the first to third embodiments in that an area corresponding to the display area 601 of the display application 6 (hereinafter referred to as “input area 401E”) by a drag operation of a pointing device such as a mouse. This is the point where the stroke is directly entered. That is, the information processing system of the fourth embodiment includes a PC device 2E used by a user who performs a remote conference and a server device 3 as a minimum configuration. Hereinafter, the same reference numerals are given to the same parts in the fourth embodiment and the first to third embodiments, and the description thereof will be omitted as appropriate.

  FIG. 33 shows a display example of the display application 6 on the display 201 of the PC device 2E in the fourth embodiment. As shown in FIG. 33, in the fourth embodiment, the user of the PC device 2E moves the mouse pointer 608 by dragging a pointing device such as a mouse, and the processing means 24 moves the mouse pointer in the input area 401E. The movement locus of 608 is processed as a handwritten stroke. Specifically, the processing unit 24 coordinates points in the coordinate system (that is, the coordinate system of the background image) on the display area 601 obtained by sampling the movement trajectory information of the mouse pointer 608 input from the pointing device with a predetermined time width. Is specified as coordinate information. Here, the “movement trajectory information” is common to the entry information generated by the electronic pen 1 and the coordinate information, and is similar to the entry information generated by the electronic pen 1 and is an example of “entry information” in the present invention. is there. Note that the movement trajectory information and the entry information generated by the electronic pen 1 do not necessarily have the same data content and format, and may have different data content and format.

  FIG. 34 is a flowchart illustrating a processing procedure on the object information transmission side executed by the PC device 2E in the fourth embodiment. The processing flow shown in FIG. 34 is executed every time an input signal from a mouse which is an example of a pointing device is received.

  First, the processing means 24 of the PC device 2E determines whether or not the left button of the mouse has been pressed (step S241). If it is determined that the left button of the mouse has been pressed (step S241; Yes), the processing unit 24 determines whether or not the mouse pointer 608 is included on the input area 401E (step S242). When the mouse pointer 608 is included on the input area 401E (step S242; Yes), the processing unit 24 sets the entry information of the pen ID to “being written” (step S243). At this time, as shown in the last line of the database shown in FIG. 11, the PC device 2 stores and identifies the mouse pen ID as “Mouse”, for example. Then, the processing means 24 performs real-time drawing processing based on the flowchart of FIG. 15, creates a drawing target line according to the movement of the mouse pointer 608, and draws it on the display 201 (step S244).

  On the other hand, when it is determined that the left button of the mouse is not pressed (step S241; No), or when it is determined that the mouse pointer 608 is not included on the input area 401E (step S242; No), the processing unit 24. Determines whether the pen ID entry state of the mouse is “writing” or “writing drawing” (step S245). Then, when the entry state of the mouse pen ID is “Writing” or “Writing / Drawing” (step S245; Yes), the processing means 24 performs the processing (object information generation processing) of the flowchart of FIG. And storage processing). On the other hand, when the entry state of the pen ID of the mouse is not “writing” or “writing drawing” (step S245; No), the processing unit 24 ends the processing of the flowchart.

  In the fourth embodiment described above, the handwritten stroke by the pointing device is drawn with a fixed line type. Instead, the input area 401E and the line are included in the area corresponding to the display area 601 of the display application 6. The seed change instruction areas 501E and 502E may be provided so that the user can specify the line type when displaying the handwritten stroke by the pointing device. In this case, the flowchart of FIG. 34 is changed as follows. If it is determined in step S242 that the mouse pointer 608 is not included in the input area 401E (step S242; No), the processing unit 24 determines whether or not the mouse pointer 608 is included in the line type change instruction areas 501E and 502E. Determination is made (see step S206 in FIG. 14). If the processing unit 24 determines that the line type change instruction areas 501E and 502E have been entered, the processing unit 24 defines the line type information of the mouse pen ID in the line type change instruction areas 501 and 502. The updated line type is updated (see step S207 in FIG. 14). On the other hand, if it is determined that the entry is not in the line type change instruction areas 501 and 502, the processing unit 24 executes the process of step S245.

  In the fourth embodiment, real-time drawing processing, object information generation processing and storage processing, drawing update processing, display update processing, object information inquiry and reception processing, modification examples, and the like are described in the first to third embodiments. It is the same.

[Operational effects of the information processing system of the fourth embodiment]
According to the information processing system of the fourth embodiment, the PC device 2E can share display contents with another PC device 2 that conducts a remote conference based on the object information transmitted from the server, and the mouse. Written content can be received by a pointing device such as. Further, since the minimum configuration necessary for the information processing system of the fourth embodiment is only the server device 3 and the PC device 2E, the user can construct the information processing system according to the fourth embodiment relatively easily. .

<Fifth Embodiment>
In the fifth embodiment, the information processing system is constructed by arbitrarily combining the first to fourth embodiments. Specifically, any one or a plurality of embodiments of the first to fourth embodiments are selected and applied to each PC device 2 of the information processing system. For example, one PC device 2 used for a remote conference receives entry information for the input form 4 from the electronic pen 1 based on the first embodiment, and another PC device 2 used for the same remote conference is a second embodiment. The entry information for the screen 8 may be received from the electronic pen 1 based on the form. In yet another example, the PC device 2 used for the remote conference receives entry information for the transparent sheet 9 from the electronic pen 1 based on the third embodiment, and fills in the locus of the mouse pointer 608 based on the fourth embodiment. It may be specified as information. The operational effects of the fifth embodiment are the same as those of the first to fourth embodiments.

DESCRIPTION OF SYMBOLS 1, 1A, 1B, 1C ... Electronic pen 2, 2A, 2B, 2C, 2D, 2E ... PC apparatus 3 ... Server apparatus 4 ... Input paper 5 ... Instruction paper 6 ... Display application 7 ... Projector 8 ... Screen 9 ... Transparent sheet 10, 100A, 100B ... information processing system 21 ... input means 22, 32 ... reception means 24, 34 ... processing means 25, 35 ... storage means 26 ... display means 27, 37 ... transmission means

Claims (20)

A PC device;
An input device that is electrically connected to the PC device and generates entry information relating to a handwritten stroke;
A server device connected to the PC device via a communication network, and an information processing system comprising:
The PC device
Display means;
Entry information receiving means for receiving the entry information from the input device;
First drawing means for drawing on the display means based on the entry information;
Object information receiving means for receiving object information which is information related to drawing of each handwritten stroke from the server device;
Second drawing means for drawing on the display means based on the object information received by the object information receiving means,
The second drawing means includes
When the first drawing means is drawing a handwritten stroke based on the entry information, the information processing system delays drawing based on the object information until drawing of the handwritten stroke is completed. The PC device
Object information generating means for generating object information for each stroke of the handwritten stroke based on the entry information;
Object information transmitting means for transmitting the object information to the server device;
Object information inquiry means for making an inquiry about the presence or absence of object information transmitted from another PC device at predetermined intervals to the server device,
2. The information according to claim 1, wherein the object information receiving unit receives the object information from the server device when the object information transmitted from the other PC device exists in the server device. Processing system. The display means displays a plurality of pages,
The object information includes a page number specified from the entry information,
When the page number included in the object information matches the page number of the page being displayed by the display means, the second drawing means performs drawing based on the object information for the object information. The information processing system according to claim 1 or 2. The object information includes line type information specified from the entry information,
The information processing system according to any one of claims 1 to 3, wherein the second drawing unit performs drawing based on the object information based on the line type information with respect to the object information. The input device is an electronic pen that reads a coded pattern from an input sheet on which a coded pattern is formed,
When the entry information received by the entry information receiving means indicates a pen-up, the second drawing means determines that the drawing of one stroke corresponding to the electronic pen that has transmitted the entry information has been completed. The information processing system according to any one of claims 1 to 4, wherein: The input device is a mouse;
The second drawing means, when it is determined that the left click of the mouse is finished, it is determined that the drawing of one stroke corresponding to the mouse is completed. 5. The information processing system according to any one of 4.   The object information generation means generates a smooth curve based on coordinate points indicated by each entry information corresponding to one stroke of the handwritten stroke, and generates object information based on the curve. The information processing system described.   The object information is a coordinate point constituting the curve converted from a coordinate system of coordinate points indicated by each entry information to a coordinate system based on a page on which the handwritten stroke is displayed by the display means. The information processing system according to claim 7, comprising a set.   The said 2nd drawing means draws the said curve on the said display means, after converting the coordinate point corresponded to the said handwritten stroke contained in the said object information into a smooth curve. The information processing system according to any one of the above. The PC device
Further comprising mode switching means for switching the setting of whether or not the remote conference mode based on external input,
The information processing system according to claim 1, wherein the object information receiving unit receives object information only in the remote conference mode. A program that is electrically connected to an input device that generates entry information related to handwritten strokes, and that is connected to a server device via a communication network and is installed and executed in a PC device that includes display means
Entry information receiving means for receiving the entry information from the input device;
First drawing means for drawing on the display means based on the entry information;
Object information receiving means for receiving object information which is information relating to drawing of each handwritten stroke from the server device;
Causing the PC device to function as second drawing means for drawing on the display means based on the object information received by the object information receiving means;
The second drawing means includes
When the first drawing means draws a handwritten stroke based on the entry information, the program delays the drawing based on the object information until the drawing of the handwritten stroke is completed. Object information generating means for generating object information for each stroke of the handwritten stroke based on the entry information;
Object information transmitting means for transmitting the object information to the server device;
The PC device further functions as an object information inquiry means for making an inquiry about the presence or absence of object information transmitted from another PC device at predetermined intervals to the server device,
12. The program according to claim 11, wherein the object information receiving unit receives the object information from the server device when the object information transmitted from the other PC device exists in the server device. . The display means displays a plurality of pages,
The object information includes a page number specified from the entry information,
When the page number included in the object information matches the page number of the page being displayed by the display means, the second drawing means performs drawing based on the object information for the object information. The program according to claim 11 or 12. The object information includes line type information specified from the entry information,
The program according to claim 11, wherein the second drawing unit performs drawing based on the object information based on the line type information for the object information. The input device is an electronic pen that reads a coded pattern from an input sheet on which a coded pattern is formed,
When the entry information received by the entry information receiving means indicates a pen-up, the second drawing means determines that the drawing of one stroke corresponding to the electronic pen that has transmitted the entry information has been completed. 15. The program according to any one of claims 11 to 14, wherein: The input device is a mouse;
12. The second drawing unit, when it is determined that the left click of the mouse is finished, it is determined that drawing of a handwritten stroke for one stroke corresponding to the mouse is completed. 14. The program according to any one of 14.   The object information generation means generates a smooth curve based on coordinate points indicated by each entry information corresponding to one stroke of the handwritten stroke, and generates object information based on the curve. The listed program.   The object information is a coordinate point constituting the curve converted from the coordinate system of the coordinate point indicated by each entry information into a coordinate system based on a page on which the handwritten stroke is displayed by the display unit. The program according to claim 17, comprising a set.   The said 2nd drawing means draws the said curve on the said display means after converting the coordinate point corresponded to the said handwritten stroke contained in the said object information into a smooth curve. The program as described in any one. Further functioning the PC device as mode switching means for switching the setting of whether or not the remote conference mode based on external input,
The program according to any one of claims 11 to 19, wherein the object information receiving unit receives object information only in the remote conference mode.

Images