JP2013182122A - Electronic pen system and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an electronic pen system capable of appropriately setting an order of reproducing or displaying a content associated with a teaching material, and to provide a program.SOLUTION: A computer device stores association area information relating to an association area of a textbook 2 with which a content is associated. The computer device sets an order of reproduction or display of the content associated with the association area in a spread page of the textbook 2 in which a plurality of association areas are provided, on the basis of entry information received from an electronic pen 1.

Description

  The present invention relates to a technique using a coding pattern readable by an electronic pen.

  2. Description of the Related Art Conventionally, electronic pens that digitize written information have been developed, and “Anot pen” developed by Swedish company Anoto is known as a representative one. For example, as described in Patent Document 1, the anotopen reads a predetermined dot pattern printed on a sheet, generates entry information, and transmits the entry information to the terminal device.

  Patent Document 2 describes a rotation correction processing function that corrects an arrangement of dots on an image caused by an angle of an electronic pen facing a dot pattern. Furthermore, Patent Literature 3 describes a lesson support system that displays teaching material data on a screen based on a scenario prepared in advance during a lesson. In this lesson support system, when the button displayed as “next page” is selected on the screen, the entire display area provided in the screen transitions according to a scenario prepared in advance.

Japanese Patent No. 3874498 Special table 2003-529853 gazette JP 2002-072850 A

  During a class, contents such as videos related to the contents are linked in advance to a predetermined area of the teaching material that is open, and in the lesson, the instructor taps the associated teaching material area with an electronic pen, Conveniently, the associated content can be played back and viewed by the student. On the other hand, when a plurality of contents are associated with each page or spread page of the teaching material, the lecturer may mistake the order of reproducing the contents in the class. Therefore, in this case, it is convenient that the order of reproducing or displaying the plurality of contents can be set in advance. Therefore, the present invention provides an electronic pen system and a program capable of appropriately setting the order in which the content associated with the learning material is reproduced or displayed.

  An electronic pen system according to the present invention includes a teaching material on which a coded pattern is printed, a computer device, and reads the coded pattern to calculate the position coordinate or code value of the pen tip, and includes the position coordinate or code value. An electronic pen system comprising: an electronic pen that transmits entry information to the computer device, wherein the computer device receives from the electronic pen storage means for storing information on an association area of the teaching material associated with content And a reproduction order setting means for setting the reproduction or display order of the contents associated with the association area on the teaching material page or the spread page provided with a plurality of association areas based on the entry information.

  The electronic pen system includes an educational material on which a coded pattern is printed, a computer device, and an electronic pen. The computer device stores information on an association area of teaching materials associated with content. Further, the computer device sets the order of reproduction or display of the content associated with the association area on the teaching material page or the spread page provided with a plurality of association areas based on the entry information received from the electronic pen. As described above, the electronic pen system determines the reproduction order of the content associated with the page or the spread page having a plurality of association areas among the pages or spread pages of the teaching material based on the entry information of the electronic pen. Thereby, in a lesson or the like, the user can reproduce or display the content in the expected order without making a mistake in the order of the content to be reproduced.

  In one aspect of the electronic pen system, the reproduction order setting means recognizes the selection order in which the association area is selected by the electronic pen based on the entry information, and based on the selection order, the reproduction or display order. Determine. According to this aspect, the user can determine the order of reproduction or display of the content associated with the same page or the same spread page by sequentially selecting the association area with the electronic pen.

  In another aspect of the electronic pen system, the paper on which the teaching material or the coded pattern is printed is provided with an order setting icon for starting the processing for determining the order of the reproduction or display, and the reproduction order. The setting unit recognizes the selection order in which the association area is selected by the electronic pen after the order setting icon is selected by the electronic pen. According to this aspect, the electronic pen system can determine the order of reproduction or display of the contents associated with the same page or the same spread page at any timing of the user.

  In another aspect of the electronic pen system, on the paper on which the teaching material or the coding pattern is printed, the content is reproduced or displayed based on the reproduction or display order set by the reproduction order setting unit. A reproduction icon is provided, and the computer device has reproduction control means for reproducing or displaying the content corresponding to the order to be reproduced or displayed every time it is detected that the reproduction icon is selected by the electronic pen. Also have. According to this aspect, the user can reproduce or display the contents for which the order has been set on the computer device in order only by performing an operation of selecting a reproduction icon with the electronic pen.

  In another aspect of the electronic pen system, the computer device has display control means for displaying electronic teaching materials obtained by digitizing the teaching materials on a display means, and the display control means has the order of reproduction or display. When displaying the association area of the electronic teaching material corresponding to the set content on the display means, the display of the order of the reproduction or display of the content is attached in association with the association area. By doing in this way, when the electronic teaching material is displayed on the display, the computer device can easily recognize the set order of reproduction or display.

  In another aspect of the electronic pen system, the computer device includes display control means for displaying electronic learning material obtained by digitizing the learning material on display means, and the display control means includes the association area of the electronic learning material. Is displayed on the display means, the association area is color-coded for each corresponding content type. By doing in this way, the electronic pen system can make a user recognize easily the kind of content linked | related with the correlation area.

  In another aspect of the electronic pen system, the computer device displays an electronic teaching material obtained by digitizing the teaching material on a display unit, and based on the entry information, on the handwriting entered in the teaching material by the electronic pen. Display control means for drawing a stroke based on the electronic learning material, the display control means being filled in the learning material by the electronic pen based on the entry information even when the content is being reproduced or displayed A stroke based on handwriting is drawn on the electronic teaching material. By doing in this way, the electronic pen system can execute writing on the electronic teaching material with the electronic pen even when the content is reproduced, and the convenience can be improved.

  In another aspect of the electronic pen system, the electronic pen system further includes a screen having a coding pattern attached to the surface, and a projector that emits light based on an image signal transmitted from the computer device to the surface of the screen, The computer apparatus includes: a display control unit that causes the projector to project and display a reproduction icon for reproducing or displaying the content based on the reproduction or display order set by the reproduction order setting unit; When the displayed icon for reproduction is selected, the apparatus further includes reproduction control means for reproducing or displaying the content corresponding to the order to be reproduced or displayed. According to this aspect, the user can play or display the contents in which the playback order is set in order by tapping the playback icon projected on the screen by the projector with the electronic pen.

  In another aspect of the electronic pen system, the teaching material realizes a predetermined function when selected by the electronic pen and a content display area for displaying the content of the teaching material for each page or spread page. And an operation area provided with a plurality of icons. According to this aspect, the user can perform various operations with the electronic pen on the teaching material, and convenience is improved.

  The program according to the present invention is caused to function as any one of the computer devices described above. The computer apparatus according to the present invention can be configured by installing and functioning these programs in the computer apparatus.

  According to the present invention, the electronic pen system determines the reproduction order of the content in a page having a plurality of areas associated with the content or a spread page based on information entered in the electronic pen. Thereby, in a lesson or the like, the user can reproduce or display the content in the expected order without making a mistake in the order of the content to be reproduced.

1 shows a configuration of an electronic pen system according to an embodiment. It is a top view of the textbook in the open state. It is an example of a display of an electronic textbook. The relationship between the dot pattern dot and the value to which the dot is converted is shown. (A) shows a dot pattern typically, and (b) is a figure showing an example of information corresponding to it. It is the schematic which shows the structure of an electronic pen. It is a functional block diagram of a computer apparatus. The data structure of association area information is shown. The position change of the electronic pen on the textbook accompanying the setting of the reproduction order is shown. The display screen of the computer apparatus on which the facing page in which the reproduction order is set is displayed is shown. A state in which a reproduction operation with an electronic pen is performed on a spread page is shown. A display example of the file “worksheet 1.pdf” is shown. The top view of the operation paper equivalent to the operation area of a textbook is shown. The top view of the textbook which concerns on a modification is shown.

  Hereinafter, preferred embodiments for carrying out the present invention will be described with reference to the drawings. The electronic pen system described below is suitably used for school lessons, etc., and an arbitrary range of textbooks (teaching materials) and content such as videos are associated with each other. This is a system in which the playback order can be set in advance. The “page spread” refers to a set of pages that are lined up or down or left and right when a textbook is opened.

[Configuration of electronic pen system]
FIG. 1 shows a configuration of an electronic pen system according to the present embodiment. As shown in FIG. 1, the electronic pen system is filled in with an electronic pen 1 used by a teacher, a textbook (teaching material) 2 in which a dot pattern (coded pattern) is printed on almost the entire surface of each page, and an electronic pen 1. A computer device 3 that receives and processes information and the like, a projector 4 that is connected to the computer device 3 and projects the same image as the display screen of the display onto the dot screen 5, and a dot screen 5 on which a dot pattern is printed Prepare. Hereinafter, each of these components will be described.

(textbook)
First, the textbook (electronic pen book) 2 will be described. FIG. 2 shows a top view of the textbook 2 in a state where a certain page (pages 22 and 23) is opened. As shown in FIG. 2, each spread page of the textbook 2 includes a content display area 20 in which the contents of the textbook 2 are described, and an operation area 40 in which icons 41 to 61 to be operated by the electronic pen 1 are printed. Have In the content display area 20 and the area including the icons 41 to 61, a dot pattern (coded pattern) indicating position coordinates described later is printed. In addition, a dot pattern having a different coordinate range is printed on each page of the textbook 2. The dot pattern is printed with ink containing carbon that absorbs infrared rays so that it can be read by the electronic pen 1, and other characters and drawings are printed with ink that does not absorb infrared rays.

  The content display area 20 includes areas (also referred to as “association areas”) 21 and 22 associated with contents such as moving images, images, PDF (registered trademark), and audio data. When the association areas 21 and 22 are tapped with the electronic pen 1, the content associated with the area (also referred to as “association content”) is reproduced or displayed by the computer device 3.

  Next, functions associated with the icons 41 to 61 will be described. The icon 41 is an operation area for displaying a table of contents page of an electronic textbook 2A described later on the display of the computer device 3. The icon 42 is an operation area for displaying the electronic textbook 2 </ b> A in front of the content when the content is reproduced or displayed on the display of the computer device 3. The icon 43 is an operation area for returning the double-page spread of the electronic textbook 2A displayed on the display of the computer device 3 by two pages. The icon 44 is an operation area for advancing the facing page of the electronic textbook 2A displayed on the display of the computer device 3 by two pages.

  Icons 45 and 46 are operation areas for switching to a writing mode with the electronic pen 1. Specifically, the icon 45 is an operation area for switching to a writing mode in which a stroke of a free curve that reproduces the handwriting of the electronic pen 1 on the textbook 2 is drawn on the electronic textbook 2A. The icon 46 is an operation area for switching to a writing mode in which a straight stroke based on the handwriting of the electronic pen 1 on the textbook 2 is drawn on the electronic textbook 2A. The icon 47 is an operation area for designating the stroke color drawn in the writing mode corresponding to the icon 45 or 46 as red, and the icon 48 is an operation area for designating the stroke color as blue. It is. As will be described later, the user can select the icon 45 or 46 with the electronic pen 1 and draw a stroke based on the handwriting of the electronic pen 1 on the electronic textbook 2A even during playback of the content. .

  The icon 49 is an operation area for switching between display and non-display of the stroke drawn in the writing mode corresponding to the icon 45 or 46. The icon 50 is an operation area for switching to a mode in which the display position of the stroke drawn on the electronic textbook 2 </ b> A is moved based on the drag operation of the electronic pen 1. The icon 51 is an operation area for enlarging and displaying an area designated by the drag operation of the electronic pen 1. The icon 52 is an operation area for deleting a stroke existing at a position where the electronic pen 1 is tapped. The icon 53 is an operation area for erasing all strokes drawn on the spread page on which the icon 53 is printed. The icon 54 is an operation area for starting playback of a stopped or paused moving image. The icon 55 is an operation area for rewinding the moving image being reproduced. The icon 56 is an operation area for pausing the moving image being played back. The icon 57 is an operation area for fast-forwarding the moving image being reproduced. The icon 58 is an operation area for stopping the moving image being reproduced.

  The icon 59 is an operation area for highlighting the association area in the electronic textbook 2A with a color. A display example when the icon 59 is selected will be described with reference to FIG. The order setting icon 60 is an operation area for instructing the start and end of a mode (also referred to as “playback order setting mode”) for setting the playback order of content associated with the association area on the same spread page. is there. The playback order setting mode will be described in detail in the section “Setting the playback order”. The reproduction icon 61 is an operation area for giving an instruction to sequentially reproduce the associated content according to the set reproduction order. The function associated with the icon 61 will be described in detail in the [Reproduction of content] section.

(projector)
The projector 4 is a projector that emits light obliquely downward from a vertically upward direction with respect to the dot screen 5, and is installed on, for example, a classroom wall. The projector 4 receives an image signal from the computer apparatus 3 and projects light constituting the same image as the screen of the computer apparatus 3 onto a predetermined projection range 50. As will be described later, an electronic textbook 2A (refer to FIG. 3 described later) obtained by digitizing the textbook 2 is displayed in the projection range 50. The projector 4 may be a front projector that emits light toward the front of the dot screen 5 or a rear projector that emits light toward the back of the dot screen 5.

(Dot screen)
In the dot screen 5, a dot pattern (coded pattern) is printed on a white resin substrate with ink containing carbon that absorbs infrared rays so that the electronic pen 1 can read it. It has a layer structure protected by a resin layer or the like.

  Light constituting the same image as the screen of the computer device 3 is projected by the projector 4 onto the projection range 50 of the dot screen 5. Specifically, an electronic textbook 2A obtained by digitizing the textbook 2 shown in FIG.

  FIG. 3 shows a display example of the projection range 50 of the dot screen 5. As described above, the display shown in FIG. 3 is the same as the display on the display screen of the computer device 3. The computer device 3 is installed with an application for displaying and operating the electronic textbook 2A. As shown in FIG. 3, the electronic textbook 2 </ b> A projected on the dot screen 5 by being displayed on the display screen of the computer device 3 is a content display area in which the same image as the content display area 20 of the textbook 2 is projected and displayed. 20A and icons 41A to 61A on which the same images as the icons 41 to 61 of the textbook 2 are projected and displayed. The content display area 20A includes a colored association area 21A in which the same image as the association area 21 shown in FIG. 2 is projected and displayed, and an association area 22A in which the same image as the association area 22 in FIG. 2 is projected and displayed. . When the user taps the icons 41 </ b> A to 61 </ b> A with the electronic pen 1 to read the dot pattern of the area where the icon is projected, the computer device 3 receives the entry information from the electronic pen 1 and taps it. Depending on the function assigned to the icon, the displayed page of the electronic textbook 2A is changed or the writing mode by the electronic pen 1 is started. Similarly, when the user taps one of the association areas 21 </ b> A and 22 </ b> A with the electronic pen 1 and reads the dot pattern of the area where the icon is projected, the computer device 3 receives the entry information from the electronic pen 1. Then, the content associated with the tapped association area is reproduced or displayed.

  Further, in the electronic textbook 2A, each association area is displayed in different colors for each type of associated content. Specifically, in FIG. 3, the association area 21 </ b> A associated with the PDF file whose extension is set to “pdf” is displayed in a state of being colored blue. On the other hand, the association area 22A associated with the moving image file having the extension “mp4” is displayed in a state of being colored red. In this way, the user can easily grasp the position and range of each association area and the type of content associated with the association area. Note that the user switches between the state shown in FIG. 3 in which the association area is colored and highlighted and the state where the association area is not colored by tapping the icon 59 or the icon 59A with the electronic pen 1. Is possible.

  In addition to the display of the electronic textbook 2A shown in FIG. 3, the operation area 40A is hidden in the electronic textbook 2A and only the content display area 20A is displayed on the entire screen in the projection range 50 and the display screen of the computer apparatus 3. It is possible. Specifically, when a predetermined input is detected based on a user operation, the computer device 3 switches between the display of the electronic textbook 2A shown in FIG. 2 and the display of only the content display area 20A.

(Dot pattern)
Subsequently, an Anoto dot pattern (coded pattern) printed on each page of the textbook 2 and the dot screen 5 will be described with reference to FIGS. 4 and 5. FIG. 4 is a diagram for explaining the relationship between dots of a dot pattern and values to which the dots are converted. As shown in FIG. 4, each dot of the dot pattern is associated with a predetermined value depending on its position. That is, each dot is associated with a value of 0 to 3 depending on which direction the top, bottom, left, or right is shifted from the reference position of the virtual grid (the intersection of the vertical and horizontal lines). The value of each dot can be further converted into a first bit value for the X coordinate and a second bit value for the Y coordinate. The position coordinates on the pages of the textbook 2 and the dot screen 5 are determined by the combination of the information thus associated. In addition, a dot pattern is formed so that a coordinate area does not overlap between the pages of the textbook 2 or each page of the textbook 2 and the dot screen 5.

  FIG. 5A shows an arrangement of a certain dot pattern. As shown in FIG. 5 (a), 6 × 6 dots within a range of about 2mm in length and width are unique patterns regardless of which portion of each page of textbook 2 or dot screen 5 takes 6 × 6 dots. It is arranged to become. The dot pattern formed by these 36 dots holds the position coordinates (for example, the position of the dot pattern on each page of the textbook 2 or the dot screen 5). FIG. 5B is a diagram in which each dot shown in FIG. 5A is converted into a value associated with the regularity shown in FIG. 4 according to the shift direction from the reference position of the lattice. This conversion is performed by the electronic pen 1 that captures an image of a dot pattern.

(Electronic pen)
Next, the electronic pen 1 will be described with reference to FIG. FIG. 6 is a schematic view showing the structure of the electronic pen 1. As shown in FIG. 6, the electronic pen 1 includes an ink cartridge 104, an LED 105, a CMOS camera 106, a pressure sensor 107, a processor 108 including a CPU, a memory 109 such as a ROM and a RAM, A communication unit 111 including a real-time clock 110, an antenna, and the like, and a battery 112 are provided. The tip of the ink cartridge 104 is a pen tip portion 103, and the user brings the pen tip portion 103 of the electronic pen 1 into contact with each page of the textbook 2 and the dot screen 5 to make a stroke (handwriting stroke). Fill out. Here, the first contact of the pen tip portion 103 of the electronic pen 1 with each page of the textbook 2 is referred to as “pen down”, and the pen tip portion 103 moves away from the contact (contact) state. Is called “pen-up”. A trajectory written between the pen-down and pen-up of the electronic pen 1 is one stroke, and characters, figures, etc. are composed of one or a plurality of strokes.

  The battery 112 is for supplying electric power to each component in the electronic pen 1. For example, the battery 112 is configured to turn on / off the electronic pen 1 itself by attaching and detaching a cap (not shown) of the electronic pen 1. May be. The real time clock 110 transmits time information indicating the current time (time stamp) and supplies the time information to the processor 108. The pressure sensor 107 detects the pressure applied through the ink cartridge 104 from the pen tip portion 103 when the user writes or taps a character or mark on each page of the textbook 2 with the electronic pen 1, that is, the writing pressure, The value is transmitted to the processor 108.

  The processor 108 switches on / off the switches of the LED 105 and the CMOS camera 106 based on the writing pressure data supplied from the pressure sensor 107. That is, when the user writes a character or the like on each page of the textbook 2 with the electronic pen 1, the pen pressure is applied to the pen tip portion 103, and when the pressure sensor 107 detects a writing pressure higher than a predetermined value, the processor 108. Determines that the user has started entry, and activates the LED 105 and the CMOS camera 106. Then, the communication unit 111 detects pen down information (hereinafter referred to as “pen down information PD”) detected by the pressure sensor 107 and identification information (hereinafter referred to as “pen ID”) of the electronic pen 1 described later. Are transmitted to the computer apparatus 3 as entry information. When the user finishes entering one stroke and moves the electronic pen 1 away from each page of the textbook 2, the pressure sensor 107 detects pen-up by detecting that the writing pressure exceeding a predetermined value is not detected. Then, the communication unit 111 associates the pen-up information detected by the pressure sensor 107 (hereinafter referred to as “pen-up information PU”) and the pen ID, and transmits the information as entry information to the computer apparatus 3.

  The LED 105 and the CMOS camera 106 are attached near the pen tip portion 103 of the electronic pen 1, and an opening 102 is formed in a portion of the housing 101 that faces the LED 105 and the CMOS camera 106. The LED 105 illuminates infrared rays toward the vicinity of the pen tip portion 103 on each page of the textbook 2. The region is slightly shifted from the position where the pen tip portion 103 contacts each page of the textbook 2. The CMOS camera 106 is provided with an infrared transmission filter that transmits infrared rays and blocks non-infrared rays. The CMOS camera 106 captures a dot pattern in an area illuminated by the LED 105, and the image data of the dot pattern is captured. This is supplied to the processor 108. Here, since carbon absorbs infrared rays, the infrared rays irradiated by the LED 105 are absorbed by the carbon contained in the dots. Therefore, the dot portion has a small amount of infrared reflection, and the portion other than the dot has a large amount of infrared reflection. By photographing with the CMOS camera 106, a threshold value is provided based on the difference in the amount of reflected infrared light, so that a dot region containing carbon can be distinguished from other regions. Therefore, even when characters, drawings, or the like are printed on each page of the textbook 2, the processor 108 can recognize the dot pattern because the printed ink does not absorb in the infrared region. Note that the shooting area by the CMOS camera 106 is a range including a size of about 2 mm × about 2 mm as shown in FIG. 5A, and the shooting by the CMOS camera 106 is performed at regular intervals of about 50 to 100 times per second. Is called. Further, the CMOS camera 106 has a sufficient depth of field in order to photograph dots clearly.

  While the user performs the entry, the processor 108 uses the dot pattern of the image data supplied by the CMOS camera 106 to determine the X and Y coordinates (hereinafter simply referred to as “page” of the textbook 2 of the stroke (handwriting) to be entered by the user. (Also called “coordinate data” or “coordinate information”) is continuously calculated. That is, the processor 108 converts the image data of the dot pattern as shown in FIG. 5A supplied by the CMOS camera 106 into the data array shown in FIG. The data is converted into a Y coordinate bit value, and X and Y coordinate data are calculated from the data array by a predetermined calculation method. Note that the processor 108 has a rotation correction processing function for correcting the arrangement of dots on the image due to the angle of the electronic pen 1 facing the dot pattern, and that function is executed during coordinate calculation. Then, the processor 108 associates the current time (time stamp: entered time information) transmitted from the real-time clock 110, writing pressure data, and X and Y coordinate data. Hereinafter, these associated data are collectively referred to as “coordinate attribute information”. The 6 × 6 dot pattern in each page of the textbook 2 and the dot screen 5 does not overlap in each page of the textbook 2 and the dot screen 5, so when the user enters characters or the like with the electronic pen 1, It can be specified by coordinate calculation by the processor 108 which position on the page of the textbook 2 or the position of the dot screen 5 the entered position corresponds to.

  The memory 109 stores property information such as a pen ID such as “pen01” for identifying the electronic pen 1, a pen manufacturer number, and a pen software version. Then, the communication unit 111 associates the pen ID, time information (time stamp), writing pressure data, and X and Y coordinate data, and transmits them to the computer apparatus 3 as entry information. In this case, transmission to the computer apparatus 3 by the communication unit 111 is immediately and sequentially performed by wireless transmission such as Bluetooth (registered trademark). Here, one or more pieces of coordinate attribute information generated between the pen-down and pen-up of the electronic pen 1 and transmitted to the computer device 3 are stored as stroke information by the computer device 3. In other words, one stroke is composed of one or a plurality of X and Y coordinates (coordinate points), and the computer apparatus 3 is configured by one or more composing one stroke by the pen-down information PD and the pen-up information PU. Recognize pieces of coordinate attribute information.

  The electronic pen 1 may include a pen portion that is not filled with ink instead of the ink cartridge 104. In this case, ink is not output from the nib portion 103. Further, when the electronic pen 1 is prepared for each of the textbook 2 and the dot screen 5, the electronic pen 1 having the ink cartridge 104 is prepared for the textbook 2, and for the dot screen 5 An electronic pen 1 having a pen unit may be prepared.

(Computer device)
Next, the computer apparatus 3 will be described. The computer device 3 includes, as hardware, an antenna device capable of data communication with the electronic pen 1, a processor such as a CPU, a memory such as a ROM and a RAM, a display, a mouse, a keyboard, and the like. The computer apparatus 3 may be configured by a tablet PC such as iPad (registered trademark), a PDA (Personal Data Assistance), a PC (personal computer) equipped with a display, or the like.

  FIG. 7 is a functional block diagram of the computer apparatus 3. Functionally, the computer apparatus 3 includes an input unit 31 such as a mouse and a keyboard, a communication unit 32, a storage unit 33, a processing unit 34, a display unit 36, and an interface 38. The communication unit 32 includes an antenna receiving circuit and the like, receives entry information from the electronic pen 1, and transmits the received information to the processing unit 34. The display means 36 is constituted by a display or the like, and displays the contents instructed by the processing means 34. The interface 38 is an interface for connecting to the projector 4. The interface 38 transmits an image signal for displaying an image on the display unit 36 to the projector 4 based on a processing command of the processing unit 34, and projects the same image as the image on the display unit 36 in synchronization.

  The processing means 34 is constituted by a processor such as a CPU and controls the entire computer device 3. The processing unit 34 includes a display control unit 341, a reproduction order setting unit 342, and a reproduction control unit 343. Then, when the icons 41 to 61 printed on the textbook 2 are tapped with the electronic pen 1 and the dot pattern is read, the processing unit 34 is described later based on the entry information generated and transmitted by the electronic pen 1. Refer to the coordinate definition information and recognize the tapped icon. Then, the processing unit 34 performs a predetermined process according to the function associated with the tapped icon.

  Specifically, when the icons tapped with the electronic pen 1 are the icons 41 to 44, the display control unit 341 of the processing unit 34 displays a predetermined page of the electronic textbook 2A according to the function associated with the tapped icon. It is displayed on the display means 36 in a spread state.

  When the icon tapped with the electronic pen 1 is the icon 45 or 46, the display control unit 341 starts the writing mode. Specifically, the display control unit 341 draws a free-form curve or a straight line stroke on the electronic textbook 2A based on the position coordinates indicated by the entry information received in the writing mode. In this case, the display control unit 341 draws a stroke regardless of whether the content is being reproduced. In the writing mode, the display control unit 341 may display the electronic textbook 2A and the content being displayed side by side. If the content is a moving image, the display control unit 341 displays the content within the association area corresponding to the content. May be. When the content is audio data, the display control unit 341 displays only the electronic textbook 2A while hiding the display of software for reproducing the audio data while outputting audio to a speaker (not shown). Also good.

  When the icons tapped with the electronic pen 1 are the icons 47 to 53, the display control unit 341 executes a predetermined process according to the function associated with the tapped icon. In addition, when the icons tapped with the electronic pen 1 are the icons 54 to 58, the reproduction control unit 343 performs reproduction control of the moving image content to be reproduced according to the function associated with the tapped icon. When the icon tapped with the electronic pen 1 is the icon 59, the display control unit 341 displays the association area in different colors according to the type of the associated content, as described in FIG.

  When the icon tapped with the electronic pen 1 is the icon 60, the processing means 34 starts the reproduction order setting mode. In the reproduction order setting mode, the reproduction order setting means 342 of the processing means 34 determines the reproduction order of the contents associated with the association area on the same spread page. This process will be described in detail in the section “Setting the playback order”. When the icon tapped with the electronic pen 1 is the icon 61, the reproduction control unit 343 reproduces the associated content to be reproduced next in accordance with the set reproduction order. This process will be described in detail in the [Content Playback] section. Further, when the association area of the textbook 2 is tapped with the electronic pen 1, the processing unit 34 refers to association area information described later, and reproduces or displays the content associated with the tapped association area.

  Further, the processing unit 34 causes the projector 4 to project the calibration mark onto the dot screen 5 by displaying at least two or more calibration marks on the display unit 36, and the calibration mark is projected. The coordinate conversion function for converting the coordinate system related to the dot pattern of the dot screen 5 to the coordinate system related to the display of the display means 36 by causing the electronic pen 1 to read the dot pattern at the determined position and receiving the entry information. Obtain (calibration processing function). Further, when the processing unit 34 receives the coordinate system data related to the dot pattern of the dot screen 5 using the coordinate conversion function obtained by the calibration process, the processing unit 34 displays the coordinate system applied to the display of the display unit 36. Convert to data. When the processing unit 34 recognizes that the icons 41A to 61A displayed on the dot screen 5 are selected based on the coordinate system data applied to the display after conversion, the icons 41 to 61 of the textbook 2 are selected. The same processing as that performed is executed. Further, when the processing unit 34 recognizes that the association area displayed on the dot screen 5 is selected based on the coordinate system data applied to the display after conversion, the processing unit 34 refers to the association area information described later and is selected. The content associated with the associated area is reproduced or displayed.

  The storage means 33 is configured by a memory such as a ROM or a RAM. The storage unit 33 stores the entry information received from the electronic pen 1 for each pen ID according to the processing instruction of the processing unit 34. The storage unit 33 stores an electronic textbook 2A obtained by digitizing each textbook 2. In addition, the storage unit 33 indicates the coordinate range indicated by the dot display pattern corresponding to the content display area 20 and the icons 41 to 61 of each page of the textbook 2 and the dot pattern corresponding to the dot screen 5 for each type of the textbook 2. Coordinate range information (also referred to as “coordinate definition information”) is stored.

  Furthermore, the storage unit 33 stores content associated with each association area and also stores information associated with each association area (also referred to as “association area information”). FIG. 8 shows the data structure of the association area information. As shown in FIG. 8, the association area information includes an item “area number” indicating the identification number of each association area, an item “textbook ID” indicating the identification information of the textbook 2, and the page number of the association area. “Page number” item, “in-page position” item indicating the relative position in the page, “association content” item indicating the path and file name of the associated content, and the same spread page Information corresponding to the item “reproduction order” indicating the reproduction order of the associated content is associated with each other. Here, it is assumed that the textbook ID of the textbook 2 shown in FIG. 2 is “R01”, and the position in the page is the corner position coordinates (Xn, Yn), the height (Y direction), and the width (X direction). It is stipulated in. In addition, the value of the item “playback order” is set based on the processing described in the “playback order setting” section described later.

  The association area information other than the reproduction order may be stored in the storage unit 33 in advance, or may be generated by the processing unit 34 based on a user input. In the latter case, for example, in the state where the electronic textbook 2A is displayed, the processing unit 34 allows the user to specify an area to be an association area by a drag operation with a mouse or the like, and the content to be associated with the association area is a GUI for selecting a file. (Graphical User Interface) or the like is displayed to allow the user to specify. The associated content may be video data generated as a result of the instructor taking a picture using a video camera or the like for the lesson, still image data, or a worksheet. May be.

[Set playback order]
Next, a method for setting the playback order of related content when a plurality of related areas exist in the same spread page will be specifically described. Schematically, the reproduction order setting unit 342 determines the reproduction order of the associated content based on the order of the association areas tapped by the electronic pen 1 in the reproduction order setting mode.

  FIG. 9 is a diagram illustrating a change in position associated with the setting of the reproduction order of the electronic pen 1 on the textbook 2. First, based on a user operation, the icon 60 is tapped with the electronic pen 1 (first step). In this case, the processing means 34 recognizes that the icon 60 has been tapped based on the received entry information, and starts the reproduction order setting mode. Thereafter, the user taps the plurality of association areas existing in the same spread page with the electronic pen 1 in the order in which the associated contents are to be reproduced, according to the user's intention.

  In the example shown in FIG. 9, the association area 21 is tapped with the electronic pen 1 based on the user's operation (second step). In this case, the reproduction order setting unit 342 of the processing unit 34 recognizes that the association area 21 has been tapped with the electronic pen 1 based on the received entry information. Then, the reproduction order setting unit 342 sets the reproduction order of the content associated with the association area 21 among the association contents associated with the spread pages (here, pages 22 and 23) to the first.

  Thereafter, the association area 22 is tapped with the electronic pen 1 based on the user's operation (third step). In this case, the reproduction order setting unit 342 of the processing unit 34 recognizes that the association area 22 has been tapped with the electronic pen 1 based on the received entry information. Then, the reproduction order setting unit 342 sets the reproduction order of the associated content associated with the association area 22 among the associated content associated with the spread pages to the second.

  Next, based on a user operation, the icon 60 is tapped again with the electronic pen 1 (fourth step). In this case, the reproduction order setting unit 342 recognizes that the icon 60 has been tapped based on the received entry information, and ends the reproduction order setting mode. As a result, the reproduction order setting unit 342 determines the reproduction order of the spread pages on which the operation with the electronic pen 1 has been performed. Specifically, the reproduction order setting unit 342 indicates that the reproduction order of “worksheet 1.pdf”, which is the content associated with the association area 21, is No. 1 as shown in FIG. 8 described above. The information is stored in the association area information to the effect that the reproduction order of “fault XX town.mp4”, which is the content associated with the association area 22, is second.

  As described above, the playback order setting unit 342 sets the playback order of the associated content in the order that the user intentionally taps without requiring a complicated operation on an arbitrary spread page having a plurality of related areas. Can be determined accordingly.

[Display after setting playback order]
Next, the display screen of the computer apparatus 3 after the reproduction order is set will be described with reference to FIG. FIG. 10 shows a display screen of the computer apparatus 3 on which a spread page in which the reproduction order is set is displayed.

  As shown in FIG. 10, when displaying the spread page in which the reproduction order is set, the display control unit 341 displays the reproduction order number of the corresponding associated content on the association areas 21A and 22A in which the reproduction order is set. Superimposed display. In the example illustrated in FIG. 10, “(1)” is displayed in the association area 21A in which the playback order is set to No. 1, and “((1))” is displayed in the association area 22A in which the playback order is set to No. 2. 2) ”is attached.

  In this way, when the display control unit 341 displays the spread page in which the reproduction order is set on the display of the computer device 3, the display showing the set reproduction order so that the user can visually recognize the set reproduction order. Is attached to the corresponding association area. Thereby, the user can easily grasp the set reproduction order on the display of the computer device 3 or on the dot screen 5. When the icon 59 or 59A is tapped with the electronic pen 1, the display control unit 341 may switch the display / non-display of the reproduction order as in the case of highlighting by color.

[Play content]
Next, a method for reproducing content for which the reproduction order has been set will be described. In general, each time the icon 61 is tapped, the playback control unit 343 starts playback of related content in the playback order to be played back.

  FIG. 11 is a diagram illustrating that an operation with the electronic pen 1 is performed on a facing page in which the reproduction order is set. Preferably, only the content display area 20 of the electronic textbook 2A is displayed on the display screen of the computer device 3 and the projection range 50 of the dot screen 5 at the time of reproduction shown in FIG.

  In FIG. 11, first, the icon 61 is tapped with the electronic pen 1 based on a user operation. In this case, the reproduction control means 343 of the processing means 34 recognizes that the icon 61 has been tapped based on the received entry information. In this case, the reproduction control unit 343 refers to the association area information shown in FIG. 8 and reproduces or displays the associated content in which the reproduction order is set to No. 1 on the spread pages (pages 22 and 23). Specifically, the reproduction control unit 343 displays “worksheet 1.pdf” associated with the association area 21 on the display screen. In this case, “worksheet 1.pdf” is also projected and displayed in the projection range 50 of the dot screen 5.

  FIG. 12 shows a display example of the file “worksheet 1.pdf”. As illustrated in FIG. 12, the file “worksheet 1.pdf” has answer columns 70 </ b> A to 70 </ b> E. Then, when the student or the like writes with the electronic pen 1 to the answer columns 70A to 70E of the “worksheet 1.pdf” displayed on the dot screen 5, the processing means 34 displays the coordinate information included in the entry information, Based on the coordinate conversion function, it is converted into coordinate information on the screen of the display means 36, and a stroke is drawn on the display means 36 based on the coordinate information. Further, the processing unit 34 transmits an image signal for displaying an image on the display unit 36 to the projector 4 to project the same image as the image on the display unit 36 in synchronization. Thereby, the stroke drawn by the writer of the electronic pen 1 is similarly projected onto the dot screen 5.

  The reproduction of the associated content will be described with reference to FIG. 11 again. After the file “worksheet 1.pdf” is displayed on the display, the icon 61 is tapped again with the electronic pen 1. In this case, the reproduction control means 343 of the processing means 34 recognizes that the icon 61 has been tapped based on the received entry information. In this case, the reproduction control unit 343 refers to the association area information shown in FIG. 8 and reproduces or displays the associated content in which the reproduction order is set to No. 2 on the spread page (pages 22 and 23). Specifically, the reproduction control unit 343 reproduces the moving image content “fault XX town.mp4” associated with the association area 22. In this case, the moving image content “tomographic XX town.mp4” is also projected and displayed in the projection range 50 of the dot screen 5. Note that the reproduction control means 343 may forcibly end the content “worksheet 1.pdf” when reproducing the moving image content “fault XX town.mp4”. Only the process of changing the display position to the back of “mp4” may be performed.

  As described above, the reproduction control unit 343 can appropriately reproduce the content without causing an error in the reproduction order during the lesson in accordance with the lesson scheduled by the lecturer who is the user.

  Even when the operation area 40A is displayed in the projection range 50 of the dot screen 5 and the icon 61A is tapped with the electronic pen 1, the computer device 3 similarly reproduces the associated content according to the set reproduction order. To do. Specifically, when the display position of the icon 61A within the projection range 50 is tapped with the electronic pen 1, the reproduction control unit 343 applies the coordinate information of the received entry information to the display of the display unit 36 using the coordinate conversion function. Conversion to the coordinate system is made, and based on the converted position coordinates, the coordinate definition information is referenced to recognize that the icon 61A has been designated. In this case, the reproduction control unit 343 refers to the association area information and reproduces the associated content in the reproduction order to be reproduced for the spread page currently displayed on the display.

[Operational effects of the electronic pen system of the present embodiment]
According to the electronic pen system of the present embodiment, the computer device 3 stores association area information related to the association area of the textbook 2 associated with the content. Then, the computer device 3 sets the order of reproduction or display of the content associated with the association area on the spread page of the textbook 2 provided with a plurality of association areas based on the entry information received from the electronic pen 1. Thereby, in a lesson or the like, the user can reproduce or display the content in the expected order without making a mistake in the order of the content to be reproduced.

  Further, the operation area 40 of the textbook 2 is provided with an icon 60 for designating the start and end of the reproduction order setting mode, and the computer device 3 displays the association area selected by the electronic pen in the reproduction order setting mode. Based on the selection order, the playback or display order of the associated content is set. Thereby, the user can easily determine the order of reproduction or display of the contents associated with the same page or the same spread page by operating the electronic pen.

  Further, the operation area 40 of the textbook 2 is provided with an icon 61 for reproducing or displaying the content based on the set reproduction order, and the computer apparatus 3 detects that the icon 61 has been selected by the electronic pen 1. Each time, the content corresponding to the order to be reproduced or displayed is reproduced or displayed. Thereby, the user can reproduce or display the content in which the order is set on the computer device 3 in the order only by performing an operation of selecting the icon 61 with the electronic pen 1.

[Modification]
Next, a modification of this embodiment will be described. The following modifications may be applied in any combination to the above-described embodiment.

(Modification 1)
The electronic pen system includes an electronic board such as a pressure-sensitive board (sheet) configured to be able to detect pressure, or an electromagnetic board in which wires are embedded immediately below the surface of the board, instead of the dot screen 5. Also good. For example, in the case of a pressure-sensitive board, when pressure is applied with a predetermined pen (stylus), two electrically conductive sheets provided in the board come into contact with each other to generate an electric signal, and a computer is generated based on the electric signal. The device 3 detects the contact position. In the case of an electromagnetic board, the electronic pen system includes a stylus in which a coil is embedded instead of the electronic pen 1, and when the stylus is brought into contact with the electromagnetic board, the electromagnetic board contacts based on electromagnetic induction with the stylus. An electrical signal corresponding to the position is generated, and the computer device 3 detects the contact position based on the electrical signal.

  Then, regardless of which input system board (sheet) is used, the computer device 3 converts the coordinate system on the board into the coordinate system on the display of the display means 36 in the same manner as the dot screen 5. A coordinate conversion function for performing the calculation is obtained in advance. When the computer device 3 receives the coordinate data corresponding to the contact position on the board using the obtained coordinate conversion function, the computer device 3 converts the coordinate data into the coordinate system data applied to the display of the display means 36 and displays it. It is detected which icon 41A to 61A in the electronic textbook 2A has been selected.

(Modification 2)
In the association area information shown in FIG. 8, the item “association content” stores the path and file name of the association content as information for specifying the association content. Instead, the item “associated content” stores a URL (Uniform Resource Locator) for opening a video site where electronic content can be browsed, and a server device that can communicate with the computer device 3 via a network. A URL for playing a video or the like may be stored.

  In this case, when the associated content in the reproduction order to be reproduced is specified by the URL in the association area information, the reproduction control unit 343 displays a predetermined WEB site based on the URL or the server device. To start streaming playback. In this case, the WEB site and video displayed based on the URL are examples of the “content” in the present invention.

(Modification 3)
In the electronic textbook 2A shown in FIG. 3, the association areas 21A and 22A are highlighted with colors. Instead of this, or in addition to this, the display control means 341 displays the file name of the associated content or / and the image or video in which the content is displayed in a small size at the position corresponding to the association area 21A, 22A. You may let them. For example, the display control unit 341 colors the association areas 21A and 22A with the same color and displays the file name of the association content at a position overlapping or close to these areas.

  Also by this, the display control means 341 can make the user recognize the content type or file name associated with each association area 21A, 22A.

(Modification 4)
In the description of FIG. 2 and the like, each spread page of the textbook 2 is provided with an operation area 40 on which a dot pattern indicating a different coordinate range is printed. However, the configuration to which the present invention is applicable is not limited to this.

  Instead of this, for example, in the operation area 40 of the textbook 2, a dot pattern indicating a coordinate range common to each spread page may be printed. Then, when any of the icons 41 to 61 is tapped with the electronic pen 1, the processing means 34 recognizes which icon is designated based on the entry information, and the electronic textbook being displayed on the display screen of the computer device 3. A process based on the function assigned to the selected icon is performed on the 2A facing page.

  In another example, the operation area 40 may be configured separately from the textbook 2. FIG. 13 shows a top view of the operation sheet 8 corresponding to the operation area 40 of the textbook 2. A dot pattern is printed on a substantially front surface of the operation paper 8. Then, when any icon 41 to 61 on the operation sheet 8 is tapped with the electronic pen 1, the processing unit 34 recognizes which icon is designated based on the entry information, and the electronic textbook being displayed on the display screen. The processing based on the function assigned to the designated icon is performed on the 2A facing page.

(Modification 5)
When the position and range of the association area are already determined at the time of creation of the textbook 2, the association area of the textbook 2 may be colored similarly to the association area of the electronic textbook 2A.

  FIG. 14 is a top view of the textbook 2 according to the modification. In FIG. 14, the association area 21 of the textbook 2 is printed in blue, and the association area 22 is printed in red. As described above, the association areas 21 and 22 of the textbook 2 are color-coded according to the type of the association content, so that the user can easily tap each association area with the electronic pen 1 in the reproduction order setting mode. Improves.

(Modification 6)
The computer apparatus 3 may set the playback order of the associated content for each page instead of setting the playback order of the associated content within the same spread page. For example, in this case, the icon 60 and the icon 61 are printed for each page of the textbook 2. When the icon 60 is selected by the electronic pen 1, the computer device 3 starts the reproduction order setting mode for the page on which the icon 60 is printed. When the icon 61 is selected by the electronic pen 1, Reproduction according to the reproduction order is performed on the page on which the icon 61 is printed.
(Modification 7)
In the above embodiment, the Anoto method is used for the electronic pen, the dot pattern (coded pattern), and the entry information. However, the Anoto method is not necessarily limited. Further, the dot pattern (coded pattern) is not limited to the one indicating the position coordinates, and may be a code value for identifying the icon or a code value for identifying the association area.

DESCRIPTION OF SYMBOLS 1 ... Electronic pen 2 ... Textbook 2A ... Electronic textbook 3 ... Computer apparatus 4 ... Projector 5 ... Dot screen 8 ... Operation paper

Claims (10)

Teaching materials with printed coding patterns;
A computer device;
An electronic pen that reads a coded pattern, calculates a position coordinate or code value of a pen tip, and transmits entry information including the position coordinate or code value to the computer device;
In an electronic pen system comprising:
The computer device includes:
Storage means for storing information of an association area of the learning material associated with the content;
Reproduction order setting means for setting the reproduction or display order of the contents associated with the association area in the teaching material page or the spread page provided with a plurality of association areas based on the entry information received from the electronic pen ,
An electronic pen system characterized by comprising:   The reproduction order setting means recognizes a selection order in which the association area is selected by the electronic pen based on the entry information, and determines the order of reproduction or display based on the selection order. The electronic pen system according to 1. The paper on which the teaching material or the coded pattern is printed is provided with an order setting icon for starting the process of determining the reproduction or display order,
The electronic device according to claim 2, wherein the reproduction order setting unit recognizes the selection order in which the association area is selected by the electronic pen after the order setting icon is selected by the electronic pen. Pen system. Reproduction icons for reproducing or displaying the content based on the reproduction or display order set by the reproduction order setting unit are provided on the sheet on which the teaching material or the coded pattern is printed,
The computer device includes:
4. A reproduction control means for reproducing or displaying the content corresponding to the order to be reproduced or displayed each time it is detected that the reproduction icon is selected by the electronic pen. The electronic pen system according to any one of the above. The computer device includes:
Display control means for displaying electronic teaching materials obtained by digitizing the teaching materials on a display means;
The display control means, when displaying the association area of the electronic teaching material corresponding to the content for which the playback or display order is set, on the display means, the reproduction or display of the content corresponding to the association area The electronic pen system according to any one of claims 1 to 4, wherein a display of the order is attached. The computer device includes:
Display control means for displaying electronic teaching materials obtained by digitizing the teaching materials on a display means;
The display control means, when displaying the association area of the electronic teaching material on the display means, displays the association area in a color-coded manner for each corresponding content type on the display means. The electronic pen system as described in any one of thru | or 5. The computer device includes:
Displaying electronic teaching materials obtained by digitizing the teaching materials, and display control means for drawing strokes based on handwriting written in the teaching materials by the electronic pen on the electronic teaching materials based on the entry information,
The display control means draws a stroke based on a handwriting entered in the teaching material by the electronic pen on the electronic teaching material based on the entry information even when the content is being reproduced or displayed. The electronic pen system according to any one of claims 1 to 6. A screen with a coded pattern on the surface;
A projector that emits light based on an image signal transmitted from the computer device to the surface of the screen, and
The computer device includes:
Display control means for projecting and displaying a reproduction icon on the screen by the projector for reproducing or displaying the content based on the reproduction or display order set by the reproduction order setting means;
8. The reproduction control unit according to claim 1, further comprising: a reproduction control unit configured to reproduce or display the content corresponding to an order to be reproduced or displayed when the reproduction icon projected and displayed is selected. Electronic pen system as described in.   The learning material includes a content display area for displaying the content of the learning material for each page or spread page, and an operation area provided with a plurality of icons for realizing a predetermined function when selected by the electronic pen. The electronic pen system according to claim 1, wherein the electronic pen system is an electronic pen system.   A program for causing a computer apparatus according to any one of claims 1 to 8 to function.