JP5915221B2 - Computer apparatus, information processing system, and program - Google Patents

Description

  The present invention relates to the technical field of displaying photographs and the like.

  2. Description of the Related Art Conventionally, electronic pens that digitize written information have been developed, and “Anot pen” developed by Swedish company Anoto is known as a representative one. For example, Patent Document 1 describes that the anotopen reads a predetermined dot pattern printed on a sheet, generates entry information, and transmits the entry information to the terminal device. Patent Document 2 describes a rotation correction processing function that corrects the arrangement of dots on an image caused by the angle of an electronic pen facing the dot pattern. Patent Document 3 describes a technique for drawing the answer contents of each answerer entered with an electronic pen in a collaborative learning system using the electronic pen.

  In addition, Patent Document 4 describes a technique related to the present invention. Patent Document 4 describes a camera incorporating an electronic compass.

Japanese Patent No. 3874498 Special table 2003-529853 gazette JP 2009-276479 A JP 2009-88683 A

  By the way, it is convenient if a photograph of a landscape photographed by a camera can be arranged and displayed in accordance with the direction in which the photograph was photographed. SUMMARY OF THE INVENTION Accordingly, it is a main object of the present invention to provide a computer apparatus, an information processing system, and a program capable of appropriately arranging and displaying photographs taken by a camera in accordance with a photographing direction.

In one aspect of the present invention, the computer apparatus includes an image data input unit to which image data of a photograph taken by a camera is input, and the photograph corresponding to the image data input to the image data input unit. Display control means for arranging and displaying the photograph according to the direction in which the photograph was taken , and receiving means for receiving entry information relating to a handwritten stroke written by an electronic pen on a sketch paper on which a coded pattern is formed, The display control means, based on the entry information received by the receiving means, a sketch corresponding to the handwritten stroke entered on the sketch paper in accordance with the direction in which the sketch is entered, Arrange with the photo to display .

In the above computer apparatus, image data of a photograph taken by a camera, such as a landscape, is input to the image data input means. Then, the display control means displays the photograph corresponding to the image data input to the image data input means, arranged in accordance with the direction in which the photograph is taken. As a result, the photograph taken by the camera can be appropriately arranged and displayed according to the direction in which the photograph was taken. Such a display is convenient when used in a weather class or the like. For example, it can be used effectively when learning the position of the sun according to the time, how to evaluate the weather according to the shape of the clouds and the amount of clouds. Moreover, according to said computer apparatus, both the photograph image | photographed with the camera and the sketch filled in with the electronic pen can be appropriately arranged and displayed according to the direction. Furthermore, the sketch can be properly evaluated by comparing the sketch with the photograph.

  The “direction” represents a direction in a horizontal plane at a certain point in relation to a reference direction, in other words, an azimuth. “Direction” includes four directions of east, west, south, and north with azimuth intervals of 90 degrees, and eight directions of east, west, south, north, northeast, southeast, northwest, and southwest with azimuth intervals of 45 degrees. Is used. In addition, 16 orientations with an azimuth interval of 22.5 degrees and 32 orientations with an azimuth interval of 11.25 degrees may be used.

  In one aspect of the above computer apparatus, the image data input means is input with the image taking direction of the photograph detected by a sensor built in the camera, together with the image data, and the display control means The photograph is displayed in an arrangement corresponding to the photographing direction input to the data input means. According to this aspect, using the direction detected by the sensor built in the camera (shooting direction), it is possible to arrange and display the photos taken by the camera in accordance with the direction. The “shooting direction” is a direction in which the lens of the camera is facing, and means the direction of the landscape shot by the camera.

  In another aspect of the above computer apparatus, the image data input means inputs the shooting date and time when the photograph was taken by the camera together with the image data, and the display control means sends the image data input means to the image data input means. The inputted shooting date and time are arranged and displayed in the direction along with the photograph. Thereby, the date and time when the photo was taken (shooting date and time) can be appropriately displayed together with the photo.

  In another aspect of the above computer apparatus, the display control means arranges and displays a plurality of the photographs in association with the plurality of directions. In this aspect, a plurality of photographs taken in a plurality of directions can be appropriately arranged and displayed in association with the direction in which those photographs are drawn.

  Preferably, in the above computer apparatus, the display control means displays the whole or a part of the plurality of photographs side by side in the horizontal direction or the vertical direction, and displays a part of the plurality of photographs. In accordance with the operation, the portion to be displayed in the plurality of photographs can be scrolled.

  Preferably, in the above-described computer apparatus, the display control unit displays the plurality of photos in a ring shape and rotates the whole of the plurality of photos arranged in a ring shape according to a user operation. Thereby, a photograph can be displayed in the aspect based on the arrangement order of the actual direction.

  Preferably, in the above computer apparatus, the sketch sheet is provided with a sketch entry field for entering the sketch, and a direction selection field for selecting a direction in which the sketch is entered, and the sketch entry field, Different encoding patterns are formed in the direction selection field, and the display control means is configured to select the direction based on the entry information corresponding to the entry with the electronic pen in the sketch entry field and the direction selection field. The sketches entered in the sketch entry column are displayed in an arrangement corresponding to the direction selected in the selection column. Accordingly, the sketch can be appropriately displayed in association with the direction in which the sketch is written with the electronic pen.

  Preferably, in the above computer apparatus, the sketch sheet is provided with a sketch entry field for entering the sketch and a direction entry field for entering the direction in which the sketch is entered, and the sketch entry is provided. Different coding patterns are formed in the field and the direction entry field, and the display control means is based on the entry information corresponding to the entry with the electronic pen in the sketch entry field and the direction entry field, The sketches entered in the sketch entry column are displayed in an arrangement corresponding to the direction entered in the direction entry column. This also makes it possible to display the sketch appropriately in association with the direction in which the sketch is written with the electronic pen.

  Also preferably, the sketch paper is provided with a date and time entry column for entering the date and time when the sketch was entered, and a coding pattern is formed in the date and time entry column, and the display control means includes the date and time Based on the entry information corresponding to the entry with the electronic pen in the entry field, the date and time entered in the date and time entry field are arranged and displayed in the direction together with the sketch. Thereby, the date and time when the sketch was entered can be appropriately displayed together with the sketch.

  In another aspect of the above-described computer apparatus, the display control unit displays all or a part of the plurality of photographs and the plurality of sketches side by side in the horizontal direction or the vertical direction in association with the plurality of directions. When a part of the plurality of photographs and the plurality of sketches is displayed, a part to be displayed in the plurality of photographs and the plurality of sketches can be scrolled according to a user operation.

  In another aspect of the present invention, an information processing system comprising the computer device and a camera that generates image data of a photographed photograph, wherein the image data input means of the computer device is generated by the camera. Image data is input.

  In still another aspect of the present invention, the computer device, a camera that generates image data of a photograph taken, a sketch paper on which a coded pattern is formed and handwritten strokes are written by an electronic pen, and the sketch paper An information processing system including an electronic pen that reads the formed coding pattern and generates entry information related to the coding pattern. The image data input unit of the computer device includes the image generated by the camera. Data is input, and the receiving means of the computer device receives entry information relating to the coded pattern formed on the sketch paper, which is read by the electronic pen.

  In still another aspect of the present invention, the program causes the computer apparatus to function. By executing the program, the above-described computer device can be realized.

  According to the present invention, photographs taken by a camera can be appropriately arranged and displayed according to the direction in which the photograph was taken.

1 is a system configuration diagram of an information processing system according to a first embodiment. It is a functional block diagram of a camera. It is a functional block diagram of the computer apparatus concerning 1st Embodiment. An example in which the operator is photographing a landscape in a different direction is shown. The pull-down menu of the tool menu used when displaying a photograph is shown. The example of the display screen of the photograph which concerns on 1st Embodiment is shown. The processing flow concerning a 1st embodiment is shown. The example of the display screen of the photograph which concerns on the modification of 1st Embodiment is shown. It is a system configuration figure of the information processing system concerning a 2nd embodiment. The sketch paper concerning a 2nd embodiment is shown. It is explanatory drawing which shows the relationship between the arrangement | positioning of the dot in a dot pattern, and the value converted. (A) schematically shows a dot pattern, and (b) is a diagram showing an example of information corresponding thereto. It is the schematic which shows the structure of an electronic pen. It is a functional block diagram of the computer apparatus concerning 2nd Embodiment. An example of how each entrant sketches a landscape in a different direction. The example of the sketch entered in the sketch paper concerning 2nd Embodiment is shown. The pull-down menu of the tool menu used when displaying a sketch and a photograph is shown. The example of the display screen of the sketch and photograph which concerns on 2nd Embodiment is shown. The processing flow concerning a 2nd embodiment is shown. The sketch paper which concerns on the modification of 2nd Embodiment is shown. The sketch example entered in the sketch paper concerning the modification of 2nd Embodiment is shown.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

<First Embodiment>
First, a first embodiment according to the present invention will be described.

[Configuration of information processing system]
FIG. 1 is a system configuration diagram of an information processing system 10 according to the first embodiment. As shown in FIG. 1, an information processing system 10 captures a landscape or the like, generates image data of the photographed photograph, and receives image data of the photograph photographed by the camera 3, and displays a display (display means). And a computer device 2 for displaying a photograph or the like.

  The camera 3 is a digital camera and generates image data of a photograph taken. Further, the camera 3 incorporates a triaxial type geomagnetic sensor (electronic compass) and the like, and detects a shooting direction. The computer device 2 is a laptop personal computer or the like. The camera 3 and the computer device 2 are used by an operator such as a teacher. The projector 4 is a device that receives display information from the computer device 2 and projects and displays the screen of the computer device 2 on the screen 6.

  Here, a usage form of the information processing system 10 shown in FIG. 1 will be briefly described. An operator photographs the scenery about various directions (for example, east, west, south, and north) using the camera 3 outdoors. The computer apparatus 2 receives the image data thus captured by the camera 3 and displays a photograph corresponding to the image data. Specifically, the computer apparatus 2 receives a shooting direction together with the image data from the camera 3, and arranges and displays the photos according to the shooting direction. In one example, such an information processing system 10 is used for observation of the sun and clouds in a weather class.

[camera]
Next, the camera 3 will be described. FIG. 2 shows a functional block diagram of the camera 3. The camera 3 includes an imaging unit 31, an imaging direction detection unit 32, a clock unit 33, a control unit 34, and a recording medium 35.

  The photographing means 31 is an image sensor (optical sensor) that generates image data by converting received light into an electrical signal, and is configured by a CCD or a CMOS, for example. The shooting direction detection means 32 is constituted by a triaxial type geomagnetic sensor (electronic compass) or the like, and is a shooting direction by the camera 3 (specifically, a direction in which the lens of the camera 3 is facing). Means the direction of the landscape.) The clock means 33 transmits time information indicating the current time (time stamp).

  The control unit 34 receives data from the imaging unit 31, the imaging direction detection unit 32, and the clock unit 33, and stores in the recording medium 35 data with the imaging direction and the imaging date / time added to the image data obtained by imaging. Let Specifically, the control unit 34 adds a numerical value (an angle or the like) indicating the direction detected by the shooting direction detection unit 32 when the image is taken by the shooting unit 31 to the image data as it is. Alternatively, the control unit 34 converts the numerical value indicating the direction detected by the shooting direction detection unit 32 when the shooting unit 31 has shot the image into data indicating one of the east, south, west, and north directions, or , East, southeast, south, southwest, west, northwest, north, and northeast converted to data indicating the direction are attached to the image data as shooting directions. On the other hand, the control unit 34 generates the shooting date and time (data indicating the date and time when the shooting was performed) based on the data transmitted from the clock unit 33 when shooting is performed by the shooting unit 31, and Add the shooting date and time to the image data. Each time shooting is performed, the control unit 34 adds a shooting direction and a shooting date and time to each piece of image data obtained by shooting as described above.

  The recording medium 35 stores image data with a shooting direction and a shooting date and time by the control unit 34. The recording medium 35 is configured to be removable from the camera 3. For example, the recording medium 35 is a memory card.

[Computer device]
Next, the computer apparatus 2 according to the first embodiment will be described. The computer apparatus 2 is configured as a hardware such as a processor such as a CPU, a memory such as a ROM or a RAM, a display, a personal computer including a mouse, a keyboard, or the like. FIG. 3 is a functional block diagram of the computer apparatus 2 according to the first embodiment. Functionally, the computer apparatus 2 includes an input unit 21, a processing unit 24, a storage unit 25, a display unit 26, a transmission unit 27, and an interface 28.

  The input means 21 includes a mouse and a keyboard for inputting various commands and data. When the display unit 26 is a touch panel system, the touch panel provided on the display screen of the display unit 26 also functions as the input unit 21. The display means 26 is constituted by a display or the like, and displays the contents instructed by the processing means 24. The transmission means 27 is a means for transmitting the same content as the content displayed on the display means 26 from the projector 4 to the screen 6 and transmitting the display signal to the display means 26 to the projector 4 in synchronization.

  The storage means 25 is configured by a memory such as a hard disk, ROM, or RAM. The storage unit 25 stores a program for controlling the processing unit 24 and stores various types of information generated by the execution of the program according to instructions from the processing unit 24.

  The interface 28 is attached with the recording medium 35 of the camera 3 described above, and reads data stored in the recording medium 35. Specifically, the interface 28 reads out the image data with the shooting direction and the shooting date and time from the recording medium 35, and outputs the image data to the processing unit 24. For example, the interface 28 is a memory card reader. The interface 28 is an example of the “image data input unit” in the present invention.

  The processing means 24 is constituted by a processor such as a CPU and controls the entire computer apparatus 2. Specifically, the processing unit 24 includes a display control unit 241. Based on the image data stored in the recording medium 35 of the camera 3 read by the interface 28, the display control unit 241 matches the photograph corresponding to the image data with the shooting direction attached to the image data. Arrange and display. That is, the display control unit 241 displays a plurality of photos taken by the camera 3 in association with the direction in which the photos were taken.

  Specifically, first, the display control unit 241 recognizes the shooting direction attached to each image data input from the interface 28, and determines an arrangement for displaying a photograph corresponding to each image data. In this case, the display control unit 241 arranges the directions defined on the display screen (for example, “east → south → west → north” or “east → south → south → southwest → west → northwest → north → northeast”). The arrangement for displaying the photos corresponding to each image data is determined on the basis of the shooting direction attached to each image data so that the photos are displayed side by side. Then, the display control means 241 displays the photos corresponding to the respective image data side by side in accordance with the arrangement thus determined. Further, the display control means 241 displays the shooting date and time attached to each image data in association with the direction along with the photo.

  When a numerical value (an angle or the like) indicating the direction detected by the shooting direction detection unit 32 of the camera 3 is input as it is as a shooting direction, the display control unit 241 displays such a value as east, south, west. Or the north direction, or the east, southeast, south, southwest, west, northwest, north, and northeast directions, and the photograph is displayed side by side using the converted direction. For example, when four image data are input, the display control unit 241 converts the direction to one of east, south, west, and north. When eight image data are input, the display control unit 241 Convert to one of the following directions: southeast, south, southwest, west, northwest, north and northeast. On the other hand, a numerical value indicating the direction detected by the shooting direction detection means 32 is converted into data indicating any one of east, south, west and north, or east, southeast, south, southwest, west, northwest, north When data converted into data indicating either the northeast or the northeast is input as a shooting direction, the display control unit 241 displays the photos side by side using the input direction as they are.

[Shooting example]
FIG. 4 shows an example of a situation where the operator is shooting a landscape in a different direction. FIG. 4 illustrates a situation where the operator is shooting a landscape in four directions, east, south, west, and north, using the camera 3. In one example, the operator uses a azimuth magnetic needle or the like, or refers to the direction in which the lens displayed on the display of the camera 3 faces, confirms the direction in which the operator is facing, and determines a predetermined direction ( Shoot landscapes in the East, West, South and North).

[Display screen example]
Next, an example of a screen displayed on the display unit 26 of the computer apparatus 2 will be described with reference to FIGS.

  FIG. 5 shows a screen on which a pull-down menu of a tool menu 90 used when displaying a photograph is shown. Note that, for example, the display of a photograph is executed by an application such as “education tool”.

  As shown in FIG. 5, as a pull-down menu of the tool menu 90, a direction array menu 90a for displaying photographs arranged in accordance with a shooting direction and a menu 90b for displaying application version, property information, and the like are prepared. Has been. When these menus are selected, each function is executed by the processing means 24.

  FIG. 6 shows an example of a photo display screen according to the first embodiment. This screen is displayed when the direction menu 90a of the tool menu 90 is selected. Further, this screen is displayed on the display means 26 by the display control means 241. In FIG. 6, for convenience of explanation, the portion of the photograph is hatched (the same applies hereinafter).

  As shown in FIG. 6, the display control unit 241 displays photographs 61 a, 61 b, 61 c (61) photographed by the camera 3 side by side in the left-right direction according to the photographing direction of each photograph 61. In this case, the display control unit 241 displays each photograph 61 taken by the camera 3 in the left-right direction in accordance with the order of the direction such as “east → south → west → north”. Specifically, first, the display control means 241 recognizes the shooting direction attached to each image data input from the interface 28, and determines the arrangement for displaying the photograph 61 corresponding to each image data. Then, the display control unit 241 displays the photographs 61 corresponding to each image data side by side in accordance with the arrangement determined based on the shooting direction.

  Further, as shown in FIG. 6, the display control unit 241 displays the shooting dates and times 62 a, 62 b, and 62 c (62) attached to each image data input from the interface 28 in association with the direction along with the photograph 61. . In addition, the display control means 241 displays information input to the computer apparatus 2 for recording by the operator (teacher) as indicated by reference numeral 63. For example, the operator inputs information related to the photographed landscape (information such as the ratio of clouds, the type of clouds, and the weather).

  Further, the display control unit 241 enables the scroll bar 64 to display a part of the plurality of photographs 61 and the plurality of photographing dates / times 62 photographed by the camera 3. In the example shown in FIG. 6, the entire photograph 61 b is displayed for the “South” direction, but only a part of the photographs 61 a and 61 c is displayed for the “East” direction and the “West” direction. No photo is displayed for the direction of “north” (the shooting date and time are not displayed). The operator can appropriately switch the portion to be displayed in the plurality of photographs 61 and the like by scrolling the screen by operating the scroll bar 64 with a mouse, a touch panel, etc. (However, even if the screen is scrolled, The information indicated by reference numeral 63 is not scrolled).

  In addition, although the example which displays four photographs 61 about four directions of east, south, west, and north was shown above, eight directions, east, southeast, south, southwest, west, northwest, north, and northeast, were shown. When the photograph 61 is taken, the display control means 241 displays the eight photographs 61 side by side in association with the photographing directions of those photographs. For example, the display control means 241 displays eight photographs 61 side by side in the order of “east → southeast → south → southwest → west → northwest → north → northeast”. However, if there is a direction in which the photograph 61 is not taken in the east, south, west and north directions, or the photograph 61 is in the east, southeast, south, southwest, west, northwest, north and northeast directions. If there is a direction that has not been photographed, the photograph 61 for that direction is not displayed (for example, the direction is displayed in a blank).

  In the above example, the photograph 61 and the shooting date / time 62 are displayed side by side in the horizontal direction. However, the photo 61 and the shooting date / time 62 may be displayed side by side in the vertical direction. In this case, a scroll bar for a vertical stroke is displayed instead of the scroll bar 64 for a horizontal stroke. Further, in the above description, the scroll bar 64 for scrolling the screen is displayed. However, when the display unit 26 is configured as a touch panel, the scroll bar 64 may not be displayed. In this case, it is only necessary to scroll the screen when an operation of tracing the screen with a finger is performed, and the display control means 241 can perform “east” → “south” → “west” even when scrolling in one direction. The same sketch may be repeatedly displayed as “→” north ”→“ east ”→.

  In the above description, an example in which a plurality of photographed photographs 61 and a part of the photographing date and time 62 are displayed on the screen has been shown. However, the entire plurality of photographs 61 and photographing date and time 62 may be displayed on the screen. In this case, it is only necessary to display an image obtained by reducing the image of the photograph 61 and the shooting date and time 62 when a part is displayed. In this case, the scroll bar 64 may not be displayed.

[Processing flow]
Next, the processing flow according to the first embodiment will be described with reference to FIG. FIG. 7 shows a processing flow for displaying a photograph. This processing flow is repeatedly executed by the computer apparatus 2 at a predetermined cycle.

  First, the interface 28 reads out the image data with the shooting direction and the shooting date and time from the recording medium 35, and the processing unit 24 stores the image data in the storage unit 25 (step S11). Next, the display control means 241 of the processing means 24 recognizes the shooting direction attached to each image data (step S12). When a numerical value (an angle or the like) indicating the direction detected by the shooting direction detection unit 32 of the camera 3 is input as the shooting direction as it is, the display control unit 241 uses such a value as the east, south, west, and north. Or in any direction of east, southeast, south, southwest, west, northwest, north, and northeast. On the other hand, a numerical value indicating the direction detected by the shooting direction detection means 32 is converted into data indicating any one of east, south, west and north, or east, southeast, south, southwest, west, northwest, north When the data converted into data indicating either the northeast or the northeast is input as the shooting direction, the display control unit 241 uses the input direction as it is.

  Thereafter, the display control means 241 determines the arrangement of the photos based on the recognized shooting direction (step S13). Specifically, the display control unit 241 arranges the directions defined on the display screen (for example, “east → south → west → north” or “east → south → south → southwest → west → northwest → north → Based on the shooting direction attached to each image data, the arrangement of the photos corresponding to each image data is determined so that the photos are displayed side by side in the order of “Northeast”. Then, the display control means 241 displays the photos corresponding to each image data side by side according to the arrangement determined in step S13 (step S14).

[Effects of First Embodiment]
According to 1st Embodiment described above, the photograph image | photographed with the camera 3 can be appropriately arranged and displayed according to the direction where the photograph was image | photographed. Such a display according to the first embodiment is convenient when used in a weather class or the like. For example, it can be used effectively when learning the position of the sun according to the time, how to evaluate the weather according to the shape of the clouds and the amount of clouds.

[Modification of First Embodiment]
In the above, an example is shown in which photographs are displayed side by side in the left-right direction (see FIG. 6), but this is not a limitation. In another example, photographs can be displayed side by side in a circle.

  FIG. 8 shows an example of a photo display screen according to a modification of the first embodiment. This screen is displayed on the display means 26 by the display control means 241 when the direction array menu 90a (see FIG. 5) of the tool menu 90 is selected. This screen is displayed instead of the screen shown in FIG. In FIG. 8, for convenience of explanation, the portion of the photograph is shown by hatching.

  As shown in FIG. 8, the display control means 241 displays the photographs 71 a, 71 b, 71 c, 71 d (71) photographed by the camera 3 in an annular manner in accordance with the photographing direction of each photograph 71. Specifically, the display control unit 241 displays the photographs 71 in a circular pattern in accordance with the direction of the direction “south → west → north → east” (clockwise order). In this case, the display control means 241 displays the photograph 71a for the “south” direction on the top, displays the photograph 71b for the “west” direction on the right, and displays the photograph 71c for the “north” direction on the bottom. And the photograph 71d for the direction of “east” is displayed on the left. Further, as shown in FIG. 8, the display control unit 241 displays the shooting dates and times 72 a, 72 b, 72 c, 72 d (72) in association with the directions along with the photos 71. Specifically, the display control unit 241 also displays the shooting date and time 72 in an annular manner as in the case of the photograph 71.

  Further, the display control unit 241 displays the entire photograph 71 and the shooting date and time 72 arranged in a ring shape as indicated by an arrow R in accordance with an operation of the touch panel by the operator (an operation of tracing the screen with a finger) or the like. Rotate. That is, the display control unit 241 rotates the entire photograph 71 and the shooting date / time 72 clockwise or counterclockwise according to the operation by the operator. Thereby, the direction arrange | positioned up and down and right and left interchanges, and the photograph 71 and the imaging | photography date / time 72 displayed on the top and bottom, left and right will interchange.

  According to the display screen of the photograph 71 according to the modified example described above, the photograph 71 can be displayed in a manner that is more compliant with the actual order of orientation. This makes it possible to easily grasp the position of the sun and clouds in a weather class, for example.

  In addition, although the example which displays four photographs 71 about four directions of east, south, west, and north was shown in Drawing 8, eight east, southeast, south, southwest, west, northwest, north, and northeast are shown. When the photos 71 are taken in the direction, the display control unit 241 displays the eight photos 71 in a ring-like manner in association with the directions in which the photos 71 were taken. However, if there is a direction in which the photograph 71 is not taken in the east, south, west and north directions, or the photograph 71 is in the east, southeast, south, southwest, west, northwest, north and northeast directions. If there is a direction that has not been photographed, the photograph 71 for that direction is not displayed (for example, the direction is displayed in a blank). Further, in the example of FIG. 8, the display control means 241 is arranged so that the upper side (the sky) of the photograph is positioned outside the ring, but is arranged so that the upper side of the photograph is positioned inside the ring. Also good.

Second Embodiment
Next, a second embodiment according to the present invention will be described. The second embodiment is different from the first embodiment in that not only the photograph taken by the camera 3 as described above but also a sketch drawn by the electronic pen is displayed side by side. Specifically, in the second embodiment, photographs taken by the camera 3 are arranged and displayed in accordance with the photographing direction, and the sketch drawn by the electronic pen is matched to the direction in which the sketch is drawn. Arrange and display.

  In the following, the configuration different from the first embodiment will be mainly described, and the description of the same configuration as the first embodiment will be omitted as appropriate. That is, functions and components that are not particularly described are the same as those in the first embodiment. Moreover, the component which attached | subjected the code | symbol same as the component shown in 1st Embodiment has the same meaning, and abbreviate | omits the description suitably.

[Configuration of information processing system]
FIG. 9 is a system configuration diagram of an information processing system 10a according to the second embodiment. As shown in FIG. 9, the information processing system 10a according to the second embodiment is the same as the first embodiment in that it includes electronic pens 1A, 1B,... (1) and sketch papers 5A, 5B,. The configuration is different from the information processing system 10. The information processing system 10a according to the second embodiment is different from the information processing system 10 according to the first embodiment in that the information processing system 10a according to the second embodiment includes the computer apparatus 2a instead of the computer apparatus 2.

  The electronic pen 1 is used by a writer such as a student to write a sketch or the like on the sketch paper 5, and transmits the entered information to the outside by a wireless communication method such as Bluetooth (registered trademark). For example, an anotopen can be used as the electronic pen 1. The sketch paper 5 is distributed to each of a plurality of writers and printed with a dot pattern (coded pattern) that can be read by the electronic pen 1. The computer device 2a receives the entry information transmitted from each electronic pen 1 and receives image data of a photograph taken by the camera 3 and displays a sketch or a photograph on a display (display means). The computer device 2a is used by an operator such as a teacher.

  Here, a usage form of the information processing system 10a shown in FIG. 9 will be briefly described. Each entrant draws a sketch of a landscape on the sketch paper 5 with the electronic pen 1 outdoors. For example, each entrant draws sketches for different directions (eg, east, west, south, and north). On the other hand, the operator uses the camera 3 outdoors to photograph landscapes in various directions (for example, east, west, south, and north). Specifically, the operator shoots the landscape drawn by each writer, that is, the same direction as the direction sketched by each writer. The computer device 2a receives the entry information from the electronic pen 1 of each writer and receives image data from the camera 3, and both the sketch drawn by the electronic pen 1 and the photograph taken by the camera 3. Are arranged and displayed according to the direction. In one example, such an information processing system 10a is used for observation of the sun and clouds in a weather class.

[Sketch paper]
The sketch paper 5 according to the second embodiment will be described with reference to FIG. As shown in FIG. 10, on the sketch sheet 5, a sketch entry field 5a for entering a sketch, a direction selection field 5b for selecting a direction in which the sketch is entered, and a date and time when the sketch is entered are entered. The date entry column 5c is provided. A plurality of directions (north, northeast, east, southeast, south, southwest, west, northwest) are printed in the direction selection column 5b, and a direction in which a sketch is entered can be selected from the plurality of directions. ing. Specifically, the direction selection column 5b can select a direction in which a sketch is written by enclosing one of a plurality of directions with a circle. The date and time entry column 5c can be filled with the date and time (hereinafter referred to as “sketch entry date and time”) when the sketch is entered.

  In the sketch entry field 5a, the direction selection field 5b, and the date / time entry field 5c, unique dot patterns (coded patterns) having different coordinate ranges are printed. The dot pattern is printed with ink containing carbon that absorbs infrared rays. On the other hand, on the sketch paper 5, characters (characters such as sketches, characters indicating directions, characters such as dates), frame lines, figures, and the like are printed with ink that does not absorb in the infrared region.

  A plurality of sketch papers 5 are prepared for distribution to each entry person or group. However, a unique dot pattern may be printed on the plurality of sketch papers 5 or transmitted by the electronic pen 1. Since the entry person can be identified by the pen ID to be used, the same dot pattern may be printed on the same area of each sketch paper 5.

[Dot pattern]
Next, an Anoto dot pattern (coded pattern) printed on the sketch paper 5 will be described with reference to FIGS. 11 and 12. FIG. 11 is a diagram for explaining the relationship between the dots of the dot pattern printed on the sketch paper 5 and the values to which the dots are converted. As shown in FIG. 11, each dot of the dot pattern is associated with a predetermined value depending on its position. In other words, each dot is associated with a value of 0 to 3 depending on which direction the top, bottom, left, or right is shifted from the reference position of the grid (intersection of the vertical line and horizontal line). The value of each dot can be further converted into a first bit value for the X coordinate and a second bit value for the Y coordinate. The position coordinates on the sketch paper 5 are determined by the combination of the information thus associated.

  FIG. 12A shows an arrangement of a certain dot pattern. As shown in FIG. 12A, 6 × 6 dots are arranged in a range of about 2 mm in length and width so that a unique pattern is obtained no matter what part of sketch paper 5 is taken 6 × 6 dots. Has been. The dot pattern formed by these 36 dots holds position coordinates (for example, where the dot pattern is on the sketch paper 5). FIG. 12B is a diagram in which each dot shown in FIG. 12A is converted into a corresponding value based on the regularity shown in FIG. 11 according to the shift direction from the reference position of the lattice. This conversion is performed by the electronic pen 1 that captures an image of a dot pattern.

[Electronic pen]
Next, the electronic pen 1 will be described with reference to FIG. FIG. 13 is a schematic view showing the structure of the electronic pen 1. As shown in FIG. 13, the electronic pen 1 includes an ink cartridge 104, an LED 105, a CMOS camera 106, a pressure sensor 107, a processor 108 including a CPU, a memory 109 such as a ROM and a RAM, A communication unit 111 including a real-time clock 110, an antenna, and the like, and a battery 112 are provided. The tip of the ink cartridge 104 is a pen tip portion 103, and each entry person makes a stroke (handwritten stroke) on the sketch paper 5 by bringing the pen tip portion 103 of the electronic pen 1 into contact with the sketch paper 5. Alternatively, the pen tip portion 103 of the electronic pen 1 is brought into contact with the sketch paper 5 and tapped (tap on the sketch paper 5 by the pen tip portion 103). The ink of the ink cartridge 104 does not include a material that absorbs infrared rays (for example, carbon). Here, contact of the pen tip portion 103 of the electronic pen 1 with the sketch paper 5 is referred to as pen-down, and removal of the pen tip portion 103 from the contacted (contacted) state is referred to as pen-up. A trajectory written between the pen-down and pen-up of the electronic pen 1 is one stroke, and characters, figures, etc. are composed of one or a plurality of strokes. Since the ink of the ink cartridge 104 does not include a material that absorbs infrared rays, the reading of the dot pattern by the processor 103 is not hindered.

  The battery 112 is for supplying electric power to each component in the electronic pen 1. For example, the battery 112 is configured to turn on / off the electronic pen 1 itself by attaching and detaching a cap (not shown) of the electronic pen 1. May be. The real time clock 110 transmits time information indicating the current time (time stamp) and supplies the time information to the processor 108. The pressure sensor 107 detects a pressure applied through the ink cartridge 104 from the pen tip portion 103 when the writer enters a handwritten stroke on the sketch paper 5 or taps the sketch paper 5 with the electronic pen 1, that is, a writing pressure. Is transmitted to the processor 108.

  The processor 108 switches on / off the switches of the LED 105 and the CMOS camera 106 based on the writing pressure data supplied from the pressure sensor 107. That is, when the writer enters or taps a handwritten stroke on the sketch paper 5 with the electronic pen 1, the pen pressure is applied to the pen tip 103, and when the pressure sensor 107 detects a writing pressure higher than a predetermined value. The processor 108 determines that the writer has started filling in and activates the LED 105 and the CMOS camera 106. Then, the communication unit 111 associates pen-down information PD detected by the pressure sensor 107 with a pen ID described later, and transmits it as entry information to the computer apparatus 2a. Further, when the writer finishes entering one stroke and moves the electronic pen 1 away from the sketch paper 5, the pressure sensor 107 detects pen-up because no writing pressure exceeding a predetermined value is detected. Then, the communication unit 111 associates the pen-up information detected by the pressure sensor 107 with the pen ID, and transmits it as entry information to the computer apparatus 2a.

  The LED 105 and the CMOS camera 106 are attached near the pen tip portion 103 of the electronic pen 1, and an opening 102 is formed in a portion of the housing 101 that faces the LED 105 and the CMOS camera 106. The LED 105 illuminates infrared rays toward the vicinity of the pen tip portion 103 on the sketch paper 5. The region is slightly shifted from the position where the pen tip 103 contacts the sketch paper 5. The CMOS camera 106 is provided with an infrared transmission filter that transmits infrared rays and blocks non-infrared rays, captures a dot pattern in the area illuminated by the LED 105, and supplies image data of the dot pattern to the processor 108. Here, since carbon absorbs infrared rays, the infrared rays irradiated by the LED 105 are absorbed by the carbon contained in the dots. Therefore, the dot portion has a small amount of infrared reflection, and the portion other than the dot has a large amount of infrared reflection. By photographing with the CMOS camera 106, a threshold value is provided based on the difference in the amount of reflected infrared light, so that a dot region containing carbon can be distinguished from other regions. Therefore, even when characters, frame lines, etc. are printed on the sketch paper 5, the processor 108 can recognize the dot pattern because the printed ink does not absorb in the infrared region. Note that the imaging area by the CMOS camera 106 is a range including a size of about 2 mm × about 2 mm as shown in FIG. 12A, and the imaging by the CMOS camera 106 is performed at regular intervals of about 50 to 100 times per second. Is called. Further, the CMOS camera 106 has a sufficient depth of field in order to photograph dots clearly.

  The processor 108 uses the dot pattern of the image data supplied by the CMOS camera 106 to perform X and Y coordinates (hereinafter, simply “ "Position coordinates" or "coordinate information") is continuously calculated. That is, the processor 108 converts the image data of the dot pattern as shown in FIG. 12 (a) supplied by the CMOS camera 106 into the data array shown in FIG. 12 (b). The data is converted into a Y coordinate bit value, and X and Y coordinate data are calculated from the data array by a predetermined calculation method. Note that the processor 108 has a rotation correction processing function for correcting the arrangement of dots on the image due to the angle of the electronic pen 1 facing the dot pattern, and this function is exhibited during the coordinate calculation. Then, the processor 108 associates the current time (time stamp: entered time information) transmitted from the real-time clock 110, writing pressure data, and X and Y coordinate data. Hereinafter, these associated data are collectively referred to as “coordinate attribute information”. Since the 6 × 6 dot pattern on the sketch paper 5 does not overlap in the sketch paper 5, when the writer enters characters or the like with the electronic pen 1, the written position is any position on the sketch paper 5. Can be specified by coordinate calculation by the processor 108.

  The memory 109 stores property information such as pen IDs “pen01” to “pen05”, pen manufacturer numbers, and pen software versions for identifying the electronic pens 1A to 1E. Then, the communication unit 111 associates the pen ID, time information (time stamp), writing pressure data, and X and Y coordinate data, and transmits them to the computer apparatus 2a as entry information. Transmission to the computer apparatus 2a by the communication unit 111 is immediately and sequentially performed by wireless transmission such as Bluetooth (registered trademark). Here, one or a plurality of coordinate attribute information generated between the pen-down and pen-up of the electronic pen 1 and transmitted to the computer device 2a is stored as stroke information by the computer device 2a. In other words, one stroke is composed of one or a plurality of X and Y coordinates (coordinate points), and the computer apparatus 2a uses one or more pen-down information and pen-up information to constitute one or a plurality of strokes. Recognize coordinate attribute information.

  In the situation where the electronic pen 1 cannot communicate with the computer device 2a (for example, the situation where radio waves do not reach), the entry information as described above is stored in the memory 108. Then, the electronic pen 1 transmits the entry information stored in the memory 108 in this way to the computer apparatus 2a collectively when it can communicate with the computer apparatus 2a. In one example, when the writer is sketching outdoors, the electronic pen 1 stores the entry information in the memory 108, and the writer can return to the indoor classroom and communicate with the computer apparatus 2a. When the status is reached, the entry information is sent together.

[Computer device]
Next, the computer apparatus 2a according to the second embodiment will be described. The computer device 2a includes, as hardware, an antenna device capable of data communication with the electronic pen 1, a processor such as a CPU, a memory such as ROM and RAM, a display, a personal computer including a mouse and a keyboard, and the like. .

  FIG. 14 is a functional block diagram of the computer apparatus 2a according to the second embodiment. The computer apparatus 2a according to the second embodiment is different from the computer apparatus 2 according to the first embodiment in that the computer apparatus 2a includes a receiving unit 22. The computer device 2a according to the second embodiment includes the processing unit 24a instead of the processing unit 24, and also includes the storage unit 25a instead of the storage unit 25, and the computer device 2 according to the first embodiment. Different.

  The receiving means 22 is constituted by an antenna receiving circuit or the like, receives entry information from the electronic pen 1 and transmits it to the processing means 24a.

  The storage means 25a is configured by a memory such as a hard disk, ROM, or RAM. The storage unit 25 a stores coordinate definition information related to the coordinate range of the dot pattern printed on the sketch paper 5. Specifically, the storage means 25a stores the coordinate ranges of different dot patterns assigned to the sketch entry field 5a, the direction selection field 5b, and the date / time entry field 5c of the sketch paper 5, respectively. That is, the storage means 25a stores coordinate definition information in which the sketch entry field 5a, the direction selection field 5b, the date and time entry field 5c are associated with the coordinate range of the dot pattern assigned to each of them. The coordinate range of the dot pattern is defined by the rectangular corner position coordinates (Xn, Yn), height (H), and width (W).

  The storage unit 25a stores the entry information received from the electronic pen 1 for each pen ID according to the processing instruction of the processing unit 24a. Further, the storage unit 25a secures a storage area according to the instruction of the processing unit 24a. The coordinate attribute information included in the entry information until the up is stored as stroke information for each pen ID. In addition to the above, the storage unit 25a stores various information generated by the execution of the program in accordance with instructions from the processing unit 24a.

  The processing means 24a is constituted by a processor such as a CPU and controls the entire computer apparatus 2. Specifically, the processing unit 24 a includes a display control unit 242. The display control means 242 arranges and displays both the sketch entered by the electronic pen 1 and the photograph taken by the camera 3 in accordance with the direction (the direction in which the sketch is entered and the photographing direction). Specifically, the display control unit 242 performs a process for displaying a sketch and a process for displaying a photograph taken by the camera 3 in parallel, and arranges the sketch and the photograph according to the direction. To display. Since the process for displaying the photograph taken by the camera 3 is the same as that in the first embodiment (that is, the process performed by the display control unit 241), the description thereof is omitted, and a sketch is displayed below. The process for making it perform is demonstrated.

  The display control means 242 is based on the filling information received by the receiving means 22 and corresponding to the writing on the sketch paper 5 by the electronic pen 1, and the handwriting stroke corresponding to the sketch written on the sketch paper 5 (in the following, The handwritten stroke corresponding to the sketch is also simply called “sketch”.) Are arranged and displayed in accordance with the direction in which the sketch is written. In other words, the display control means 242 displays the sketches (landscapes) entered by a plurality of writers in association with the direction in which those sketches are entered (the direction of the landscape drawn as a sketch).

  More specifically, the display control means 242 first recognizes the direction selected by each writer based on the entry information corresponding to the entry in the direction selection field 5b of each sketch paper 5 by each electronic pen 1. Then, the arrangement for displaying the sketches of each writer is determined. In this case, the display control unit 242 arranges the directions defined on the display screen (for example, “east → south → west → north” or “east → south → south → southwest → west → northwest → north → northeast”). The layout for displaying the sketches of each writer is determined based on the direction selected by each writer so that the sketches are displayed side by side. Then, the display control means 242 performs handwriting strokes corresponding to the sketches entered by each writer based on the entry information corresponding to the entry in the sketch entry field 5a of each sketch paper 5 by each electronic pen 1 in this way. Display them side by side according to the determined sequence. The display control means 242 also calculates the date and time (sketch entry date and time) entered by each writer based on the entry information corresponding to the entry in the date and time entry field 5 c of each sketch paper 5 by each electronic pen 1. In addition, it is displayed in association with the direction.

  The display control unit 242 performs processing for displaying such a sketch in parallel with processing for displaying a photo similar to that in the first embodiment, so that both the sketch and the photo are in the direction (sketch Are arranged and displayed in accordance with the direction in which is filled in and the shooting direction).

[Sketch example]
Next, a sketch example using the sketch paper 5 according to the second embodiment will be described with reference to FIGS. 15 and 16.

  FIG. 15 shows an example of how each entry person sketches (draws) a landscape in a different direction. FIG. 15 shows an example in which four entrants are sketching landscapes in four directions, east, south, west, and north, respectively. In one example, each writer confirms the direction that he / she is facing using a compass, and the direction indicated by the teacher (either east, west, south or north). And sketch.

  FIG. 16 shows a sketch example entered on the sketch paper 5 by the electronic pen 1. As shown in FIG. 16, a sketch of a landscape is drawn in the sketch entry field 5a, a direction in which the sketch is drawn is selected in the direction selection field 5b, and a sketch is drawn in the date / time entry field 5c. The date and time are entered. Here, a case where the writer selects “south” as the direction in which the sketch is drawn by enclosing the “south” in the direction selection field 5b with a circle with the electronic pen 1 is illustrated. For example, each writer confirms the direction in which he / she is facing using a magnetic compass or the like, and selects the direction from the direction selection field 5b using the electronic pen 1.

[Display screen example]
Next, an example of a screen displayed on the display means 26 of the computer device 2a will be described with reference to FIGS.

  FIG. 17 shows a screen on which a pull-down menu of the tool menu 100 used when displaying sketches and photographs is shown. The display of sketches and photographs is executed by an application such as “education tool”.

  As shown in FIG. 17, as a pull-down menu of the tool menu 100, a direction array menu 100a for displaying sketches and photographs according to the direction and a pen management menu 100b for displaying the connection status of the electronic pen 1 are displayed. And a paper management menu 100c for setting an image of sketch paper 5 as a background for displaying stroke information, and a menu 100d for displaying application version, property information, and the like. When these menus are selected, each function is executed by the processing means 24a.

  FIG. 18 shows an example of a sketch and photo display screen according to the second embodiment. This screen is displayed when the direction menu 100a of the tool menu 100 is selected. Further, this screen is displayed on the display means 26 by the display control means 242. In addition, the same code | symbol is attached | subjected about the same thing as the image (refer FIG. 6) shown in 1st Embodiment, and the description is abbreviate | omitted suitably. In FIG. 18, for convenience of explanation, a portion of a photograph is represented by hatching.

  As shown in FIG. 18, the display control means 242 includes sketches 81 a, 81 b, 81 c (81) written by the respective writers with the electronic pen 1 along with the respective photographs 61 a, 61 b, 61 c (61) taken by the camera 3. ) Are distinguished by the pen ID included in the entry information, and the directions are aligned and displayed in the left-right direction. Specifically, the display control unit 242 displays the photograph 61 side by side according to the shooting direction, and displays the sketch 81 side by side according to the direction in which the sketch 81 is written below the photo 61. In this case, the display control unit 242 displays the photograph 61 and the sketch 81 side by side in the left-right direction in accordance with the order of the direction such as “east → south → west → north”. As a result, the photograph 61 and the sketch 81 corresponding to the scenery in the same direction are arranged side by side. That is, a sketch 81 in which a landscape in the direction in which the photograph 61 was taken is drawn is arranged under the photo 61 (in other words, a landscape in a direction in which the sketch 81 is drawn on the sketch 81). A photograph 61 in which is taken is placed).

  As shown in FIG. 18, the display control means 242 displays the sketch entry date and time 82 a, 82 b, 82 c (82) entered by each entry person with the electronic pen 1 in association with the direction along with the sketch 81. In this case, the display control means 242 displays the characters (numerals) obtained by performing the character recognition process on the handwritten stroke entered in the date / time entry field 5 c with the electronic pen 1 as the sketch entry date / time 82.

  Further, the display control means 242 enables the scroll bar 64 to display a plurality of photos 61 and a part of the shooting date and time 62, and to display a sketch 81 and a part of the sketch entry date and time 82 entered by each writer. ing. When the sketch 81 is described, in the example shown in FIG. 18, the entire sketch 81b is displayed in the direction of “south”, but part of the sketches 81a and 81c is displayed in the direction of “east” and the direction of “west”. Is displayed, and the sketch is not displayed for the direction of “north” (the sketch entry date and time are not displayed). The operator can appropriately switch the portion to be displayed in the plurality of photos 61 and the plurality of sketches 81 by scrolling the screen by operating the scroll bar 64 with a mouse, a touch panel, or the like (however, the screen is changed). Even when scrolling, the information indicated by reference numeral 63 does not scroll).

  In the above, an example in which four photographs 61 and four sketches 81 in four directions of east, south, west, and north are displayed, but east, southeast, south, southwest, west, northwest, north and When the photograph 61 is taken and the sketch 81 is entered in the eight directions in the northeast, the display control unit 242 displays the eight photographs 61 and the eight sketches 81 side by side in association with the directions. For example, the display control unit 242 displays the eight photographs 61 and the eight sketches 81 side by side in the order of “east → southeast → south → southwest → west → northwest → north → northeast”. However, in the east, south, west and north directions, there is a direction where the photograph 61 is not taken and / or a direction where the sketch 81 is not filled, or east, southeast, south, southwest, west, northwest. If there is a direction in which the photograph 61 is not taken and / or a direction in which the sketch 81 is not filled in the north and northeast directions, the photograph 61 and / or the sketch 81 in such a direction is displayed. (For example, the direction is displayed with a blank).

  In the above description, an example is shown in which the character obtained by performing the character recognition process on the handwritten stroke entered in the date / time entry field 5c with the electronic pen 1 is displayed as the sketch entry date / time 82. 1, the handwritten stroke entered in the date entry column 5c may be displayed as the sketch entry date 82 as it is. Further, in the above description, an example is shown in which both the photograph 61 and the sketch 81 are displayed side by side in the left-right direction, but both the photograph 61 and the sketch 81 may be displayed side by side in the vertical direction. In this case, the corresponding photograph 61 and sketch 81 are arranged side by side. In this case, a scroll bar for a vertical stroke is displayed instead of a scroll bar 64 for a horizontal stroke. Further, in the above description, the scroll bar 64 for scrolling the screen is displayed. However, when the display unit 26 is configured as a touch panel, the scroll bar 64 may not be displayed. In this case, it is only necessary to scroll the screen when an operation of tracing the screen with a finger is performed, and the display control means 241 can perform “east” → “south” → “west” even when scrolling in one direction. “→” North ”→“ East ”→... May be repeated to display the same photograph or sketch.

  In the above, an example in which a plurality of photographed photographs 61 and a part of a plurality of sketches 81 filled in by each writer is displayed on the screen is shown. However, the plurality of photographs 61 and the entire plurality of sketches 81 are displayed on the screen. May be displayed. In this case, an image obtained by reducing the image of the photograph 61 and the sketch 81 in the case of displaying a part may be displayed. In this case, the scroll bar 64 may not be displayed.

[Processing flow]
Next, a processing flow according to the second embodiment will be described with reference to FIG. FIG. 19 shows a processing flow for displaying both a photograph and a sketch. This processing flow is repeatedly executed at a predetermined cycle by the computer apparatus 2a.

  In the processing flow, processing for displaying a photograph taken by the camera 3 (steps S21 to S23) and processing for displaying a sketch filled in with the electronic pen 1 (steps S24 to S27) are performed in parallel. To be executed. Since the process of step S21-S23 is the same as the process of step S11-S13 in the process flow which concerns on 1st Embodiment, the description is abbreviate | omitted (refer FIG. 7). Here, the processing of steps S24 to S28 will be described.

  In step S24, the receiving unit 22 receives the entry information transmitted from the electronic pen 1, and the processing unit 24a stores the entry information in the storage unit 25a. Next, the display control means 242 of the processing means 24a recognizes the direction selected by each writer based on the entry information corresponding to the entry in the direction selection field 5b of each sketch paper 5 by each electronic pen 1. (Step S25). Specifically, the display control unit 242 refers to the coordinate definition information about the coordinate range of the dot pattern assigned to the direction selection field 5b stored in the storage unit 25a, so that each handwriting person can create a handwritten stroke. The position on the entered direction selection column 5b is specified, and the direction selected by each writer is recognized based on the position.

  Thereafter, the display control means 242 determines the arrangement for displaying the sketches of each writer based on the recognized direction (step S26). Specifically, the display control means 242 arranges the direction of the directions defined on the display screen (for example, “east → south → west → north” or “east → south → south → southwest → west → northwest → north → Based on the direction selected by each writer, the arrangement for displaying the sketches of each writer is determined so that the sketches are displayed side by side in the order of “northeast”.

  Next, the display control means 242 draws handwritten strokes corresponding to the sketches entered by each writer based on the entry information corresponding to the entry in the sketch entry field 5a of each sketch paper 5 by each electronic pen 1. (Step S27). In this case, the display control means 242 refers to the coordinate definition information about the coordinate range of the dot pattern assigned to the sketch entry field 5a stored in the storage means 25a, so that the handwritten stroke entered by each writer Draw.

  Next, the display control means 242 displays the photos corresponding to the respective image data side by side according to the arrangement determined in step S23, and displays the strokes drawn as described above side by side according to the arrangement determined in step S27. (Step S28). That is, the display control unit 242 arranges and displays both the photograph and the sketch according to the direction (the shooting direction and the direction in which the sketch is written).

[Effects of Second Embodiment]
According to the second embodiment described above, both the photograph taken by the camera 3 and the sketch written by the electronic pen 1 can be appropriately arranged and displayed according to the direction. Such a display according to the second embodiment can also be used effectively in learning a weather class or the like when learning the position of the sun according to the time, how to evaluate the weather according to the shape of the clouds and the amount of clouds, etc. it can. Further, according to the display according to the second embodiment, the sketch can be appropriately evaluated by comparing the sketch with the photograph.

[Modification of Second Embodiment]
In the above description, the example of the sketch paper 5 configured to select the direction in which the sketch is entered by the direction selection field 5b has been described. However, the use of such a sketch paper 5 is not limited. In another example, it is possible to use a sketch sheet configured to directly fill in the direction in which the sketch is filled with the electronic pen 1.

  FIG. 20 shows a sketch paper 51 according to a modification of the second embodiment. As shown in FIG. 20, on the sketch paper 51, a sketch entry field 51a for entering a sketch, a direction entry field 51b for entering the direction in which the sketch is entered, and the date and time when the sketch is entered are entered. The date entry column 51c is provided. In the direction entry field 51b, a unique dot pattern (coded pattern) having a different coordinate range from the sketch entry field 51a and the date / time entry field 51c is printed, and the direction in which the sketch is entered can be entered. The sketch entry field 51a and the date entry field 51c are the same as the sketch entry field 5a and the date entry field 5c described above.

  FIG. 21 shows a sketch example entered on the sketch paper 51 by the electronic pen 1. As shown in FIG. 21, a sketch of a landscape is drawn in the sketch entry field 51a, and the direction in which the sketch was drawn (in this example, “south”) is entered in the direction entry field 51b. In 51c, the date and time when the sketch was drawn (the date and time when the sketch was entered) is entered. Here, in the modification of the second embodiment, the display control means 242 refers to the coordinate range of the dot pattern assigned to the direction entry field 51b, and the handwritten stroke entered in the direction entry field 51b with the electronic pen 1. By performing the character recognition process, the direction entered by each writer is recognized. Then, the display control means 242 displays handwritten strokes corresponding to the sketches entered by each writer in an array in accordance with the direction thus recognized.

  Even when the sketch paper 51 as described above is used, the sketch written on the sketch paper 51 with the electronic pen 1 can be appropriately arranged and displayed according to the direction in which the sketch is written.

  The present invention is not limited to application to a general PC (personal computer), and can also be applied to a tablet PC such as “iPad (registered trademark)”.

  In the above description, the Anoto method is used for the electronic pen, the dot pattern (coded pattern), and the entry information, but the use of the Anoto method is not limited.

DESCRIPTION OF SYMBOLS 1 ... Electronic pen 2, 2a ... Computer apparatus 3 ... Camera 5 ... Sketch paper 10, 10a ... Information processing system 21 ... Input means 22 ... Reception means 24, 24a ... Processing means 25, 25a ... Storage means 26 ... Display means 28 ... interface

Claims (13)

Image data input means for inputting image data of a photograph taken by a camera;
Display control means for arranging and displaying the photograph corresponding to the image data input to the image data input means according to the direction in which the photograph was taken;
Receiving means for receiving entry information relating to a handwritten stroke written by an electronic pen on a sketch paper on which a coded pattern is formed; and
The display control means adjusts a sketch corresponding to the handwritten stroke entered on the sketch sheet based on the entry information received by the receiving means together with the photograph in accordance with the direction in which the sketch is entered. A computer apparatus characterized by being arranged and displayed . In the image data input means, the shooting direction of the photograph detected by a sensor built in the camera is input together with the image data,
The computer apparatus according to claim 1, wherein the display control unit displays the photograph in an arrangement according to the shooting direction input to the image data input unit. The image data input means, the shooting date and time when the photograph was taken by the camera is input together with the image data,
3. The computer apparatus according to claim 1, wherein the display control unit displays the shooting date and time input to the image data input unit, along with the photograph, arranged in accordance with the direction.   The computer apparatus according to any one of claims 1 to 3, wherein the display control unit displays a plurality of the photographs in association with the plurality of directions. The display control means includes
The whole or a part of the plurality of photos is displayed side by side in the horizontal direction or the vertical direction,
The computer apparatus according to claim 4, wherein when a part of the plurality of photographs is displayed, a part to be displayed in the plurality of photographs is scrolled according to a user operation.   5. The computer according to claim 4, wherein the display control unit displays the plurality of photographs arranged in a circle and rotates the whole of the plurality of photographs arranged in a circle according to a user operation. apparatus. The sketch sheet is provided with a sketch entry field for entering the sketch and a direction selection field for selecting a direction in which the sketch is entered, and the sketch entry field and the direction selection field are encoded differently. A pattern is formed,
The display control means has an arrangement according to the direction selected in the direction selection field based on the entry information corresponding to the entry with the electronic pen in the sketch entry field and the direction selection field, and the sketch. computer apparatus according to any one of claims 1 to 6, characterized in that to display the sketch entered in the entry field. The sketch sheet is provided with a sketch entry field for entering the sketch and a direction entry field for entering the direction in which the sketch is entered, and the coding is different depending on the sketch entry field and the direction entry field. A pattern is formed,
The display control means, based on the entry information corresponding to the entry with the electronic pen in the sketch entry field and the direction entry field, in an arrangement according to the direction entered in the direction entry field, The computer apparatus according to any one of claims 1 to 6, wherein a sketch written in a column is displayed. The sketch paper is provided with a date and time entry column for entering the date and time when the sketch was entered, and a coded pattern is formed in the date and time entry column,
The display control means displays the date and time entered in the date and time entry column according to the direction along with the sketch based on the entry information corresponding to the entry with the electronic pen in the date and time entry column. computer apparatus according to any one of claims 1 to 8, characterized in that to. The display control means includes
In correspondence with a plurality of the directions, the whole or a part of the plurality of photographs and the plurality of sketches are displayed side by side in the horizontal direction or the vertical direction,
When to display a portion of the plurality of photo and the plurality of sketches, claim 1 according to the operation of the user, characterized in that to scroll the portion to be displayed in the plurality of photo and the plurality of sketches The computer apparatus as described in any one of thru | or 9 . A computer device according to any one of claims 1 to 10 ,
An information processing system comprising a camera that generates image data of a photograph taken,
The information processing system according to claim 1, wherein the image data input means of the computer device receives the image data generated by the camera. A computer device according to any one of claims 1 to 10 ,
A camera that generates image data of the photograph taken;
Sketch paper on which a coded pattern is formed and handwritten strokes are filled in with an electronic pen;
An electronic pen that reads the coded pattern formed on the sketch paper and generates entry information related to the coded pattern,
The image data input means of the computer device receives the image data generated by the camera,
The information processing system according to claim 1, wherein the receiving unit of the computer apparatus receives entry information related to the coded pattern formed on the sketch paper, which is read by the electronic pen. A program for causing a computer apparatus according to any one of claims 1 to 10 to function.