JP2010262090A - Learning support program and learning support device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a learning support program and a learning support device, for learning natural pronunciation of an original text. <P>SOLUTION: The learning support device (10) includes a storage medium (48) and a computer (42). The storage medium stores original content information (781a, 781d) showing the content of the original text (ES) to be the object of language learning, and accent position information (781c) showing the position of accent with respect to the original on time base. The computer displays (S111), while notifying (S55, S105) the content of the original text on the basis of the original content information, accent images (G, AP1, AP2, ...) for notifying the position of the accent on time base with respect to the original text to a user on a screen (14) on the basis of the accent position information. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a learning support program and a learning support apparatus, and more particularly to a learning support program and a learning support apparatus that support language learning, for example.

  As this type of background art, for example, a language learning system disclosed in Patent Document 1 is known. The language learning system includes a teacher terminal and a plurality of learner terminals. The teacher terminal generates speech synthesis data from the original text, selects a rhythm sound suitable for the generated speech synthesis data, specifies the tempo of the selected rhythm sound, plays the rhythm sound at the specified tempo, and Distribute playback rhythm sound to each learner's terminal. Each learner's terminal receives and outputs the playback rhythm sound while displaying the characters of the original text.

JP 2008-164701 A

  By the way, in natural pronunciation of the original text such as English (pronunciation when talking or reading), generally speaking, if the string is separated by the position of the accent (strongly pronounced sound), the time length of each section (pronunciation range) Will not be uniform.

  In this regard, in the background art of Patent Document 1, the learner pronounces the original text in accordance with the reproduced rhythm sound, that is, the time length of each section is expanded or contracted so that the accent comes to the position of the reproduced rhythm sound. The time length is uniform. Such original pronunciation is suitable for singing, but feels unnatural in conversation and reading.

  Further, such a nonuniform time length of each section is difficult to distinguish from the characters of the original text.

  Therefore, a main object of the present invention is to provide a novel learning support program and learning support apparatus.

  Another object of the present invention is to provide a learning support program and a learning support apparatus that can acquire the natural pronunciation of the original text.

  The present invention employs the following configuration in order to solve the above problems. The reference numerals in parentheses, supplementary explanations, and the like indicate correspondence relationships with embodiments described later to help understanding of the present invention, and do not limit the present invention in any way.

  According to a first aspect of the present invention, there is provided a computer of a learning support apparatus having a storage medium storing original content information indicating the content of an original text to be language-learned and accent position information indicating the position of the original text on the time axis. The original text content notifying means for notifying the user of the content of the original text based on the original text content information, and when the original text content is notified by the original text content notifying means, the user can determine the position of the accent on the time axis with respect to the original text. This is a learning support program that causes an accent image for notification to be displayed on the screen based on the accent position information.

  In the first invention, the learning support program includes original text content information (781a, 781d) indicating the content of the original text (ES) to be language-learned and accent position information indicating the position of the original text on the time axis ( The computer (42) of the learning support device (10) having the storage medium (48) storing 781c) is caused to function as original text content notification means (S55, S105) and accent image display means (S111). The original text content notifying means notifies the user of the content of the original text based on the original text content information. Based on the accent position information, the accent image display means displays an accent image (G, AP1, AP2,...) Indicating the position of the accent on the time axis with respect to the original text when the content of the original text is reported by the text content notifying means. Display on screen (14).

  According to the first aspect of the invention, by visually indicating the position of the original accent on the time axis while notifying the content of the original, the user is divided by the timing at which the accent should be pronounced and the accent. Knowing the time length of each section, you can learn the natural pronunciation of the original text.

  A second invention is a learning support program subordinate to the first invention, wherein the computer is adapted to time the original text informed by the original text content notifying means at the position of the accent on the time axis based on the accent position information. It further functions as a time section image display means for displaying a time section image which is divided into sections and shows the position and length of the time section on the time axis on the screen.

  In the second invention, the learning support program causes the computer to further function as time interval image display means (S113). The time interval image display means divides the original text notified by the original text content notification means into time intervals at the position of the accent on the time axis based on the accent position information, and the position and length of the time interval on the time axis. Are displayed on the screen.

  According to the second aspect of the present invention, it is possible to intuitively recognize the relative length (tempo expansion / contraction) of each time section by further displaying the time section image.

  A third invention is a learning support program subordinate to the first invention, wherein the computer is used as highlighting means for highlighting an accent image to be pronounced at present among accent images displayed by the accent image display means. Make it work further.

  In the third invention, the learning support program further causes the computer to function as highlighting means (S43). The highlighting display unit highlights an accent image to be pronounced at the present time among the accent images displayed by the accent image display unit.

  According to the third aspect, the original text can be pronounced at an appropriate speed by highlighting the accent image as time progresses.

  A fourth aspect of the invention is a learning support program according to the third aspect of the invention, in which the computer is used to time the original sentence informed by the original sentence content notifying means at the position of the accent on the time axis based on the accent position information. The time interval image display means is further displayed as the time interval image display means for dividing the interval and displaying the time interval image indicating the position and length of the time interval on the time axis on the screen. The highlighting means is displayed by the time interval image display means. Of the time interval images, the time interval image corresponding to the current time is further highlighted.

  In the fourth invention, the learning support program causes the computer to further function as time interval image display means (S113). The time interval image display means divides the original text notified by the original text content notification means into time intervals at the position of the accent on the time axis based on the accent position information, and the position and length of the time interval on the time axis. Is displayed on the screen. The highlighting means further highlights the time interval image corresponding to the current time among the time interval images displayed by the time interval image display means.

  According to the fourth aspect of the present invention, the current time interval can be clearly recognized by further highlighting the time interval image.

  A fifth invention is a learning support program according to any one of the first to fourth inventions, wherein the computer displays a current position image indicating a current position on the time axis on a screen. It further functions as a display means.

  In the fifth invention, the learning support program causes the computer to further function as current position image display means (S15). The current position image display means displays a current position image (P) indicating the current position on the time axis on the screen.

  According to the fifth aspect, since the user can know the positional relationship between the accent and the current time, the speed of pronunciation (tempo) can be adjusted.

  A sixth invention is a learning support program according to any one of the first to fifth inventions, wherein the computer reproduces an accent sound corresponding to an accent at time intervals based on accent position information. As a further function.

  In the sixth invention, the learning support program causes the computer to further function as accent sound reproduction means (S39, 34). The accent sound reproducing means reproduces the accent sound corresponding to the accent at time intervals based on the accent position information.

  According to the sixth aspect of the invention, the user can also audibly recognize the time length of each section through the accent sound, so that natural pronunciation can be easily performed.

  The seventh invention is a learning support program according to any one of the first to sixth inventions, wherein the original text content information includes original text information (781a) indicating the original text, and the original text content notifying means (S105). ) Displays the original text on the screen based on the original text information.

  According to the seventh aspect, by displaying the original text, the contents of the original text can be visually notified and the correct spelling can be recognized.

  An eighth invention is a learning support program according to the seventh invention, wherein the computer displays the original text displayed by the original text content notifying means at a time interval at a position on the time axis of the accent based on the accent position information. The character string highlighting means for highlighting the character string belonging to the time section corresponding to the current time among the character strings of the original text displayed by the text content notification means.

  In the eighth invention, the learning support program causes the computer to further function as character string highlighting means (S43). The character string highlighting means divides the original text displayed by the original text content notifying means into time intervals at the position on the time axis of the accent based on the accent position information, and the character string of the original text displayed by the original text content notifying means is displayed. Among them, the character string belonging to the time interval corresponding to the current time is highlighted.

  According to the eighth aspect, by further highlighting the character string in units of time intervals, the original sentence can be pronounced in appropriate character string units.

  A ninth invention is a learning support program according to any one of the first to eighth inventions, wherein the original text content information further includes original text audio information (781d) indicating the original voice, and the original text content notifying means ( In step S55, the original sound is reproduced based on the original sound information.

  According to the ninth aspect, by reproducing the voice of the original text, that is, the model, the contents of the original text can be audibly notified and the correct pronunciation can be recognized.

  A tenth invention is a learning support program according to the ninth invention, further causing the computer to function as a reproduction speed setting means for setting a reproduction speed of the original sound, wherein the original content notifying means is a reproduction speed setting means. The original sound is reproduced at the reproduction speed set by the means.

  In the tenth invention, the learning support program causes the computer to further function as reproduction speed setting means (S1). The reproduction speed setting means sets the reproduction speed of the original sound, and the original content notification means reproduces the original sound at the reproduction speed set by the reproduction speed setting means.

  According to the tenth aspect, the learning effect can be enhanced by setting the reproduction speed according to the proficiency level.

  An eleventh invention is a learning support program according to the ninth or tenth invention, wherein the original text content notifying means repeatedly reproduces the original voice.

  According to the eleventh aspect, it is possible to acquire the correct pronunciation by repeatedly reproducing the model and causing the user to repeat the pronunciation.

  The twelfth invention is a learning support program subordinate to the eleventh invention, and the original text content notifying means reproduces the volume so that the volume decreases every time the original text is repeated.

  According to the twelfth aspect of the present invention, it is easy to acquire correct pronunciation by gradually generating a sound without depending on a model.

  A thirteenth aspect of the invention is a learning support program according to the twelfth aspect of the present invention, in which the computer is activated without performing reproduction by the original content notifying unit after the predetermined number of times of voice reproduction by the original content notifying unit has been completed. It further functions as reproduction display control means for controlling display by the image display means, and recording means for recording user's pronunciation when controlled by the reproduction display control means as user pronunciation information.

  In the thirteenth invention, the learning support program causes the computer to further function as reproduction display control means (S51) and recording means (S53, 86). The reproduction display control means controls to display the accent image display means without performing reproduction by the original text content notification means after the predetermined number of times of sound reproduction by the original text content notification means has been completed. The recording means records the user's pronunciation as controlled by the reproduction display control means as user pronunciation information.

  According to the thirteenth aspect, the model is gradually reduced and is not played back at the end, and the user's pronunciation is recorded at this time, so that the pronunciation that has been worn is clearly recorded (so that the model does not get worn). Can do.

  A fourteenth aspect of the invention is a learning support program according to the fifth aspect of the invention, in which the computer records the user's pronunciation as the user pronunciation information when the current position image is displayed on the screen by the current position image display means. It further functions as a recording means.

  In the fourteenth invention, the learning support program causes the computer to further function as recording means (S53, 86). The recording means records the user's pronunciation when the current position image is displayed on the screen by the current position image display means as user pronunciation information.

  According to the fourteenth aspect, the user's pronunciation can be recorded while showing the user how fast the model is pronounced through the current position image.

  A fifteenth aspect of the invention is a learning support program according to the thirteenth or fourteenth aspect of the invention, in which the computer is operated based on the user pronunciation information recorded by the recording means and the original content information stored in the storage medium. It further functions as an evaluation means for evaluating the accuracy of the rhythm in which the user utters the original text.

  In the fifteenth aspect, the learning support program causes the computer to further function as evaluation means (S78). The evaluation means evaluates the accuracy of the rhythm in which the user utters the original text based on the user pronunciation information recorded by the recording means and the original text content information stored in the storage medium.

  According to the fifteenth aspect, by evaluating the recorded one, the learning result can be recognized and the willingness to learn can be increased.

  A sixteenth aspect of the present invention is a learning support program according to any one of the first to fifteenth aspects of the present invention, wherein the computer reproduces a tick sound that follows a predetermined rhythm before a predetermined point in time when the user starts sounding. It further functions as a tick sound reproducing means.

  In the sixteenth invention, the learning support program causes the computer to further function as tick sound reproducing means (S27). The tick sound reproducing means reproduces a tick sound according to a predetermined rhythm before a predetermined time point when the user starts to pronounce.

  According to the sixteenth aspect, the timing of the start of sound generation can be recognized by the tick sound. In addition, the average speed of pronunciation can be recognized in advance.

  The seventeenth invention stores original text content information (781a, 781d) indicating the content of an original text (ES) to be language-learned and accent position information (781c) indicating the position of the original text on the time axis. Storage medium (48), original text content notifying means (S55, S105) for notifying the user of the content of the original text based on the original text content information, and an accent on the original text when the original text content notifying means is informed A learning support apparatus comprising accent image display means (S111) for displaying an accent image (G, AP1, AP2,...) On the screen based on the accent position information for informing the user of the position on the time axis. 10).

  In the seventeenth invention, similarly to the first invention, the user can acquire the natural pronunciation of the original text.

  According to the present invention, a learning support program and a learning support device that can acquire the natural pronunciation of the original text are realized.

  The above object, other objects, features and advantages of the present invention will become more apparent from the following detailed description of embodiments with reference to the drawings.

It is an external view of the game device which is one Example of this invention, and shows the front in an open state. It is an external view of a game device, and shows the side in an open state. 2A and 2B are external views of a game apparatus, wherein FIG. 1A shows one side surface in a closed state, FIG. 1B shows an upper surface in the closed state, FIG. 1C shows the other side surface in the closed state, and FIG. Each is shown. It is an illustration figure which shows a mode that the game device was hold | gripped by the user. It is a block diagram which shows an example of the electric constitution of a game device. It is an illustration figure which shows an example of a training screen. It is an illustration figure which shows an example of a reproduction | regeneration screen. (A)-(C) is an illustration figure which shows a part of change (first half) of a training screen. (A)-(C) are the illustrations which show the other part (latter half) of the change of a training screen. (A)-(E) is an illustration figure which shows an example of the animation effect given to a tick mark. (A)-(E) is an illustration figure which shows an example of the animation effect given to an accent mark. It is an illustration figure which shows a part of memory map of a main memory. It is an illustration figure which shows an example of problem data. It is an illustration figure which shows the relationship between the constant and variable utilized by a language learning process, and a speech waveform, a tick mark, and an accent mark along a time axis. It is a flowchart which shows a part of CPU operation | movement. It is a flowchart which shows another part of CPU operation | movement. It is a flowchart which shows the other part of CPU operation | movement. It is a flowchart which shows a part of others of CPU operation | movement. It is a flowchart which shows another part of CPU operation | movement. It is a flowchart which shows the other part of CPU operation | movement.

  1 to 3 show the appearance of a game apparatus 10 that is an embodiment of the present invention. The game apparatus 10 is a folding game apparatus. FIGS. 1 and 2 show the game apparatus 10 in an open state (open state), and FIG. 3 shows the game apparatus 10 in a closed state (closed state). ing. FIG. 1 is a front view of the game apparatus 10 in the open state, and FIG. 2 is a side view of the game apparatus in the open state. The game apparatus 10 has two display devices (LCDs 12 and 14) and two cameras (cameras 16 and 18), picks up an image with the camera, displays the picked-up image on a screen, and data of the picked-up image Can be saved.

  The game apparatus 10 has a small size that can be held with both hands or one hand in the open state.

  The game apparatus 10 has two housings, a lower housing 20 and an upper housing 22. The lower housing 20 and the upper housing 22 are connected so as to be openable and closable (foldable). In this embodiment, each of the housings 20 and 22 has a horizontally long rectangular plate shape, and is connected so as to be rotatable at the long side portions of each other.

  The upper housing 22 is rotatably supported by a part of the upper side of the lower housing 20. As a result, the game apparatus 10 is in a closed state (a state where the angle between the lower housing 20 and the upper housing 22 is approximately 0 ° (see FIG. 3)) and an open state (the lower housing 20 and the upper housing 22 are separated from each other). A state where the angle formed is about 180 ° (see FIG. 2)). The user normally uses the game apparatus 10 in the open state, and stores the game apparatus 10 in the closed state when the game apparatus 10 is not used. In addition to the above-described closed state and open state, the game apparatus 10 can set an angle between the lower housing 20 and the upper housing 22 to an arbitrary value between the closed state and the open state by a frictional force generated in the hinge. Can be maintained at an angle. That is, the upper housing 22 can be made stationary with respect to the lower housing 20 at an arbitrary angle.

  First, the structure provided in the lower housing 20 will be described. As shown in FIG. 1, the game apparatus 10 includes a lower LCD (liquid crystal display device) 12. The lower LCD 12 has a horizontally long shape, and is arranged such that the long side direction coincides with the long side direction of the lower housing 20. The lower LCD 12 is housed in the lower housing 20. The lower LCD 12 is provided on the inner side surface of the lower housing 20. Therefore, when the game apparatus 10 is not used, it is possible to prevent the screen of the lower LCD 12 from becoming dirty or damaged by keeping the game apparatus 10 closed. In this embodiment, an LCD is used as the display device. However, any other display device such as a display device using EL (Electro Luminescence) may be used. The game apparatus 10 can use a display device having an arbitrary resolution. When the game apparatus 10 is used as an imaging apparatus, the lower LCD 12 is mainly used for displaying an image captured by the camera 16 or 18 in real time (through display).

  The inner surface of the lower housing 20 is formed in a substantially flat shape. In the center of the inner side surface, an opening 20b for exposing the lower LCD 12 is formed. An opening 20c is formed on the left side (the y-axis negative side in the drawing) of the opening 20b, and an opening 20d is formed on the right side of the opening 20b. The openings 20b and 20c are for exposing the key tops (the upper surfaces of the buttons 24a to 24e). The screen of the lower LCD 12 housed in the lower housing 20 is exposed from the opening 20b, and the key tops are exposed from the openings 20c and 20d. Thus, on the inner side surface of the lower housing 20, non-screen areas (dotted line areas A1 and A2 shown in FIG. 1; specifically, each of the left and right sides of the opening 20b for the lower LCD 12 provided in the center. An area for arranging the buttons 24a to 24e; a button arrangement area) is provided.

  The lower housing 20 is provided with buttons 24a to 24i and a touch panel 28 as input devices. As shown in FIG. 1, among the buttons 24 a to 24 i, the direction input button 24 a, the button 24 b, the button 24 c, the button 24 d, the button 24 e, and the power button 24 f are provided on the inner surface of the lower housing 20. The direction input button 24a is used for, for example, a selection operation, and the buttons 24b to 24e are used for, for example, a determination operation or a cancel operation. The power button 24f is used to turn on / off the power of the game apparatus 10. Here, the direction input button 24a and the power button 24f are provided on one side (left side in FIG. 1) with respect to the lower LCD 12 provided near the center of the lower housing 20, and the buttons 24b to 24e are provided on the lower side. It is provided on the other side (right side in FIG. 1) with respect to the LCD 12. The direction input button 24 a and the buttons 24 b to 24 e are used for performing various operations on the game apparatus 10.

  3A is a left side view of the game apparatus 10 in the closed state, FIG. 3B is a front view of the game apparatus 10, and FIG. 3C is a right side view of the game apparatus 10. FIG. 3D is a rear view of the game apparatus 10. As shown in FIG. 3C and as shown in FIG. 3A, the volume button 24 i is provided on the left side surface of the lower housing 20. The volume button 24 i is used to adjust the volume of the speaker 34 provided in the game apparatus 10. Further, as shown in FIG. 3D, the button 24 h is provided at the right end portion of the upper surface of the lower housing 20. The button 24 g is provided at the left end portion of the upper surface of the lower housing 20. The buttons 24g and 24h are used to perform, for example, a shooting instruction operation (shutter operation) on the game apparatus 10. Both buttons 24g and 24h may function as shutter buttons. In this case, a right-handed user can use the button 24h, and a left-handed user can use the button 24g. Is good. Note that the game apparatus 10 may always make each button 24g and 24h effective as a shutter button, or set right-handed or left-handed (by allowing the user to input settings through a menu program or the like) Data is stored), only the button 24h may be valid when the right-handed setting is set, and only the button 24g may be enabled when the left-handed setting is set.

  As shown in FIG. 1, the game apparatus 10 further includes a touch panel 28 as an input device different from the operation buttons 24 a to 24 i. The touch panel 28 is mounted on the screen of the lower LCD 12. In this embodiment, the touch panel 28 is a resistive film type touch panel. However, the touch panel is not limited to the resistive film type, and any press-type touch panel can be used. In this embodiment, the touch panel 28 having the same resolution (detection accuracy) as that of the lower LCD 12 is used. However, the resolution of the touch panel 28 and the resolution of the lower LCD 12 are not necessarily the same. Further, an insertion port 30 (dotted line shown in FIGS. 1 and 3D) is provided on the right side surface of the lower housing 20. The insertion slot 30 can accommodate a touch pen 36 that is used to perform an operation on the touch panel 28. In addition, although the input with respect to the touchscreen 28 is normally performed using the touch pen 36, it is also possible to operate the touchscreen 28 not only with the touch pen 36 but with a user's finger | toe.

  As shown in FIG. 3A, a cover portion 11 </ b> B that can be opened and closed is provided on the right side surface of the lower housing 20. A connector (not shown) for electrically connecting the game apparatus 10 and the memory card 38 is provided inside the cover portion 11B. The memory card 38 is detachably attached to the connector. The memory card 38 is used, for example, for storing (saving) data of an image captured by the game apparatus 10.

  As shown in FIG. 1, three LEDs 26 a to 26 c are attached to the left side portion of the shaft portion 20 a of the lower housing 20. Here, the game apparatus 10 can perform wireless communication with other devices, and the first LED 26a lights up when wireless communication is established. The second LED 26b is turned on while the game apparatus 10 is being charged. The third LED 26c is lit when the power of the game apparatus 10 is on. Therefore, the three LEDs 26 a to 26 c can notify the user of the communication establishment status, the charging status, and the power on / off status of the game apparatus 10.

  As described above, the lower housing 20 is provided with the input device (touch panel 28 and buttons 24 a to 24 i) for performing operation input to the game apparatus 10. Therefore, the user can hold the lower housing 20 and operate the game device 10 when using the game device 10. FIG. 4 is a diagram illustrating a state in which the user holds the game apparatus 10 with both hands. As shown in FIG. 4, with the LCDs 12 and 14 facing the direction of the user, the user uses the palm and middle finger of both hands, the ring finger and the little finger on the side surface and the outer surface (on the opposite side of the inner surface). Grip the surface). By gripping in this way, the user can operate the buttons 24a to 24e with the thumb while holding the lower housing 20, and can operate the buttons 24g and 24h with the index finger.

  On the other hand, the upper housing 22 is provided with a configuration (camera) for capturing an image and a configuration (display device) for displaying the captured image. Hereinafter, the structure provided in the upper housing 22 will be described.

  As shown in FIG. 1, the game apparatus 10 includes an upper LCD 14. The upper LCD 14 is accommodated in the upper housing 22. The upper LCD 14 has a horizontally long shape and is arranged such that the long side direction coincides with the long side direction of the upper housing 22. The upper LCD 14 is provided on the inner surface of the upper housing 22 (the surface that becomes the inner side when the game apparatus 10 is closed). Therefore, when the game apparatus 10 is not used, the screen of the upper LCD 14 can be prevented from becoming dirty or damaged by keeping the game apparatus 10 closed. As in the case of the lower LCD 12, instead of the upper LCD 14, a display device having any other method and arbitrary resolution may be used. In other embodiments, a touch panel may also be provided on the upper LCD 14.

  The game apparatus 10 includes two cameras 16 and 18. Each camera 16 and 18 is housed in the upper housing 22. As shown in FIG. 1, the inner camera 16 is attached to the inner surface of the upper housing 22. On the other hand, as shown in FIG. 3B, the outer camera 18 has a surface opposite to the surface to which the inner camera 16 is attached, that is, the outer surface of the upper housing 22 (when the game apparatus 10 is in a closed state). Attached to the outer surface). As a result, the inner camera 16 can capture an image in the direction in which the inner surface of the upper housing 22 faces, and the outer camera 18 has a direction opposite to the image capturing direction of the inner camera 16, that is, the outer surface of the upper housing 22 is It is possible to image the direction in which it faces. As described above, in this embodiment, the two cameras 16 and 18 are provided so that the imaging directions are opposite to each other. Therefore, the user can image two different directions without changing the game apparatus 10. For example, the user can shoot a scene viewed from the game apparatus 10 with the inner camera 16 and shoot a scene viewed from the game apparatus 10 in the direction opposite to the user with the outer camera 18. Can do.

The inner camera 16 is attached to the center of a shaft portion 22 a formed at the lower center portion of the upper housing 22. That is, the inner camera 16 is attached to the center of the portion where the two housings 20 and 22 are connected. Accordingly, when the game apparatus 10 is opened, the inner camera 16 is disposed between the two LCDs 12 and 14 (see FIG. 1). In other words, the inner camera 16 is arranged near the center of the game apparatus 10. The “center of the game apparatus 10” means the center of the operation surface of the game apparatus 10 (the surface formed by the inner surfaces of the housings 20 and 22 in the open state). It can be said that the inner camera 16 is disposed in the vicinity of the center in the horizontal direction of the LCDs 12 and 14.
In this embodiment, when the game apparatus 10 is in the open state, the inner camera 16 is disposed near the center of the game apparatus 10, so that when the user shoots the user himself / herself with the inner camera 16, the user What is necessary is just to hold | maintain the game device 10 in the position which opposes. In other words, if the game device is gripped at the normal gripping position, the user is positioned near the center of the imaging range, and the user can easily be within the imaging range.

  As shown in FIG. 3B, the outer camera 18 is disposed on the upper portion of the upper housing 22 (a portion far from the lower housing 20) when the game apparatus 10 is in the open state. Note that the outer camera 18 does not photograph a user who holds the game apparatus 10, and therefore, it is not necessary to provide the outer camera 18 at the center of the game apparatus 10.

Further, as shown in FIG. 1 or FIG. 3B, the microphone 32 is housed in the upper housing 22. Specifically, the microphone 32 is attached to the shaft portion 22 a of the upper housing 22. In this embodiment, the microphone 32 is attached to the periphery of the inner camera 16 (the side of the y-axis in the figure), and more specifically, is attached to the side of the inner camera 16 on the y-axis positive direction side. In addition, in the shaft portion 22a, a microphone hole 22c is provided at a position corresponding to the microphone 32 (side the inner camera 16) so that the microphone 32 can detect sound outside the game apparatus 10. The microphone 32 may be housed in the lower housing 20. For example, the microphone hole 22 c is provided in the inner surface of the lower housing 20, specifically, the lower left portion (button arrangement region A 1) of the inner surface of the lower housing 20, and the microphone 32 is a microphone in the lower housing 20. It can arrange | position in the vicinity of the use hole 22c.
Further, the microphone 32 has a sound collection direction (direction in which the sensitivity is maximized) substantially parallel to the imaging direction (optical axis) of the inner camera 16 (in other words, the sound collection direction and imaging direction are substantially parallel to the y axis, respectively). It is attached in the direction. As a result, the sound emitted within the imaging range of the inner camera 16 is preferably captured by the microphone 32. That is, the detection of the microphone 32 input and the user detection can be performed simultaneously, and the detection accuracy can be improved.

  As shown in FIG. 3B, the fourth LED 26 d is attached to the outer surface of the upper housing 22. The fourth LED 26d is attached around the outer camera 18 (in this embodiment, on the right side of the outer camera 18). The fourth LED 26d is turned on at the time when photographing is performed by the inner camera 16 or the outer camera 18 (the shutter button is pressed). Further, it lights up while a moving image is shot by the inner camera 16 or the outer camera 18. The fourth LED 26d can notify the subject of photographing that photographing by the game apparatus 10 has been performed (performed).

  Further, the inner side surface of the upper housing 22 is formed in a substantially flat shape. As shown in FIG. 1, an opening 21B for exposing the upper LCD 14 is formed at the center of the inner side surface. The screen of the upper LCD 14 housed in the upper housing 22 is exposed from the opening 21B. Further, one sound release hole 22d is formed on each of the left and right sides of the opening 21B. A speaker 34 is accommodated in the upper housing 22 behind the sound release hole 22d. The sound release hole 22d is a hole for releasing sound from the speaker 34 to the outside.

  In this way, on the inner side surface of the upper housing 22, non-screen areas (dotted line areas B1 and B2 shown in FIG. 1; specifically, the speaker 34 is provided on the left and right sides of the opening 21B for the upper LCD 14 provided in the center. A region for placement; a speaker placement region) is provided. The two sound release holes 22d are arranged in the vicinity of the center in the left-right direction of each speaker arrangement region in the left-right direction, and the lower region (region closer to the lower housing 20) of each speaker arrangement region in the up-down direction. ).

  As described above, when the non-screen area is provided at the same position in the left-right direction in the lower housing 20 and the upper housing 22, the game apparatus 10 is held in a horizontal manner as shown in FIG. In addition, it is configured to be easy to hold even when it is held vertically (when the game apparatus 10 is rotated 90 ° counterclockwise or clockwise from the state shown in FIG. 4).

  As described above, the upper housing 22 is provided with the cameras 16 and 18 that are configured to capture an image, and the upper LCD 14 that is a display means for displaying the captured image. On the other hand, the lower housing 20 is provided with an input device (touch panel 28 and buttons 24a to 24i) for performing an operation input to the game apparatus 10. Therefore, when using the game apparatus 10 as an imaging apparatus, the user grasps the lower housing 20 while viewing the captured image (image captured by the camera) displayed on the upper LCD 14 and inputs to the input apparatus. It can be performed.

  Further, a microphone 32 that is a structure for inputting sound is provided in the vicinity of the camera 16 of the upper housing 22, and thus the game apparatus 10 can also be used as a recording apparatus. Further, the user can input voice through the microphone 32, and the game apparatus 10 can execute game processing and language learning processing (described later) based on the microphone input information.

  FIG. 5 is a block diagram showing an internal configuration (electrical configuration) of the game apparatus 10. As shown in FIG. 5, the game apparatus 10 includes a CPU 42, a main memory 48, a memory control circuit 50, a storage data memory 52, a preset data memory 54, a memory card interface (memory card I / F) 44, a wireless communication module. 56, a local communication module 58, a real-time clock (RTC) 39, a power supply circuit 46, and an interface circuit (I / F circuit) 40 and other electronic components. These electronic components are mounted on an electronic circuit board and housed in the lower housing 20 (or the upper housing 22).

  The CPU 42 is information processing means for executing various programs. When the game apparatus 10 is used as an imaging apparatus, a program for that purpose is stored in a memory (for example, the storage data memory 52) in the game apparatus 10. When the CPU 42 executes the program, the game apparatus 10 functions as a photographing apparatus. The program executed by the CPU 42 may be stored in advance in a memory in the game apparatus 10, may be acquired from the memory card 38, or acquired from another device through communication with another device. May be.

  A main memory 48, a memory control circuit 50, and a preset data memory 54 are connected to the CPU 42. A storage data memory 52 is connected to the memory control circuit 50. The main memory 48 is a storage means used as a work area or buffer area for the CPU 42. That is, the main memory 48 stores various data used for game processing and language learning processing, and stores programs acquired from the outside (such as the memory card 38 and other devices). In this embodiment, for example, PSRAM (Pseudo-SRAM) is used as the main memory 48. The storage data memory 52 is a storage unit for storing a program executed by the CPU 42, data of images taken by the cameras 16 and 18, and the like. The storage data memory 52 is composed of, for example, a NAND flash memory. The memory control circuit 50 is a circuit that controls reading and writing of data with respect to the storage data memory 52 in accordance with instructions from the CPU 42. The preset data memory 54 is a storage unit for storing data (preset data) such as various parameters set in advance in the game apparatus 10. As the preset data memory 54, a flash memory connected to the CPU 42 by an SPI (Serial Peripheral Interface) bus can be used.

  The memory card I / F 44 is connected to the CPU 42. The memory card I / F 44 reads and writes data to and from the memory card 38 attached to the connector in accordance with instructions from the CPU 42. In this embodiment, image data captured by each camera 16 and 18 is written to the memory card 38, or image data stored in the memory card 38 is read from the memory card 38 and stored in the storage data memory 52. Or

  The wireless communication module 56 is, for example, IEEE 802.11. It has a function of connecting to a wireless LAN by a method compliant with the b / g standard. Further, the local communication module 58 has a function of performing wireless communication with the same type of game device by a predetermined communication method. The wireless communication module 56 and the local communication module 58 are connected to the CPU 42. The CPU 42 transmits / receives data to / from other devices via the Internet using the wireless communication module 56, and transmits / receives data to / from other game devices of the same type using the local communication module 58. be able to.

  In addition, an RTC 60 and a power supply circuit 46 are connected to the CPU 42. The RTC 60 counts the time and outputs it to the CPU 42. The CPU 42 calculates the current time (date) based on the time counted by the RTC 60 and detects the operation timing such as image capture. The power supply circuit 46 controls power from a power source (battery; housed in the lower housing) of the game apparatus 10 and supplies power to each component of the game apparatus 10.

  Further, the game apparatus 10 includes a microphone 32 and a speaker 34. The microphone 32 and the speaker 34 are each connected to the I / F circuit 40. The microphone 32 detects the user's voice and outputs a voice signal to the I / F circuit 40. The speaker 34 outputs a sound corresponding to the sound signal from the I / F circuit 40. The I / F circuit 40 is connected to the CPU 42. The touch panel 28 is connected to the I / F circuit 40. The I / F circuit 40 includes a voice control circuit that controls the microphone 32 and the speaker 34 and a touch panel control circuit that controls the touch panel. The voice control circuit performs A / D conversion and D / A conversion on the voice signal, or converts the voice signal into voice data of a predetermined format. The converted voice data (user's pronunciation data when language learning is performed) is written in the voice area of the main memory 48 (recording area 86 when language learning is performed: see FIG. 12). When the game apparatus 10 is used as a recording apparatus, the sound data stored in the sound area is later written into the storage data memory 52 via the memory control circuit 50 (if necessary, the memory card I / It is recorded on the memory card 38 via F44). The voice data (microphone input information) stored in the voice area is also used for various game processes. The touch panel control circuit generates touch position data in a predetermined format based on a signal from the touch panel 28 and outputs it to the CPU 42. The touch position data indicates the coordinates of the position where the input is performed on the input surface of the touch panel 28. The touch panel control circuit reads signals from the touch panel 28 and generates touch position data at a rate of once per predetermined time. The CPU 42 can know the position where the input has been made on the touch panel 28 by acquiring the touch position data.

  The operation unit 24 includes the buttons 24a to 24i and is connected to the CPU 42. Operation data indicating the input status (whether or not the button is pressed) for each of the buttons 24 a to 24 i is output from the operation unit 24 to the CPU 42. The CPU 42 obtains operation data from the operation unit 24 to execute processing according to the input to the operation unit 24.

  Each camera 16 and 18 is connected to the CPU 42. Each camera 16 and 18 captures an image in accordance with an instruction from the CPU 42 and outputs the captured image data to the CPU 42. The CPU 42 writes the image data from each camera 16 and 18 in an image area (not shown) of the main memory 48. When the game apparatus 10 is used as an imaging apparatus, the image data stored in the image area is later written into the storage data memory 52 via the memory control circuit 50 (if necessary, the memory card I / It is recorded on the memory card 38 via F44). The image data stored in the image area is also used for various game processes.

  Each LCD 12 and 14 is connected to the CPU 42. Each LCD 12 and 14 displays an image in accordance with an instruction from the CPU 42. When the game apparatus 10 is used as an imaging apparatus, the CPU 42 displays an image acquired from one of the cameras 16 and 18 on the upper LCD 14 and displays an operation screen generated by a predetermined process on the lower LCD 12. When a game is played on the game apparatus 10, a game image is displayed on one or both of the LCDs 12 and 14.

  When language learning is performed with the game apparatus 10 configured as described above, the CPU 42 displays a training screen as shown in FIG. 6 on the upper LCD 14 (hereinafter “LCD 14”). The training screen shows the English language text that is the subject of language learning, such as “Did you hear about the poster?”, The Kana sentence KS that indicates the pronunciation of the English text ES, such as “Didge here about the pooster?” Guide bar GB1 to GB3 that show along the guide G the pronunciation time length of the pronunciation range that should be continuously pronounced when the English ES is pronounced, and the position of the accent when the English ES is pronounced Accent marks AP1 to AP3 shown along G are included. Of the accent marks AP1 to AP3, AP2 is highlighted to allow the user to recognize the position of the accent that should be pronounced relatively strongly compared to the other two.

  In general, when a sentence such as English ES is pronounced, several accents appear in one sentence. In this embodiment, a range delimited by such accents (a range from one accent to the next immediately before the next accent) is defined as a tone generation range (hereinafter referred to as “sound generation range”) to be continuously generated. Therefore, the accent marks AP1, AP2,... Are displayed at the top positions of the guide bars GB1, GB2,.

  The CPU 42 also reproduces the English ES audio through the speaker 34 with the training screen displayed on the LCD 14. The training screen further includes a current position pointer P that indicates the current position along the guide G.

  The CPU 42 also reproduces a predetermined number, here two tic sounds at a constant rhythm (time interval) before starting to reproduce the English ES. For this reason, the training screen further includes tick marks TP1 and TP2 that indicate the position of the tick sound along the guide G.

  The training screen configured as described above changes as shown in FIGS. 8A to 8C and FIGS. 9A to 9C. FIGS. 8A to 8C show screen changes during the period in which the tick sound is reproduced (that is, the preparation period before the user pronounces the English sentence ES), and FIGS. 9A to 9C. Indicates a screen change during a period in which the accent sound is reproduced (that is, a pronunciation period in which the user pronounces the English sentence ES). 8A to FIG. 8C and FIG. 9A to FIG. 9C, animation effects added to the tick marks TP1 and TP2 and the accent marks AP1 to AP3 (FIGS. 10 and 11). Reference: below) has been removed.

  First, regarding the preparation period, as shown in FIG. 8A, initially, the English sentence ES, the kana sentence KS, and the guide bars GB1 to GB3 are all displayed in gray, and the current position pointer P is near the left end of the guide G. It is placed at the position of the tick mark TP1.

  When the first tick sound corresponding to the tick mark TP1 is reproduced, as shown in FIGS. 8B and 8C, the current position pointer P starts moving to the right at a constant speed. The second tick sound is reproduced when the current position pointer P passes the next tick mark TP2. Then, at the timing when the current position pointer P passes the first accent mark AP1, the audio reproduction of the English sentence ES is started. The user can recognize the rhythm of the pronunciation and the timing of the pronunciation start through the two tic sounds reproduced in advance.

  As shown in FIGS. 8A to 8C, during the period in which the current position pointer P is located between the tick mark TP1 and the accent mark AP1, the English ES, the Kana sentence KS, and the guide bars GB1 to GB3 are No change occurs. On the other hand, animation effects as shown in FIGS. 10A to 10E are applied to the tick marks TP1 and TP2.

  FIG. 10A corresponds to the timing when the first tick sound is output. At this time, the tick mark TP1 is highlighted to the maximum. Thereafter, as shown in FIG. 10B, as the current position pointer P moves toward the tick mark TP2, the degree of emphasis of the tick mark TP1 decreases. As shown in FIG. 10C, when the current position pointer P passes the tick mark TP2, that is, at the timing when the second tick sound is output, the tick mark TP2 is highlighted to the maximum. The emphasis of the tick mark TP1 is released before the current position pointer P reaches the tick mark TP2.

  Thereafter, as shown in FIG. 10D, as the current position pointer P moves toward the accent mark AP1, the degree of emphasis of the tick mark TP2 decreases, and as shown in FIG. The emphasis of the tick mark TP2 is canceled before the current position pointer P reaches the accent mark AP1. By adding such an animation effect to the tick marks TP1 and TP2, the timing for starting sound generation can be easily obtained.

  Next, regarding the sound generation period, as shown in FIG. 9A, when the current position pointer P enters the range of the guide bar GB1 via the accent mark AP1, the color of the guide bar GB1 changes from gray to red. Further, the color corresponding to the guide bar GB1, that is, “Did you” and “Didge” in the English sentence ES and the kana sentence KS also changes from gray to red.

  Thereafter, as shown in FIG. 9B, when the current position pointer P enters the range of the guide bar GB2 through the accent mark AP2, the color of the guide bar GB2 changes from gray to red. Of ES and Kana sentence KS, the color corresponding to the guide bar GB2, that is, “hear about the” and “here about the” also changes from gray to red. On the other hand, the colors of the guide bars GB1, “Did you” and “Diddy” change from red to black.

  Thereafter, as shown in FIG. 9C, when the current position pointer P enters the range of the guide bar GB3 via the accent mark AP3, the color of the guide bar GB3 changes from gray to red. Of ES and Kana sentence KS, the color corresponding to the guide bar GB3, that is, “poster?” And “Powster?” Also changes from gray to red. On the other hand, the colors of the guide bars GB2, “hear about the” and “here about the” change from red to black. The colors of the guide bars GB1, “Did you” and “Didge” remain black.

  Thus, the user changes the color of the guide bars GB1 to GB3 as the current position pointer P advances, and also changes the corresponding portions of the English sentence ES and the kana sentence KS in the same manner, so that the user can pronounce the current pronunciation range and pronunciation. You can recognize the length of time at a glance. On the other hand, an animation effect as shown in FIGS. 11A to 11E is applied to the accent marks AP1 to AP3.

  FIG. 11A corresponds to the timing of FIG. 10E, and at this time, the accent mark AP1 is not yet highlighted. Thereafter, as shown in FIG. 11B, the current position pointer P moves toward the accent mark AP1, and as shown in FIG. 11C, the current position pointer P is changed to the accent mark AP1 and the accent mark AP1. When the predetermined position between AP2 is reached, the accent mark AP1 is highlighted up to the maximum.

  Thereafter, accentuation of the accent mark AP1 decreases as the current position pointer P moves away from the accent mark AP1, as shown in FIG. 11D, and as shown in FIG. It is released before the pointer P reaches the accent mark AP2. Thereafter, the same animation effect is applied to the accent marks AP2 and AP3. By adding such an animation effect to the accent marks AP1 to AP3, it becomes easy to recognize the timing of the accent sounding.

  The changes in the training screen as shown in FIGS. 8A to 8C and FIGS. 9A to 9C are repeated a predetermined number of times, here five times. The volume at the time of sound reproduction decreases with each repetition, and becomes zero at the last time, that is, the fifth time. The user pronounces the English ES over 5 times, but the first 4 times reproduces the sound reproduced while observing the training screen, and the last time only on the training screen (without the playback sound) Perform pronunciation. This fifth pronunciation is recorded through the microphone 32.

  When the recording is completed, the CPU 42 displays a reproduction screen as shown in FIG. 7 on the LCD 14 and reproduces the sound of the recorded pronunciation from the speaker 34 simultaneously with the vscale. This reproduction screen includes an English sentence ES, a model voice waveform WP1, a user (you) pronunciation voice waveform WP2, and a current position pointer P. The current position pointer P indicates the current position on the time axis (guide G). As the current position pointer P moves, the color of the English sentence ES changes in the same manner as that on the training screen (see FIG. 6), and drawing of two waveforms WP1 and WP2 proceeds. When the drawing is completed, the CPU 42 compares the levels of the two waveforms WP1 and WP2 mainly at the positions of the accents AP1, AP2,..., And evaluates the similarity between the model and the user pronunciation. Then, an evaluation score (not shown) indicating the evaluation result is displayed.

  Next, specific information processing for realizing such language learning will be described with reference to FIGS. FIG. 12 shows a memory map of the main memory 48 when language learning is performed. Referring to FIG. 12, a program area 48a and a data area 48b are formed in main memory 48, and main program 70 corresponding to the flowcharts of FIGS. 15 to 20 is stored in program area 48a. The main program 70 includes, as subroutines, an image processing program 72 for drawing a training screen and the like, and a voice processing program 74 for reproducing the English ES voice and recording the user's pronunciation. The program area 48a further stores an input / output control program 76 for controlling the I / F circuit 40 and the like to input and output video and audio.

  On the other hand, the data area 48b includes a problem area 78, a constant area 80, a variable area 82, a drawing area 84, a recording area 86, and the like. The problem area 78 stores problem 1 data 781, problem 2 data 782,. The structure of the problem 1 data 781 is shown in FIG. Problem 1 data 781 includes text data 781a indicating English ES, text data 781b indicating Kana sentence KS, and accent information indicating positions T (1), T (2),... 781c, waveform data 781d indicating English speech, and size information 781e indicating the data size (number of samples and / or time length W) of English speech. The problem 2 data 782,... Is configured similarly. In the example of FIG. 13, an accent is placed at the beginning of a word, but an accent may appear in the middle of the word.

  In the constant area 80, various constants (V, N, T (n), STEP, Ttick, Ntick: described later) referred to by the main program 70 are stored. The variable area 82 stores various variables (step, scale, tv, vsscale, score: described later) controlled by the main program 70. The drawing area 84 stores image data corresponding to the training screen (see FIG. 6) and the playback screen (see FIG. 7). In the recording area 86, voice data inputted through the microphone 32, that is, user's pronunciation is stored.

  FIG. 14 illustrates the relationship between the main ones of the above constants and variables and the speech waveform, tick mark, and accent mark along the time axis. “V” is the audio data of the model, “scale” is the playback magnification of “V” (eg 0.5 times, 1 time, 1.5 times), and “vscale” is “V” times “scale” times The stretched parts are shown in.

 “T” indicates the current time (elapsed time from the reference time 0), and “tv” indicates the reproduction start time.

  “Ttick” indicates the time interval of the tick sound. Therefore, the interval between the tick marks (TP1 and TP2) is “scale” times “Ttick”, that is, “Ttick × scale”. The interval between the second tick mark TP2 and the first accent mark AP1 is also “Ttick × scale”.

  “Ntick” indicates the number of tic sounds (here, 2), and “N” indicates the number of accents (here, 4). T (n) indicates the time from the reproduction start time (tv) of the nth (1 ≦ n ≦ N) accent when reproduction is performed with scale = 1. Therefore, the accent mark AP1 is “T (1) × scale” to the right from “tv”, the accent mark AP2 is “T (2) × scale” to the right from “tv”, and the accent mark AP3 is “tv”. The accent mark AP4 is positioned to the right by “T (3) × scale”, and the accent mark AP4 is positioned to the right by “T (4) × scale” from “tv”.

  Therefore, a tick sound is generated every “Ttick × scale” for a predetermined time prior to the start of audio reproduction, and the first accent sound is generated after “Ttick × scale” from the last (second) tick sound. The sound reproduction is started by “T (1) × scale” before the first accent sound (tv = Ttick × scale × 2-T (1) × scale). The second accent sound is generated after “T (2) × scale−T (1) × scale” from the first accent sound, and the third accent sound is generated from the second accent sound by “T (3)”. * Scale-T (2) * scale "occurs, and the fourth accent sound occurs after" T (4) * scale-T (3) * scale "from the third accent sound. Here, T (1), T (2),... Are constants determined for each model voice data, and therefore the interval between accent sounds is generally not constant.

  The CPU 42 executes a language learning process according to the flowcharts of FIGS. 15 to 20 based on the program and data of the main memory 48 as described above. This process is executed each time a problem is selected. Referring to FIG. 15, when any problem, for example, problem 1 is selected, first, initial values are set to various variables (see FIGS. 12 to 14) through steps S1 to S7. Specifically, “1” is set to the variable scale in step S1, and “1” is set to the variable step in step S3. Here, the variable step indicates the number of repetitions at the current time. In step S5, “(Ttick × Ntick−T (1)) × scale”, that is, the reproduction start time (see FIG. 14: described above) is calculated from the constants Ttick, Ntick, T (1) and scale, and the result is set to the variable tv. set. In step S7, “V × scale” is calculated from the constants V and scale, and the result is set in the variable vscale. Note that “V” indicates the audio data of the model, and “vscale” indicates the audio data obtained by expanding and contracting the audio data of the “V” model. To In step S9, drawing is performed on the drawing area 84, whereby a training screen as shown in FIG. This training screen drawing process is executed according to the subroutine of FIG. 20 in detail.

  Referring to FIG. 20, first, an initial screen including guide G is drawn in step S101, problem data, for example, problem 1 data 781 (see FIG. 13) is read in step S103, and included in problem 1 data 781 in step S105. An English ES and a Kana sentence KS are drawn based on the text data 781a and 781b. In step S107, the expansion / contraction rate is calculated based on the size information 781e included in the problem 1 data 781. For example, when the horizontal size (display width) of the training screen (see FIG. 6) corresponds to the time length W0, the expansion / contraction rate (k) is calculated from the time length W of the English speech (vscale) by “k = W0 / (tv + W). ) ”.

  In step S109, the tick marks TP1 and TP2 are drawn at intervals after expansion / contraction. The interval after expansion / contraction is calculated as “Ttick × scale × k” (see FIG. 14). In step S111, the accent marks AP1 to AP3 are drawn at intervals after expansion / contraction. The interval after expansion / contraction is calculated, for example, as “{T (2) × scale−T (1) × scale} × k” for the interval between the accent marks AP1 and AP2 (see FIG. 14). In step S113, the guide bars GB1 to GB3 are drawn with the length after expansion / contraction. For example, for the guide bar GB1, the length after expansion / contraction is calculated as “{T (2) × scale−T (1) × scale−D1} × k” (see FIG. 14). Here, D1 indicates the length of the gap between the end of the guide bar GB1 and the accent mark AP2. The guide bar GB2 is calculated as “{T (3) × scale−T (2) × scale−D2} × k”. As a result, a training screen as shown in FIG. 6 is displayed on the LCD 14. Thereafter, the process returns to the upper layer routine.

  Referring to FIG. 15 again, next, in step S11, “0” is set to the variable t, and in step S13, the display colors of the English sentence ES, the kana sentence KS, and the guide bars GB1 to GB3 are made gray. In step S15, the current position pointer P on the guide G is moved to a position corresponding to the variable t. In step S17, it is determined whether or not there is a speed change operation. If NO, the process proceeds to step S23. When a speed change operation is performed through the operation unit 24 or the touch panel 28, the process proceeds from step S17 to step S19, the reproduction magnification corresponding to the speed after the change is calculated, and the result is set in a variable scale. Then, after the sound reproduction or recording is stopped in step S21, the process returns to step S3. Note that if neither voice playback nor recording is performed at this time, step S21 is passed.

  Referring to FIG. 16, “1” is set to variable n in step S23. The variable n here indicates the number of ticks. In step S25, it is determined whether or not the variable t has crossed “Ttick × scale × (n−1)”. If “NO” here, the process immediately proceeds to step S31. If “YES” in the step S25, the processes of the steps S27 and S29 are executed for the n-th tick, and then the process proceeds to the step S31. In step S27, a tick sound is reproduced. In step S29, animation of a tick mark (TPn) is started. The animation of the tick marks (TP1 and TP2) proceeds in the manner shown in FIGS. 10 (A) to 10 (E).

  In step S31, the variable n is incremented, and in the next step S33, it is determined whether or not the variable n is equal to or less than a constant Ntick. If “YES” here, the process returns to the step S25, and if “NO”, the process proceeds to the step S35.

  Referring to FIG. 17, in step S35, “1” is set to variable n. The variable n here indicates an accent number. In step S37, it is determined whether or not the variable t has crossed “tv + scale × T (n)”. If “NO” here, the process immediately proceeds to a step S45. If “YES” in the step S37, the process of the steps S39 to S43 is executed for the nth accent, and then the process proceeds to the step S45. In step S39, an accent sound is reproduced. In step S41, an accent mark (APn) animation is started. In step S43, the display colors of English ES, Kana sentence KS, and guide bars GB1, GB2,. -1) "" is black and "n" is red. The animation of the accent marks (AP1, AP2,...) Proceeds in the manner shown in FIGS. 11 (A) to 11 (E). The display colors of the English sentence ES, the kana sentence KS, and the guide bars GB1 to GB3 change in the manner shown in FIG. 9 (A) to FIG. 9 (C).

  In step S45, the variable n is incremented, and in the next step S47, it is determined whether or not the variable n is a constant N or less. If “YES” here, the process returns to the step S37, and if “NO”, the process proceeds to the step S49.

  Referring to FIG. 18, in step S49, it is determined whether or not variable t straddles variable tv. If NO, the process immediately proceeds to step S57. If “YES” in the step S49, it is further determined whether or not the variable step has reached the constant STEP in a step S51. If step = STEP, the recording process is started in step S53, and then the process proceeds to step S57. If step <STEP, in step S55, the volume is calculated from the variable step, and reproduction of the variable vscale is started at the volume. In calculating the volume, the maximum value is obtained when n = 1, the value decreases as n increases, and “0 when n = N” or “minimum value when n = N−1”. A function (or a table describing such a calculation result) is used. Then, the process proceeds to step S57.

  In step S57, it is determined whether or not the variable t is equal to or greater than the value obtained by adding {reproduction time of variable vsscale} to variable tv. If NO, the process waits for a predetermined time in step S59, and then sets the value of variable t. Update (t = t + standby time). Then, the process returns to step S15. If “YES” in the step S57, it is determined again in a step S59 whether or not the variable step has reached the constant STEP. If “NO” in the step S59, the variable step is incremented in a step S63, and then, the process returns to the step S11. If “YES” in the step S59, the recording is stopped in a step S61, and then, the process proceeds to the step S73.

  Referring to FIG. 19, in step S73, the variable vscale and the recorded data (user pronunciation) in the recording area 86 are compared mainly at the level at the accent position, and an evaluation score indicating the similarity between the two is calculated. The calculation result is set in a variable score. In step S75, drawing of the playback screen (that is, English ES, waveform WP1 of variable vsscale and waveform WP2 of recorded data: see FIG. 7) in drawing area 84, playback of variable vsscale, and playback of recorded data is started. To do. In step S77, “0” is set to the variable t. In step S79, the current position pointer P is moved to a position corresponding to the variable t. In step S81, it is determined whether or not the variable t is equal to or greater than {reproduction time of variable vsscale}. If NO, the value of variable t is updated after waiting for a predetermined time in step S83 (t = t + standby). time). Then, the process returns to step S79.

  If “YES” in the step S81, the value of the variable score is drawn in a step S85. In step S87, it is determined whether or not a re-listening operation is performed. If YES, the process returns to step S75. If NO in step S87, the presence / absence of an end operation is further determined in step S89, and if NO in this case, the process returns to step S87. If “YES” in the step S89, the language learning process regarding the problem (problem 1) is ended, and the same process is executed for the next problem (problem 2).

  As is clear from the above, the game apparatus 10 of this embodiment includes a CPU 42 and a main memory 48. The main memory 48 includes original text content information (original text character information 781a and original text audio information 781d) indicating the contents of the original ES to be language-learned, and accent position information 781c indicating the position of the original text on the time axis. (See FIG. 13), the CPU 42 notifies the contents of the original text ES based on the original text content information (781a, 781d) (displays characters or reproduces voice) (S55, S105). Further, during such notification, an accent image (that is, a guide G indicating the time axis and an accent mark AP1, indicating the accent position) for notifying the user of the position of the accent on the time axis regarding the original ES. AP2,...) Are displayed on the LCD 14 based on the accent position information 781c (S111).

  Thus, by visually indicating the position of the accent (AP1, AP2,...) On the time axis (G) of the original ES while notifying the content of the original ES, the user can determine the timing at which the accent should be pronounced. Because you know the length of each section separated by accents, you can acquire the natural pronunciation of the original text.

  It should be noted that the processes such as the reproduction of the tick sound and accent sound, the animation of the tick mark and accent mark, and the display of the guide bar are started in this embodiment by determining whether or not the current time (variable t) has crossed a certain value. However, in another embodiment, the frame (number) for starting processing may be calculated in advance, and it may be started by determining whether or not the current frame has become that frame.

  In this embodiment, the language of the original document ES is English, but natural pronunciation can be acquired for other foreign languages such as German, French, and Chinese. If the original sentence ES is Japanese and the kana sentence KS is written in a foreign language, Japanese can be learned.

  Although the game apparatus 10 has been described above, the present invention notifies the original ES based on the storage medium storing the original content information 781a and 781d and the accent position information 781c, and the original content information 781a and 781d. The present invention can be applied to a learning support device (such as a PC or a portable terminal) that includes a computer that displays an accent image on the screen based on the accent position information 781c while displaying characters and / or reproducing the voice of the original ES.

10 ... Game device 14 ... (Upper) LCD
32 ... Microphone 34 ... Speaker 42 ... CPU
48 ... main memory G ... guide (time axis)
P ... Current position pointer TP1, TP2 ... Tick mark AP1 to AP4 ... Accent mark GB1 to GB4 ... Guide bar (time interval image)

Claims (17)

A computer of a learning support apparatus having a storage medium storing original text content information indicating the content of an original text to be subjected to language learning and accent position information indicating a position of the original text on the time axis with respect to the original text,
Based on the original text content information, an original text content notifying means for notifying a user of the content of the original text, and when the content of the original text is notified by the original text content notifying means, the accent on the original text on the time axis A learning support program that causes an accent image for notifying the user of the position of the image to be displayed on a screen based on the accent position information. The computer,
Based on the accent position information, the original text informed by the original text content notifying means is divided into time intervals at the position of the accent on the time axis, and indicates the position and length of the time interval on the time axis The learning support program according to claim 1, which further functions as time interval image display means for displaying a time interval image on the screen. The computer,
The learning support program according to claim 1, wherein the learning support program further functions as highlight display means for highlighting an accent image to be pronounced at the present time among the accent images displayed by the accent image display means. The computer,
Based on the accent position information, the original text informed by the original text content notifying means is divided into time intervals at the position of the accent on the time axis, and indicates the position and length of the time interval on the time axis Further function as time interval image display means for displaying the time interval image on the screen,
The learning support program according to claim 3, wherein the highlighting unit further highlights a time interval image corresponding to the current time among the time interval images displayed by the time interval image display unit. The computer,
5. The learning support program according to claim 1, further causing a current position image display unit to display a current position image indicating a current position on the time axis on the screen. The computer,
6. The learning support program according to claim 1, further causing an accent sound corresponding to the accent to be played back as an accent sound reproducing unit that reproduces the accent sound at time intervals based on the accent position information. The storage medium stores original text information indicating characters of the original text as the original text content information,
The learning support program according to any one of claims 1 to 6, wherein the source text content notifying unit displays the source text on the screen based on the source text information. The computer,
Based on the accent position information, the original text displayed by the original text content notifying means is divided into time intervals at positions on the time axis of the accent, and the current text of the original text displayed by the original text content notifying means is The learning support program according to claim 7, further causing a character string highlighting means for highlighting a character string belonging to a corresponding time section. The storage medium stores original audio information indicating the original audio as the original content information,
The learning support program according to any one of claims 1 to 8, wherein the source text content notifying means reproduces the source text based on the source text information. Allowing the computer to further function as a playback speed setting means for setting the playback speed of the original audio;
The learning support program according to claim 9, wherein the source text content notifying unit plays back the voice of the source text at a playback speed set by the playback speed setting unit.   The learning support program according to claim 9 or 10, wherein the original text content notifying means repeatedly reproduces the voice of the original text.   The learning support program according to claim 11, wherein the source text content notifying unit reproduces the source text so that the volume decreases every time the source text is repeated. The computer,
A reproduction display control means for controlling the display by the accent image display means without performing the reproduction by the original text content notification means after the predetermined number of times of sound reproduction by the original text content notification means has been completed; and the reproduction display The learning support program according to claim 12, further causing it to function as a recording unit that records a user's pronunciation when controlled by the control unit as user pronunciation information. The computer,
6. The learning support program according to claim 5, wherein the learning support program further functions as recording means for recording a user's pronunciation as user pronunciation information when the current position image is displayed on the screen by the current position image display means. The computer,
The apparatus further functions as an evaluation unit that evaluates accuracy of a rhythm in which the user utters the original sentence based on user pronunciation information recorded by the recording unit and the original content information stored in the storage medium. Item 15. The learning support program according to item 13 or 14. The computer,
The learning support program according to any one of claims 1 to 15, wherein the learning support program further functions as a tick sound reproducing unit that reproduces a tick sound according to a predetermined rhythm before a predetermined point in time when the user starts sounding. A storage medium storing original text content information indicating the content of the original text subject to language learning and accent position information indicating the position of the accent on the time axis with respect to the original text;
Based on the original text content information, an original text content notifying means for notifying a user of the content of the original text, and when the content of the original text is notified by the original text content notifying means, the accent on the original text on the time axis A learning support apparatus comprising accent image display means for displaying an accent image for informing the user of the position at the position based on the accent position information.

Images