WO2001095079A1

WO2001095079A1 - Information processing system using balloon pattern, language translation information processing system using balloon pattern, on-line information processing system using balloon pattern, and computer-readable information storage medium

Info

Publication number: WO2001095079A1
Application number: PCT/JP2001/004743
Authority: WO
Inventors: Shoji Hayashi
Original assignee: Hayashi Office Co., Ltd.
Priority date: 2000-06-05
Filing date: 2001-06-05
Publication date: 2001-12-13
Also published as: JPWO2001095079A1; AU2001262696A1

Abstract

An information processing system for displaying contents containing a balloon pattern, which contains a balloon character display information field for displaying balloon characters. To these balloon characters, there are correlated voice information and/or character information, one of which is outputted by designating the balloon pattern with a mouse or the like. The balloon characters, the voice information and the character information may be different languages or may be set by the designation of a user. The contents are preferably cartoons. The information processing system may take a configuration of a server-client model, may be a mobile device or may have a storage medium. It is possible to add an accounting component, if necessary.

Description

Description Information processing system using balloon-shaped figures, language translation information processing system using balloon-shaped figures, online information processing using balloon-shaped figures, and computer-readable information storage medium Field

The present invention relates to an information processing system using a balloon-shaped figure using a terminal such as a computer. More specifically, the present invention relates to an information processing system using speech information 'and character information associated with a balloon-shaped figure such as a manga. A system that performs processing. In addition, using speech and text information translated into languages, users can use networks such as the Internet or electronic media such as CD-ROMs to print images such as cartoons and English including text information. The present invention relates to a language translation information processing system capable of learning a foreign language, and a computer readable information storage medium. Background art

Conventionally, technology has been developed for providing guidance on operation methods by voice in computer devices.

Techniques related to such a field include, for example, Japanese Patent Application Laid-Open No. 8-82531 “Guidance device using sound information”, which is offline as a guide for selecting an operation menu of a word processor or a personal computer. A user interface device based on sound information used in the above is disclosed.

The configuration of the voice guidance device of the present invention corresponds to an input unit for controlling a cursor or a pointer, a position detecting means for detecting a position of the cursor or the pointer on the screen, and a position detected by the position detecting unit. Music and voice It is composed of a voice control information storage means, a voice generation means for generating the musical sound and voice, and a voice output means such as a speaker.

However, this technology is not used by connecting to a network such as the Internet, but is also provided with software etc. installed in advance in equipment, and is more than a voice guide for their operation. It does not have the function of

As a technology that uses networks such as the Internet, a technology has been developed in which sound is played when a banner ad is clicked.

This means that when you click on the image of the banner ad, the sound is played on "Real Audio J" (trademark), and the sound data is stored on an audio server such as "Rea1Networks" (trademark). A hyperlink is attached to the audio file to be linked, and clicking on it will play the audio file at the hyperlink destination.

However, according to this technology, it is the first advertisement system to output sound by clicking, and clicking on the displayed banner ad will jump to the advertiser's website etc. The fact that most users are familiar with this means that users who are not interested in advertising and pass by will have the disadvantage that voice messages will not be transmitted to users.

Conventionally, for example, in a language translation system using the Internet, a user such as a student inputs data from a computer to a form on a web page and transmits the data to a server system. In general, CGI programs read and write and process the received data to and from text-format files, and return text data such as text to a predetermined Web page for display. .

For example, in Japanese Patent Application Laid-Open No. Hei 11-282828 “CD-ROM Internet“ King of Translation ”(registered trademark)”, a relatively difficult sentence was encountered in the foreign language original text on the Internet homepage. If you drag the mouse pointer to that point and flip it, the correct pronunciation of the original text will be played at the same time. A technology has been disclosed that has improved the function of displaying the translated text of the section.

However, in the system described above, uniform translation processing is performed using a single CGI program or the like, and interactive (bidirectional) matching the proficiency of each student is required. It has a problem that it is not possible to deal with the problem, entertainable processing, or complex processing using multimedia files such as images.

Further, Japanese Patent Application Laid-Open No. Hei 9-125325 "Translation Correction Support Device" discloses the following technology. According to this, a model answer is decomposed into a plurality of templates having a word / phrase structure using expert knowledge. For each template, a message for the answer that the learner will translate is prepared in advance using the expert's knowledge. Then read these, and then read the answer. Next, the student checks whether the learner has taken the basic usage correctly, then classifies the part of speech for each word in the answer, and finally parses the answer. After that, the answer sentence is evaluated for each template, and a message with the content corresponding to the evaluation is presented to the learner. Since the answer text is evaluated for each template, it is possible to obtain the same correction result as an expert who does it manually.

However, even with this technology, individual students can automatically make interactive (two-way) responses without relying on human power while making selections on the screen according to their own proficiency, etc. It is not possible to perform complex processing using multimedia files such as images, or sexual processing.

On the other hand, comics such as comics that use image information and character information, for example, are rarely used for secondary use except for commercialization of character dolls and the like of comics.

It would be desirable to be able to solve the above-mentioned problems using these online.In addition, when using these manga and other copyrighted works, the consent of the copyright holders, etc. was obtained, and the profits could be returned to the right holders. If the system, The contents of the system are also enhanced.

Japanese Patent Application Laid-Open No. 7-933368 “Comic Publication” is disclosed as a technology relating to the contents of computerized comics.

In this method, image data obtained by scanning a comic original image and sound data consisting of speech and mimic sounds in the balloon are recorded on an optical compact disc in a format that can be played back simultaneously, and page numbers and frames in magazines are recorded. Codes organized according to numerical values and contents in order are placed at the beginning of each data, and are used as headings for search and playback. Use a playback device with manual playback in normal and reverse order, pause and restart of audio by pausing, switching between manual playback for each page and continuous automatic playback. A means for recording is provided, and a means for designating a reproduction start position at the beginning of reproduction, a position where reproduction is interrupted, a page number, a heading, or the like is provided. ;

However, the above technology relates to the technology of playing or stopping the contents obtained by digitization, and plays back the speech of the speech bubble selected by the user, and automatically performs other operations in the same way as the speech. It does not have the technical characteristics to display the information of

If this can be achieved, various effects can be obtained by synthesizing images, characters, voices, etc. when playing back content. In particular, by combining images and characters with the corresponding translated characters and sounds, it is possible to provide an interactive system for learning and providing a system for learning.

SUMMARY OF THE INVENTION The present invention has been made in view of the above problems, and focuses on the shape of speech balloons used in comics and the like when reproducing content, and sets the association with character information and voice information in the speech balloon. Another object of the present invention is to provide a system in which a user can connect to a computer terminal or the Internet from a terminal, synthesize images, characters, sounds, and the like and perform various reproduction, output, and display.

In particular, combining images and text with the corresponding translated text and audio By doing so, it is possible to provide a system for performing translation and learning. It is an object of the present invention to provide an online foreign language translation system characterized by being able to output text information and voice information in a translated language according to a user's selection of image information and text information online. I do.

Another object of the present invention is to provide a system in which, in the system of the present invention, when a copyrighted work such as a comic is used, the consent of a copyright holder or the like is obtained and the profit is returned to the right holder. The purpose is to provide and enhance the content of this system.

Another object of the present invention is to provide an information storage medium storing data and a program for realizing the above system, and in particular, to provide an information storage medium usable in both a computer device and a portable information terminal. The purpose is also to do. Disclosure of the invention

In the invention described in claim 1, in order to solve the above problems, a computer provided with an input means, a control means, a display means, an output means, a storage means, etc. ゃ a user terminal such as a portable information terminal In this system, information processing is performed by user operation,

The storage means provided in the user terminal or the storage means connected to the user terminal via a computer network includes:

Image information storage means for storing image information;

Voice information storage means for storing voice information;

Information processing means for displaying character information and voice information on the user terminal is provided at least;

If necessary, a character information storage unit for storing character information is provided, and an image file, an image file, a plurality of images inserted in an HTML file, an XML file, or the like stored in the image information storage unit. Files such as slideshows including files are displayed on the user terminal. In the image file, one or a plurality of balloon-shaped figures surrounded by a linear figure having an arbitrary shape and thickness are arranged at an arbitrary position in the image and displayed on the user terminal. One or a plurality of character information display fields for displaying character information as required are arranged in the image file, and the balloon is arranged in the balloon-shaped figure arranged in the image file. A balloon character information field for displaying image information including a balloon character on predetermined coordinates in the image information represented by the shape graphic is arranged, and the image information including the balloon character is stored in the voice information storage unit. In addition to being associated with the stored predetermined voice information file, the association with the predetermined character information file stored in the character information storage unit is set in advance. It is possible to keep

Depending on the user's actions, such as placing the cursor in the balloon character information field displayed in the balloon character information field placed in the displayed balloon-shaped figure, placing the cursor, and clicking, etc. A process in which the information processing means provided in the storage means detects the operation of the user;

Transmitting to the user terminal the voice information indicating the location of the link information added to the balloon-shaped character information field arranged in the balloon-shaped figure, to a user terminal; Output and playback by

If the association with the predetermined character information file has been set in advance, the character information indicating the location in the link information added to the balloon character information field is displayed in the character information display field. And transmitting the data to the user terminal and displaying it on the display means of the user terminal. The information processing system uses a balloon-shaped figure.

In the invention described in claim 2, in order to solve the above-mentioned problems, a computer including an input unit, a control unit, a display unit, an output unit, a storage unit, and the like includes a user terminal such as a portable information terminal. Information by user operation A system in which processing is performed,

Image information storage means for storing image information;

Voice information storage means for storing voice information associated with the character information;

If necessary, a character information storage unit for storing character information is provided, and an image file, an image file, a plurality of image files inserted into an HTML file, an XML file, or the like stored in the image information storage unit. An image file displayed on a user terminal such as a file such as a slide show including an aisle may include one or more files enclosed in a linear figure having an arbitrary shape and thickness at an arbitrary position in the image. In the image file displayed on the user terminal, one or a plurality of character information display fields for displaying character information as necessary are arranged, and the image file is displayed in the image file. The arranged balloon-shaped figure includes a balloon for displaying balloon character information on predetermined coordinates in the image information represented by the balloon-shaped figure. A character information field is arranged, and the balloon character information is associated with a predetermined voice information file stored in the voice information storage means and stored in the character information storage means. It is possible to set the association with a predetermined character information file in advance,

According to the user's action such as placing a cursor in a balloon character information field displayed in a balloon character information field arranged in the displayed balloon-shaped figure, placing a cursor, and clicking, etc. A process in which the information processing means provided in the storage means detects the operation of the user;

The balloon character information file arranged in the balloon graphic The audio information indicating the location indicated by the link information added to the password is output by the audio reproducing means provided in the user terminal, and the processing for reproducing the audio information and the association with a predetermined character information file are set in advance. In this case, a process of displaying the character information indicating the location of the link information added to the balloon character information field in the character information display field by a display unit provided in the user terminal is performed. It is an information processing system using balloon-shaped figures.

In the invention described in claim 3, in order to solve the above-mentioned problems, in the invention described in claim 1 or 2,

One or more character information display fields for displaying character information, which are arranged in the image file displayed on the user terminal, display image information in which data including character information is used as an image information file. Field for

If the association with the predetermined character information file has been set in advance, the character information indicating the location in the link information added to the balloon character information field is displayed in the character information display field. And displaying the image information including the image information by a display means provided in the user terminal. The information processing system uses a balloon-shaped figure.

According to the invention set forth in claim 4, in order to solve the above-mentioned problem, a computer including an input unit, a control unit, a display unit, an output unit, a storage unit, and the like includes a user terminal such as a portable information terminal. A system in which information processing is performed by user operations,

Image information storage means for storing image information;

Voice information storage means for storing voice information;

Information processing means for displaying character information and voice information on a user terminal Is at least provided,

If necessary, a character information storage unit for storing character information is provided, and an image file, an image file, a plurality of image files inserted into an HTML file, an XML file, or the like stored in the image information storage unit. For image files displayed on user terminals such as slide shows including aisles, etc., character information or character information consisting of arbitrary shapes and sizes, etc., at arbitrary locations in the images is used as image data. One or more character information display fields for arranging image information are arranged,

The character information display field is associated with a predetermined voice information file stored in the voice information storage unit, and is linked to a predetermined character information file stored in the character information storage unit. It is possible to set the association of in advance,

The voice information stored in the voice information storage means is post-translation language voice information corresponding to the pre-translation language character information included in the character information field,

The character information stored in the character information storage means is post-translation linguistic character information corresponding to pre-translation linguistic character information included in the character information field,

The information processing means provided in the storage means according to the user's action such as placing a cursor in a character information field, placing a cursor in a character information field, and clicking on a tick, etc., in the language before translation, A process of detecting a user's action;

The link information added to the character information field indicates the location, and the audio information, which is the translated language, is transmitted to the user terminal, and is output by the audio reproduction means included in the user terminal, and the reproduction is performed. Processing,

If the association with the predetermined character information file has been set in advance, the location is indicated in the link information added to the character information display field. And send it to the display means provided in the user terminal. It is a language translation information processing system.

In the invention described in claim 5, in order to solve the above-mentioned problems, in the invention described in claims 1 to 4,

The voice information stored in the voice information storage means is a post-translation language voice information corresponding to the pre-translation language character information included in the balloon character information field,

The character information stored in the character information storage means is post-translation language character information corresponding to the pre-translation language character information included in the balloon character information field,

According to the user's actions, such as placing the cursor on the balloon character information that can be displayed in the balloon character information field, which is indicated in the pre-translation language placed in the displayed balloon-shaped figure, placing the cursor, and clicking. A process in which an information processing unit provided in the storage unit detects an operation of the user;

Transmitting the voice information, which is the translated language, to the user terminal, where the link information added to the balloon-shaped character information field arranged in the balloon-shaped figure indicates the location, to the user terminal; A process of outputting and reproducing by the audio reproducing means provided in the

If the association with the predetermined character information file has been set in advance, the location is indicated in the link information added to the balloon character information field in the character information display field. The balloon according to any one of claims 1 to 4, wherein character information in a language is transmitted to the user terminal and displayed on display means provided in the user terminal. It is characterized by being a language translation information processing system using figures. .

In the invention described in claim 6, in order to solve the above-mentioned problem, in the invention described in claim 4 or 5,

The post-translation voice information stored in the voice information storage means includes a pre-translation language character information included in the balloon character information field. Contains translated language audio information consisting of a number of translated languages,

The post-translation character information stored in the voice information storage unit includes post-translation language character information including a plurality of post-translation languages corresponding to the pre-translation language character information included in the balloon character information field. Included,

A process in which a user selects one of a plurality of translated languages to translate the pre-translation language character information;

The user is required to place the cursor at the balloon character information displayed in the balloon character information field, which is indicated in the pre-translation language placed in the displayed balloon-shaped figure, or to place the cursor and click. A process in which the information processing means provided in the storage means detects the action of the user according to the action;

Transmitting to the user terminal the voice information as the selected translated language, where the location is indicated in the link information added to the balloon-shaped character information field arranged in the balloon-shaped figure; A process of outputting and reproducing by a sound reproducing means provided in the user terminal;

If the association with the predetermined character information file has been set in advance, the character information display field indicates the location in the link information added to the balloon character information field, 6. The balloon according to claim 4 or 5, wherein the character information that is the translated language is transmitted to the user terminal and displayed by a display unit provided in the user terminal. It is characterized by being a language translation information processing system using figures.

In the invention described in claim 7, in order to solve the above-mentioned problems, in the invention described in claims 4 to 6,

In the balloon character information displayed in the balloon character information field, which is indicated by the pre-translation language arranged in the displayed balloon-shaped figure, balloon character information including a plurality of pre-translation languages is prepared in advance. An image file including the balloon-shaped figure is stored in the image information storage means, A process in which a user selects from which of a plurality of pre-translation languages the pre-translation language character information is translated;

Displaying an image file including a balloon character information field composed of the selected pre-translation language;

The user places the cursor in the balloon character information displayed in the balloon character information field, which is displayed in the balloon character information field, placed in the displayed balloon-shaped figure, or places the cursor and clicks it. A process in which the information processing means provided in the storage means detects the action of the user according to the action;

If the association with the predetermined character information file has been set in advance, the location is indicated in the link information added to the balloon character information field in the character information display field. A process of transmitting character information as a language to a user terminal and displaying the same on a display means provided in the user terminal. It is characterized by being a language translation information processing system using balloon-shaped figures.

In the invention described in claim 8, in order to solve the above-mentioned problems, in the invention described in claims 1 to 7,

Images displayed on the user terminal, such as image files inserted into HTML files or XML files stored in the image information storage means, image files, slideshow files including a plurality of image files, etc. Files can be multiple HTML files, multiple image files inserted into XML files, etc., multiple image files consisting of multiple pages, or slide shows including multiple image files that make up multiple pages. A data file The user terminal can select and display a desired page from a plurality of pages,

8. A method according to claim 1, wherein said speech-shaped figure is used to reproduce voice information associated with the selected speech-shaped figure on a selected page and to output character information. An information processing system using the balloon-shaped figure according to any one of the above.

In the invention according to claim 9, in order to solve the above-mentioned problem, in the invention according to claim 8, the data file stored in the storage means has a story such as a comic. Content data, which can be displayed on the user terminal by selecting a desired page from a plurality of pages,

9. The method according to claim 8, wherein a process of reproducing voice information associated with the selected balloon-shaped figure of the selected page using the balloon-shaped figure and outputting character information is possible. It is a content reproduction information processing system using a balloon-shaped figure.

In the invention according to claim 10, in order to solve the above-mentioned problem, in the invention according to claim 9, the storage means stores a plurality of pieces of content data having a story, such as comics. 10. Content reproduction information using a balloon-shaped figure according to claim 9, wherein the content is stored and can be selected and displayed / reproduced from a plurality of content data. It is a processing system.

In the invention according to claim 11, in order to solve the above problems, in the invention according to claims 1 to 10,

The audio information indicating the location in the link information added to the balloon-shaped character information field arranged in the balloon-shaped graphic includes a plurality of audio information having different audio reproduction speeds. Stored in the voice information storage means,

The speech balloon according to any one of claims 1 to 10, wherein a voice playback speed can be selected by setting in advance by a user. It is characterized by being an information processing system using a ladder figure.

In the invention according to claim 12, in order to solve the above problems, in the invention according to claims 1 to 11,

By setting in advance by the user, whether to automatically play voice information and text information, whether to turn pages automatically, select the number of times of voice playback, whether to display text information, The information processing system using a balloon-shaped figure according to any one of claims 1 to 11, characterized in that it is possible to select a reproduction control method such as setting of a reproduction order of the content data. Features.

In the invention described in claim 13, in order to solve the above-mentioned problem, in the invention described in claim 6 or 7 or 11 or 12,

The selection of the language before translation, the selection of the language after translation, the selection of the audio playback speed, or the selection of the playback control method uses tag information such as XML, and stores the settings selected by the user as a setting file in the storage means. Claims 6 or 7 or 11 or 12 characterized in that, by storing the information, the sound is automatically reproduced in accordance with the audio reproduction speed or reproduction control method selected with reference to the setting file during reproduction. It is an information processing system using the balloon-shaped figure described in (1).

In the invention according to claim 14, in order to solve the above problems, in the invention according to claims 1 to 13,

The storage means is a server system including a WEB server, an application server, a database server, and the like connected from a user terminal via a computer network,

Image information storage means for storing image information;

At least an information processing means for transmitting character information and voice information to the user terminal for display is provided.

If necessary, a character information storage means for storing character information is provided, According to a request from the accessing user terminal, a process of transmitting a data file including a balloon-shaped figure to the user terminal and displaying the data file,

According to the user's actions such as placing a cursor in a balloon character information field displayed in a balloon character information field arranged in the displayed balloon-shaped figure, placing a cursor and clicking, etc. A process in which the information processing means provided in the storage means detects the operation of the user;

If the association with the predetermined character information file has been set in advance, the character information indicating the location in the link information added to the balloon character information field is displayed in the character information display field. Online information using a balloon-shaped figure according to any one of claims 1 to 13, wherein the processing is performed by transmitting to a user terminal and displaying the same on a display means provided in the user terminal. It is a processing system. '

In the invention according to claim 15, in order to solve the above-mentioned problem, in the invention according to claim 14, the server system further includes a user terminal via a computer network. Authentication means for authenticating a user accessing the system,

Usage information storage means capable of storing the number of accesses by users, the usage time, the number of content data used, the number of downloads, other information, and information about users;

Usage fee calculation means for calculating a usage fee with reference to the usage information stored in the usage information storage means,

The use of the balloon-shaped figure according to claim 14, wherein it is possible to calculate the usage fee and manage the charge according to the various usage information. It is an online information processing system.

In the invention according to claim 16, in order to solve the above-mentioned problems, in the invention according to claim 14, the system further includes a copyright of text information and image information in a language before translation. Right holder information storage means for storing information about the right holder of

A license fee calculation means to be paid to the right holder; a fee for returning to the right holder is calculated and managed according to the usage fee calculated by the usage fee calculation means; It is an online information processing system using the balloon-shaped figure according to claim 14.

In the invention according to claim 17, in order to solve the above problems, in the invention according to claims 1 to 16,

The storage unit including the image information storage unit, the voice information storage unit, the information processing unit, and the character information storage unit includes:

A data capturing means for capturing, as image information, data of a printed material having image information and balloon character information,

A first data processing means for setting a balloon character information field in the captured image information and storing the data-processed image information in the image information storage means;

A second step of inputting character information and voice information associated with the character information included in the balloon character information field, converting the character information into a translated language as required, and storing the converted language in the character information storage means and the voice information storage means. Data processing means;

An information processing system using the balloon-shaped figure according to any one of claims 1 to 16, characterized by comprising: In the invention according to claim 18, in order to solve the above problem, an image information storage unit that stores image information,

Voice information storage means for storing voice information;

If necessary, a character information storage unit for storing character information is provided, including an image file, an image file, and a plurality of image files inserted in an HTML file, an XML file, or the like stored in the storage unit. The image file displayed on the user terminal, such as a file such as a slideshow, contains one or more files that are surrounded by linear figures of any shape and thickness at any place in the image. Balloon shape is placed,

In the image file displayed on the user terminal, one or a plurality of character information display fields for displaying character information as necessary are arranged, and the balloon-shaped figure arranged in the image information is provided. In the image information, a balloon character information field for displaying a balloon character at a predetermined coordinate in the image information represented by the balloon-shaped figure or a balloon character information for displaying the balloon character information is arranged, and includes the balloon character. The image information or the balloon character information is associated with a predetermined voice information file stored in the voice information storage unit, and the predetermined character information stored in the character information storage unit. The feature is that the association with the file can be set in advance. It is characterized in.

In the invention according to claim 19, in order to solve the above problems, in the invention according to claim 18,

The information storage medium is a mobile phone, a mobile information terminal, an electronic book player,

-A small information storage medium that can be loaded into a portable information terminal such as an audio playback player and can be loaded into a computer device such as a personal computer,

The image information storage means, the voice information storage means, the image information, the voice information, or the character information stored in the character information storage means provided in the information storage medium can be output or reproduced by a portable information terminal. Files for portable information terminals, or files for morphological information terminals that can be output or reproduced on portable information terminals are stored separately from files for computer devices. A computer-readable information storage medium according to claim 18, characterized in that:

In the invention according to claim 20, in order to solve the above problems, in the invention according to claim 18 or 19,

The information storage medium further includes authentication information storage means for storing user authentication information,

When downloading via a computer network or registering a trial version of the user, the authentication permission information is stored in the authentication information storage means when the accounting process or the user registration process is performed, and the playback is performed. A computer-readable information storage medium according to any one of claims 18 and 19, characterized in that the information storage medium is enabled. BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 is a system configuration diagram showing a first embodiment of the present invention. FIG. 2 is a diagram showing a configuration of the storage means.

FIG. 3 shows an example of an image file displayed using the system of the present invention.

FIG. 4 shows an example of an image file displayed using the system of the present invention.

FIG. 5 shows an example of an image file displayed using the system of the present invention. .

Fig. 6 shows an example of the relationship between the image file that is each page of the content such as comics stored in the image information storage means 15 and the balloon character information field arranged in each image file. It is a data schematic diagram which shows. FIG. 7 is a flowchart showing a basic processing flow of the first embodiment of the present invention.

FIG. 8 is a diagram showing an embodiment of the storage area of the storage means.

Figure 9 shows several translated words corresponding to the pre-translation language (here, Japanese). FIG. 7 is a diagram showing a storage area of a storage unit 10 in which speech information composed of words (here, English and French) and character information composed of a plurality of translated language languages are stored.

Fig. 10 shows storage means for storing speech information in a plurality of translated language languages (here, English and French) corresponding to the pre-translation language (here, Japanese) and character information consisting of a plurality of translated language languages. FIG. 3 shows a schematic diagram of data stored in 10.

Fig. 11 shows a case where there are multiple pre-translation languages, and multiple pre-translation languages (for example, English and Japanese) corresponding to pre-translation language 1 (for example, Japanese) and pre-translation language 2 (for example, English). FIG. 3 is a diagram showing a storage area of a storage unit 10 in which different voice information and character information including a plurality of translated language languages are stored.

Fig. 12 shows speech information composed of multiple translated language languages (here, English and Japanese) corresponding to multiple pre-translation languages (here, Japanese and English) displayed in the balloon character information field. FIG. 4 is a schematic diagram of data stored in a storage area of a storage unit 10 in which character information composed of a translated language is stored.

Fig. 13 is a diagram showing an example of translating a cartoon whose English is the pre-translation language into Japanese, which is the post-translation language, and automatically reproducing and displaying the voice and characters in the selected speech character information field. is there.

FIG. 14 is a flowchart showing an example of a basic processing flow in the second embodiment.

FIG. 15 is a basic system configuration diagram of the second embodiment. FIG. 16 is a flowchart showing a basic processing flow of the second embodiment.

FIG. 17 is a flowchart showing a basic processing flow of the second embodiment.

FIG. 18 is a flowchart showing an example of processing for storing data in the image information storage means, the voice information storage means, and the character information storage means.

Fig. 19 shows information storage that can be used in computer devices and personal digital assistants. 5 is a flowchart illustrating an example of a process using a medium.

FIG. 20 is a diagram showing a typical example of various forms shown as “balloon-shaped figures” in this specification. BEST MODE FOR CARRYING OUT THE INVENTION

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(First embodiment)

FIG. 1 is a system configuration diagram showing a basic configuration in a first embodiment of a system according to the present invention.

In the first embodiment, an information processing system using a balloon-shaped figure according to the present invention is a user terminal such as a computer or a portable information terminal having an input unit, a control unit, a display unit, an output unit, and a storage unit. Is a system in which information processing is performed by user operations.

In FIG. 1, reference numeral 1 denotes a user terminal. The user terminal 1 includes a computer terminal such as a personal computer, a wireless communication terminal such as a mobile phone or a PHS, a portable information terminal having these functions, and various other computer terminals. Further, for example, a so-called information home appliance such as an Internet TV may be used.

Next, reference numerals 10a and 10b denote storage means provided in the user terminal. The storage means 10a includes an external storage device such as a CPU, a memory, and a hard disk.

In the first embodiment, the storage means includes a storage means provided in the user terminal, or a storage means connected to the user terminal via a computer network. It includes a storage device connected to the user terminal via the network. Here, the computer network broadly includes the Internet, a network form connected by a dedicated line, a corporate LAN, a corporate LAN, a WAN, and the like. In addition, the form of the communication line used here widely includes the form of wired communication and wireless communication, such as satellite communication and Bluetooth. And the like.

The storage unit 10b is a storage unit such as a CD-ROM drive, a DVD drive, or an information storage medium that is read via another input unit. The storage means 10b includes, for example, CD-ROM, DVD FD (floppy disk), MO, SmartMedia (trademark), Memory Stick (trademark), electronic book, and any other information storage medium. The storage means included in the user terminal for realizing the system of the present invention include these storage means 10b.

The storage means 10a usually stores an operating system (OS), various application programs, various data files, and the like, but is stored in a storage medium 10b such as a CD-ROM or DVD. It is also possible to take a form in which the stored image information, character information, programs as information processing means and the like are stored in the storage means 10a.

Next, reference numeral 11 denotes input means such as a keyboard / mouse, and reference numeral 12 denotes data control, data transmission / reception, hardware control, software control, and the like using a CPU, a bus, and other OS and application programs. The control means for this is shown. Reference numeral 14 denotes an output means for transmitting and receiving data via a network and outputting data such as voice.

Next, the storage means 10a or 10b (hereinafter, simply referred to as "storage means 10" as necessary) will be described with reference to FIG.

The storage means 10 includes at least image information storage means for storing image information, voice information storage means for storing voice information, and information processing means for displaying character information and voice information on the user terminal. Have been.

Also, if necessary, a character information storage means for storing character information is provided, and if provided, voice information and character information can be automatically linked to be displayed and output. Is preferred.

In FIG. 2, reference numeral 15 denotes image information storage means, 16 denotes audio information storage means, 17 denotes character information storage means, and 18 denotes information processing means. The image information storage means 15, the voice information storage means 16, and the character information storage means 17 are storage areas provided in a storage means having a hierarchical structure such as a folder in the storage means, and a database structure such as a relational database. It may be a storage area having '

The image information storage means 15 is displayed on the user terminal such as an image file inserted in an HTML file or an XML file, an image file, a file such as a slide show including a plurality of image files, or the like. Image file is stored.

An example of an image file is shown in FIGS. 3 and 4.

FIG. 3 shows an image file storing a menu screen to be displayed on a display means provided in the user terminal 1 in order to select and display / reproduce a plurality of content data files. In the figure, an image file including a balloon-shaped figure described later is shown.

Here, the image file can include image files of various data formats such as JPEG, GIFF, PNG, TIFF, BMP, PICT, and the like, and image files of its own format.

In addition, the menu screen and the playback screen for arranging the content data file are displayed on the user terminal in a format such as an HTML file or an XML file, or a file such as a slideshow including a plurality of image files. It can be stored as an image file to be displayed, or in a file format using a unique playback program, and can be used as an image file that is inserted and displayed in the file.

The advantages of using HTML files and XML files are that the number of parts restricted by the application program and the user terminal model is small, and the storage means connected to the Internet and accessed by the system of the present invention. That the user can access. Also X

When ML or the like is used, playback control data is extracted using various tag information, or extracted data or data is extracted according to the type of computer or portable information terminal such as a mobile phone. Define files Can be

Next, as another desirable example, the image file may be a Macromedia (trademark) F1 ash (trademark) animation II, a Shockwave (trademark) movie, or other popular data files. This is a method that uses a data format. The advantage of using these is that, although plug-in software is required, it is widely spread and readily available. Also, since multiple pages can be created, pages (frames) can be moved as if turning pages. In addition, it is possible to set automatic playback and perform heterogeneous control such as repeated playback. The system of the present invention may be provided in the storage means connected to the Internet for access, so that the user can access the storage means. Still another advantage is that a text field for entering text information can be provided in the image file, and the text information entered in the text field can receive link information to another file. it can. Also, objects such as image information can be inserted into the image file, and the object can receive link information to other files.

Next, these image files will be described with reference to FIGS. 3 and 5. FIG.

In the image file, one or a plurality of balloon-like figures surrounded by a linear figure having an arbitrary shape and thickness are arranged at an arbitrary place in the image.

The balloon-shaped figure is indicated by reference numeral 19 in FIG. In this example, a plurality of balloon-shaped figures are arranged in one image file.

According to the invention set forth in claim 6, the user terminal can select and display a desired page out of a plurality of pages, and in the lower right of FIG. 5, proceed to the page first. There are control buttons for navigating, going back, and returning to the first page. The control button is image information or an object inserted in the image file, and is a predetermined other. The link information to the page is embedded.

Further, in the invention according to claim 7, the data file stored in the storage means is a content such as a comic which has a story, and a user terminal has a plurality of pages. 4 and 5 show an example of an image file displayed in such a form.

In the data file displayed on the user terminal, one or a plurality of character information display fields for displaying character information are arranged.

The character information fields are denoted by reference numerals 20a and 20b (hereinafter simply referred to as "20" as necessary) in FIG. The character information field 20 can be set in advance so that predetermined character information is displayed, but the predetermined character information stored in the character information storage means is stored in accordance with the operation of the user. A field or window that can be displayed.

The character information field 20 is a field for displaying image information in which an image information file is data including character information in the invention described in claim 3. In other words, in this case, the data file stored in the character information storage means 17 is a data file in the form of an image, and the data that is the basis of the image information contains the character information. In other words, characters are stored in an image format.

Next, the balloon-shaped figure will be described.

In the invention according to claim 1, the balloon-shaped figure arranged in the image information includes image information including a balloon character on a predetermined coordinate in the image information indicated by the balloon-shaped figure. A balloon character information field for displaying is displayed. That is, in this case, the image information displayed in the balloon character information field is a data file in an image format.

The image information including the balloon character is stored in the audio information storage unit. In addition to the association with the predetermined audio information file, the association with the predetermined character information file stored in the character information storage means can be set in advance. These may be obtained by embedding hyperlink information in the image information displayed in the balloon character information field 19.

Note that the image information displayed in the balloon character information field 19 can take a form in which, for example, an image file of a balloon-shaped figure is overlaid on the image information of the entire page in a single layer format. In addition, it is possible to adopt a form of inserting and arranging as an object in image information of the entire page. It can also be realized by setting a balloon-shaped area in the image information of the entire page using the X-axis coordinate value and the Y-axis coordinate value and embedding hyperlink information in the set area (clickable. map).

In the invention according to claim 2, in the balloon-shaped figure arranged in the image information, balloon character information is displayed on predetermined coordinates in the image information represented by the balloon-shaped figure. A balloon character information field is arranged. That is, in this case, the balloon character information field is a field for displaying character information such as text arranged in the image information, and the character information displayed in the balloon character information field is a character information format data. File.

In this case as well, the balloon character information is associated with the predetermined voice information file stored in the voice information storage unit, and the predetermined character information stored in the character information storage unit. It is possible to set the association with the information file in advance. In these, hyperlink information may be embedded in the character information displayed in the balloon character information field 19.

In FIG. 3, the hyperlink to the speech information file stored in the speech information storage means 16 set from the speech character information field 19 and the character information storage means set from the speech character information field 19 17 Linking the hyperlink to the character information file stored in 7 The section is schematically shown. If the image file is a comic file such as a comic file, one page (image file) is provided with a balloon character information field indicating one or more dialogues, and audio information corresponding to each dialogue is provided. And hyperlinks to textual information.

The voice information stored in the voice information storage means 16 includes a WAV file, an MP3 file, an RM file, and various other audio files in data formats.

The character information stored in the character information storage means is a text format or another character information file.

FIG. 6 shows an example of the relationship between an image file, which is each page of content such as comics, stored in the image information storage means 15, and a balloon character information field arranged in each image file. This is shown using a data schematic diagram.

In addition, an example of the relationship between the character information file and each balloon character information field arranged on each page of the content such as comics stored in the character information storage unit 16 will be described using a data schematic diagram. Is shown.

Further, an example of the relationship between each speech character information field arranged on each page of the content such as a manga and the like stored in the audio information storage means 17 and the audio information file will be described using a data schematic diagram. Is shown.

Next, the information processing means will be described.

The information processing means 18 allows the user to operate the image information, audio information, text information, and the HTML file that constitutes the menu screen and the like described above using the input means of the user terminal. This is an application program for performing display and output according to the operation.

Application programs include browser software, plug-in software for playing F1ash, etc., and programs for playing proprietary data files. Install these programs on storage means 10a such as a hard disk, or D— Provided in storage means 10b such as ROM or DVD, preferably provided with an autorun (Autorun) program in storage means 10b, and automatically stored when storage means 10b is loaded into the user terminal. It is possible to take the form of starting up.

Next, a basic processing flow in the first embodiment of the present invention will be described. Note that the processing flow shown here is an example, and the present invention is not limited to this.

FIG. 7 is a flowchart showing a first embodiment of the present invention. First, the storage means 10 b is loaded in the user terminal 1. When installing the information processing means, which is a program, in the storage means 10a, start the installation program and perform the installation according to the instructions. Further, the data files stored in the image information storage means 15, the voice information storage means 16, and the character information storage means 17 can be stored in the storage means 10a. After the installation, the information processing means 18 is started (S102).

If not installed, the information processing means 18 is started by the user's operation, or automatically started by the automatic reproduction program (S103), and the information processing means 18 is started.

Next, a menu screen such as which comic story to select is displayed (S104).

If there is more than one co-down Teng Tsu data file, such as a plurality of cartoon of the scan Tori one the stored data file (S 1 0 5), make a selection of content data Tafuairu (S 1 ₀ 6) 0 Conte N'data If only one file is stored, go to the next step.

Next, an image file such as the first page is displayed. Before that, an opening screen such as a cover, title, and copyright notice may be displayed.

The image file of the first page is displayed. The image to be displayed is an image file in which the balloon character information field 19 as shown in FIG. 4 is arranged as described above. Also, speaking of Fig. 4, the right side of the screen Is provided with a character information display field 20.

Next, the user can select one of the balloon character information fields 19 from the balloon character information fields arranged in one or a plurality of displayed balloon-like figures (S108) ). The selection of the balloon character information field is a user's action such as placing a cursor in the balloon character information field to be displayed or placing a cursor and clicking.

When the user makes a selection, the control means 12 detects the operation of the user, that is, the selection of the hyperlink information (S109), and transmits it to the information processing means 18 (S111). 0).

When the information processing means detects the user's operation (S111), audio information and character information corresponding to the balloon character information field are extracted and reproduced or displayed and output. The user can set the audio playback speed, number of playbacks, and other playback control information and store them as a setting file.In such a case, determine whether a playback control setting file exists. (S112), read the setting file (S113), extract voice information and character information, and move pages (extract and display image files) according to the data defined in the setting file. I do. In addition, when the setting is made so that only the audio is reproduced without displaying the character information, the display output of the character information is not performed.

In particular, if the reproduction control is not set, the process proceeds to the next step.

In addition, the character information to be displayed is not always stored in the character information storage means in advance corresponding to all the balloon character information, or there is no character information at all. The setting may be such that it is displayed in the character information display field included in the image file. In this case, it is determined whether there is character information associated with the balloon character information (S114).

Next, the link added to the selected balloon character information field The voice information indicating the location in the information is extracted (S115), and the character information indicating the location in the link information added to the selected speech character information field is extracted (S116). The extracted voice information is output as a voice by the output means 14 and reproduced (S 1 17), and the extracted text information is arranged in the image file displayed by the display means 13. Displayed in display field 20 (S1 1 8) ₀

Next, when there are a plurality of pages (image files), it is determined whether or not to move the page by user operation or selection, or by automatic reproduction (S119), and the moving is performed. If so, the destination page is displayed (S107), and the above processing is repeated.

If there are multiple balloon character information fields in the same page, return to S108 before moving the page, select another balloon character information field, and repeat the above processing.

When the reproduction is terminated, the reproduction is terminated by the user's operation or selection, or by the automatic reproduction (S120). If not, you can return to the menu screen or navigate to another page. When the processing is terminated, the information processing means 18 which is a program is usually terminated. (Second embodiment)

Next, a second embodiment of the present invention will be described.

A second embodiment of the present invention is a language translation information processing system using a balloon-shaped figure using the above system.

With the system of the present invention, the translation languages that can be played and displayed are: English, French, German, Italian, Spanish, Dutch, Russian if the Japanese language is the pre-translation language. Any language that can be translated from Japanese into a foreign language other than Japanese, such as Chinese, Chinese, Korean, and Swahili, may be used. Further, these foreign languages may be the pre-translation languages, and the post-translation languages may be Japanese. Furthermore, in order to be able to replace sentences and words in a language with corresponding to other languages, Japanese standard language and Kansai dialect, Nagoya dialect, Ainu language or Oki ref language, or ancient, medieval, early modern, etc. Language of the present It is used in virtual languages that do not actually exist, or in languages used by young people or the elderly, indulgent languages and official languages, upper-class languages, buzzwords, animations such as comics and TV programs, and drama. Languages that can replace various language sentences and words, such as distinctive phrases, heroes such as cartoon characters and dramas, and distinctive phrases of celebrities, in correspondence with other languages. It can also be used as a pre- or post-translation language.

In the present embodiment, the hardware configuration of the basic system and the data configuration of the data stored in the storage unit 10 are the same as in the first embodiment described above, and thus will not be described.

In this embodiment, the speech information stored in the speech information storage means is post-translation language speech information corresponding to pre-translation language character information included in the balloon character information field.

The character information stored in the character information storage means is post-translation language character information corresponding to the pre-translation language character information included in the balloon character information field.

FIG. 8 is a diagram showing a storage area of the storage means. The image information storage means 15 stores image information, the voice information storage means 16 stores voice information, and character information is stored as necessary. A character information storage means 17 for storing and an information processing means 18 for displaying character information and voice information on a user terminal are provided.

In the invention described in claim 4, the image information, the image file inserted in the HTML file, the XML file, or the like stored in the image information storage means, a slide show including a plurality of image files, or the like is provided. In an image file displayed on a user terminal such as a file, character information or image information using character information as image data having an arbitrary shape and size is arranged at an arbitrary location in the image. Or, a plurality of character information display fields are arranged.

The character information display field includes the voice information storage unit. It is possible to set in advance the association with the predetermined character information file stored in the character information storage means, in addition to the association with the predetermined audio information file stored in the file. Yes, the character information field is not necessarily limited to the "balloon-shaped figure" as in the other embodiments, but may be variously arranged in a page and provided in a predetermined area set in advance. Including figures and character information. In addition, even if it is not necessarily a tangible figure, for example, a form in which a predetermined area in an image file is associated with a voice information file or a character information file is included.

The voice information stored in the voice information storage means is post-translation language voice information corresponding to the pre-translation language character information included in the character information field, and is stored in the character information storage means. The character information to be translated is language character information after translation corresponding to the language character information before translation included in the character information field, and the cursor is placed in the character information field indicated by the language before translation. Information processing means provided in the storage means for detecting the user's action in response to the user's action such as clicking on the link, and the location of the link information added to the character information field. A process of transmitting voice information, which is the above-mentioned translated language, to a user terminal, outputting the voice information by voice playback means provided in the user terminal, and playing back the voice information; If the association with the character information file has been set in advance, the location is indicated in the link information added to the character information display field, and the character information in the translated language is transmitted to the user terminal. And a process of displaying by a display means of the user terminal.

Next, a language translation information processing system using a balloon-shaped figure will be described in writing.

It should be noted that, throughout the present specification of the present invention, “balloon-shaped figures”

Starting with the comic speech bubbles shown in Fig. 4, various shapes with various shapes and leader lines, circles and ellipses, cloud shapes, and irregular shapes These examples of FIG. 20 are shown, including species forms.

The balloon character information field provided in the balloon upper figure provided in the image file stored in the image information storage means is set so that character information consisting of the language before translation is displayed. For example, in Fig. 4 and Fig. 5, the comics in Japanese as the pre-translation language are translated into English as the post-translation language, and the voice and characters in the selected speech character information field are automatically reproduced and displayed. An example is shown.

In this embodiment, the basic processing flow is the same as in the first embodiment.

To explain with reference to FIG. 7, first, the user terminal 1 is loaded with the storage means 10b. When installing the information processing means, which is a program, in the storage means 10a, start the installation program and perform the installation according to the instructions. Further, the data files stored in the image information storage means 15, the voice information storage means 16, and the character information storage means 17 can be stored in the storage means 10a. After the installation, the information processing means 18 is started (S102). '

Next, a menu screen such as which comic story to select is displayed (S104). For example, in FIG. 3, a menu screen is displayed, and a story such as “〇〇の卷” is selected from among a plurality of stories. If there are a plurality of content data files such as a data file storing a plurality of comic stories (S105), a content file is selected (S106). If only one content file is stored, proceed to the next step.

Next, an image file such as the first page is displayed.

The image file of the first page is displayed, but there is an image to be displayed as described above. Is an image file in which is arranged. In FIG. 4, a character information display field 20 is arranged on the right side of the screen.

An image file constituting the entire page as shown in FIG. 4 is an image file stored in the image information storage means in FIG. 8, and one or more image files are included in the image file (page). A balloon character information field is set for the balloon-shaped figure. For example, in Fig. 4, there is a balloon character information field for each of the pre-translation languages (Japanese in this example) such as "Hatsutsu" · "Is it really true?" The voice information of the language (English in this example) is stored in the voice information storage means 16 in FIG. 8, and the corresponding text information of the translated language (English in this example) is stored in the voice information storage means 1 in FIG. 7 is stored.

Next, the user can select one of the balloon character information fields 19 from the balloon character information fields arranged in one or a plurality of displayed balloon-shaped figures (S 1 0 8). When the user makes a selection, such as placing the cursor in the balloon character information field or placing the cursor and clicking, the user's action, that is, the selection of the hyperlink information, is controlled. The means 12 detects (S109) and transmits it to the information processing means 18 (S110).

When the information processing means detects the operation of the user (S111), the speech information and the character information of the translated language corresponding to the balloon character information field are extracted and reproduced or displayed and output. Will be If the user can set the audio playback speed, number of playbacks, and other playback control information in advance and store them as a setting file, determine whether there is a playback control setting file. (S112), read the setting file (S113), extract voice information and character information, and move pages (extract and display image files) according to the data defined in the setting file. ) I do. Also, if the setting is made so that only the voice is played without displaying the character information, the display and output of the character information are not performed. In particular, if the reproduction control is not set, the process proceeds to the next step.

Next, the voice information indicating the location in the link information added to the selected balloon character information field is extracted (S115), and the audio information indicating the location is added to the link information added to the selected balloon character information field. The character information indicated by is extracted (S116). In Fig. 5, the balloon character information field 19 indicates that "Hatsutsu" · "Is it really true?" Is selected, and the extracted speech information, "Is it possibly", in the translated language (English) The sound "true thai: ..." is output and played back by the output means 14 (S117), and the extracted character information "Is it possibly true that ..." is displayed on the display means 13 The character information is displayed in the character information display field 20 arranged in the file (S118).

Next, when there are a plurality of pages (image files), it is determined whether or not to move the page by user operation or selection, or by automatic reproduction (S119), and the movement is performed. If so, the destination page is displayed (S107), and the above processing is repeated. .

When the reproduction is ended, the reproduction is ended by a user operation or selection, or by automatic reproduction (S120). If not, you can return to the menu screen or navigate to another page.

Next, another embodiment of a language translation information processing system using a balloon-shaped figure according to a second embodiment of the present invention will be described. In the invention according to claim 6, the post-translation voice information stored in the voice information storage means includes a plurality of speech character information corresponding to pre-translation language character information included in the balloon character information field. Contains post-translation language audio information consisting of the post-translation language.

The post-translation character information stored in the character information storage means includes post-translation language characters composed of a plurality of post-translation languages corresponding to the pre-translation language character information included in the balloon character information field. In other words, in the case of the comics in Fig. 5, if speech in the pre-translation language (Japanese) is displayed in the balloon character information field, the audio is played back in the translated language. The information and the character information displayed in the translated language are in a form that allows selection of, for example, English or French. Alternatively, it can be selected from English, French, Spanish, German, Chinese, and Korean.

Fig. 9 shows the storage means for storing speech information consisting of a plurality of translated language languages (here, English and French) corresponding to the pre-translation language (here, Japanese) and character information consisting of a plurality of translated language languages. This shows a storage area of 0. Fig. 10 shows the storage of speech information in a plurality of translated language languages (here, English and French) corresponding to the pre-translation language (here, Japanese) and character information consisting of a plurality of translated language languages. FIG. 3 shows a schematic diagram of data stored in means 10.

First, the user performs a process of selecting which of a plurality of translated languages to translate the pre-translation language character information into. For example, if there are English and French translation languages, select English.

Next, use the cursor in the balloon character information displayed in the balloon character information field, which is indicated in the pre-translation language placed in the displayed balloon-shaped figure, or place the cursor and click on it. The information processing means provided in the storage means performs a process of detecting the user's action according to the action of the user.

Next, the balloon-shaped character information arranged in the balloon-shaped figure The voice information in the selected translated language (selected English), which indicates the location in the link information added to the report field, is transmitted to the user terminal, and the voice reproduction means provided in the user terminal transmits the voice information. Output and play.

Further, when the association with the predetermined character information file is set in advance, the location is indicated in the character information display field in the link information added to the balloon character information field. Then, the character information in the translated language (selected English) is transmitted to the user terminal and displayed by the display means provided in the user terminal.

Further, another embodiment of a language translation information processing system using a balloon-shaped figure according to a second embodiment of the present invention will be described.

In the invention described in claim 7, the balloon character information displayed in the balloon character information field, which is indicated by the pre-translation language arranged in the displayed balloon-shaped figure, includes a plurality of pre-translation languages. Balloon character information is prepared in advance, and an image file including a balloon-shaped figure is stored in the image information storage unit.

For example, in Fig. 4 and Fig. 5, the comics in Japanese as the pre-translation language are translated into English as the post-translation language, and the voice and characters of the selected speech balloon character information field are automatically reproduced and displayed. An example is shown. According to the present embodiment, separately from this, a cartoon in which the language before translation is, for example, English is translated into Japanese, which is the language after translation, and the voice and characters of the selected speech character information field are translated. For automatic playback and display, an image file whose language before translation is English can be displayed in the balloon character information field. Also, for example, Japanese is used as the post-translation language, and Japanese speech information and Japanese character information corresponding to English in the balloon character information field are stored.

By adopting such a form, it is possible to realize a Japanese-to-English language translation and an English-to-Japanese translation by using a balloon-shaped figure. Figure 11 shows that there are multiple pre-translation languages, and that multiple pre-translation languages corresponding to pre-translation language 1 (for example, Japanese) and pre-translation language 2 (for example, English) For example, it shows a storage area of the storage means 10 in which audio information composed of, for example, English and Japanese) and character information composed of a plurality of translated language languages are stored. Also, Fig. 12 shows the speech information composed of multiple translated language languages (here, English and Japanese) corresponding to multiple pre-translation languages (here, Japanese and English) displayed in the balloon character information field. FIG. 3 shows a schematic diagram of data stored in a storage area of a storage unit 10 in which character information composed of the translated language is stored.

Fig. 13 shows an example of translating a manga whose original language is English into English, which is the translated language, and automatically reproducing and displaying the voice and characters in the selected balloon character information field. It is shown.

In the case of this embodiment, when a Japanese-English translation is selected on a menu screen or the like, an image file as shown in FIGS. 4 and 5 is displayed. When an English-Japanese translation is selected, FIG. 13 is displayed. Is displayed. FIG. 14 is a flowchart showing an example of a basic processing flow in the present embodiment.

If there are a plurality of pre-translation languages (S207), the user performs a process of selecting from which of the plurality of pre-translation languages the pre-translation language is to be translated (S207). 0 8).

In addition, if there are two types of translation: English to Japanese and Japanese to English, if you select English as the pre-translation language, the post-translation language can be automatically set to Japanese, while English-Japanese translation, English-Japanese There may be multiple translated languages, such as Japanese-English translation, Japanese-German translation, German-Japanese translation, German-English translation. In this case (S209), the user performs a process of selecting a language to be translated from a translated language or a plurality of translated languages (S210).

Next, a process of displaying an image file including a balloon character information field composed of the selected pre-translation language (S211), and a process of displaying the pre-translation language, which is arranged in the displayed balloon-shaped figure, According to the user's action such as placing a cursor on the balloon character information displayed in the balloon character information field, or placing and clicking a cursor (S2122), The information processing means provided in the storage means performs the process of detecting the user's operation (S213, S214).

Next, the speech information, which is the language after translation, whose location is indicated by the link information added to the balloon-shaped character information field, is transmitted to the user terminal, and is output and reproduced by the audio reproduction means provided in the user terminal. (S22 l) _c Also, if the association with the predetermined character information file has been set in advance, the character information display field contains the link information added to the balloon character information field. Is transmitted to the user terminal, and is displayed on the display means provided in the user terminal (S222).

(Third embodiment)

Next, a third embodiment of the present invention will be described.

In the present embodiment, the storage means is a server system including a WEB server, an application server, a database server, and the like connected from a user terminal via a computer network, and is a computer represented by the Internet. In this mode, access is made from the user terminal 1 via a network.

The system according to the present embodiment is generally realized in a system having a Web server, an application server, a database server, and the like on a network to which a computer, a portable information terminal, and the like are connected.

However, here, as a basic system configuration, Fig. 1 shows an example in which these server systems are installed in the same system, but if the application server is located in a separate location, Or, even if the database server exists in a separate location, these locations can be specified and specified by a URL or the like. Therefore, even in such a system configuration, the system of the present invention is used. Is feasible.

FIG. 15 shows a basic system configuration diagram of the present embodiment. In FIG. 15, reference numeral 1 denotes a user terminal, reference numeral 25 denotes a server system, and reference numeral 30 denotes a computer network represented by the Internet. The server system 25 includes image information storage means 15, audio information storage means 16, and character information storage means 17.

Here, user terminals for connecting to a network include computer terminals such as personal computers, wireless communication terminals such as mobile phones and PHS, portable information terminals incorporating these functions, and other various types of terminals. Kinds of computer terminals are included. Also, for example, a so-called information home appliance such as an Internet TV may be used.

Furthermore, the networks to which these are connected include wired and wireless communication lines.

Note that image information storage means for storing image information, voice information storage means for storing voice information associated with the text information, text information storage means for storing text information, and text information and voice information. The configuration of the information processing means to be transmitted to the user terminal for display is common to the various modes described above.

The user terminal 1 includes a computer terminal such as a personal computer, a wireless communication terminal such as a mobile phone or a PHS, a portable information terminal incorporating these functions, and various other computer terminals. Also, for example, a so-called information home appliance such as the Internet TV may be used.

Next, reference numerals indicate storage means provided in the user terminal. The storage means 10a includes an external storage device such as a CPU, a memory, and a hard disk. The storage means 10 usually stores an operation system (OS), various application programs, various data files, and the like.

Next, reference numeral 11 denotes input means such as a keyboard / mouse, and 12 denotes data control, data transmission / reception, hardware control, software control, and the like using a CPU, a path, and other OS and application programs. 2 shows control means for performing the above. Reference numeral 14 denotes an output means for transmitting and receiving data via a network and outputting data such as voice.

As described in the first embodiment, in the present invention, the menu screen for browsing the content data file, the playback screen for arranging the content data file, and the like are in the form of an HTML file, an XML file, or the like. Image file displayed on the user terminal such as a slide show file containing the same image file, or a file format using a unique playback program, or an image file inserted and displayed in the file You can also. Further, as an example of a desirable form, F1ash (trademark) and Shocwav (trademark) of Macromedia Corporation (trademark) have been described.

When browsing through the Intane' bets like in the present embodiment, usually the respective storage means 1 5 of the, 1 6, 1 7 the stored information or the like, web sites stored in the We _b server, etc. HTML file that is displayed on the web site. HTML An HTML file that describes data extracted by searching, etc. by linking to an XML file or the like, or from the information stored in the storage means 15, 16, 17. Etc. are generated and displayed on the screen of the accessing customer terminal.

For example, if the information to be displayed is an image file such as a gif file, a link to the image file is described in a file such as an HTML file on a website.

In addition to HTML files, it is also possible to use XHTML files, XML files, SGML files, or C-HTML files for mobile terminals.

When the user accesses this web site, the user's computer terminal requests the transmission of the web page file (HTML file, etc.), and at the same time, the link is described in the file. It also requests transmission of the image file.

By receiving these files, for example image files For example, the web page accessed by the user displays an HTML file or the like on the user's computer terminal screen via a browser software, and the image file is displayed on a part of the page. For example, if the file is an audio file, the web page accessed by the user is displayed on the user's computer terminal screen as an HTML file via browser software, and the audio page is displayed on a part of the page. The hyperlink to the file is described and displayed. Also, for example, the file to be transmitted is displayed in a part of a plurality of divided frames, such as a frame for displaying a plurality of HTML files by dividing one screen. Yeah.

Also, customers who can access and browse the system of the present invention are not limited to an unspecified number of users, and only registered users can access and search and browse information. Is also good. For example, it is possible to access the web site, enter the ID and password issued for each customer, etc., and make the access possible. Next, a basic processing flow of the present embodiment will be described.

FIG. 16 and FIG. 17 are flowcharts showing an example of the basic processing of the present embodiment.

First, the user terminal 1 is connected to a computer network such as the Internet (S300), and accesses the server system 25 (S301). As described above, the server system is provided with data files such as image information storage means, audio information storage means, and character information storage means, and programs such as information processing means. Is provided. In general, files such as HTM files and XML files provided on the WEB server that are displayed on the WEB are transmitted to the user terminal, and a menu screen is displayed.

When using the system of the present invention, in the case of performing a billing process, the use is permitted on condition that these are performed normally. Authentication processing, billing processing, etc. are performed (S302), and downloading is performed. Or browsing can be selected (S303).

In the case of downloading (S304), after downloading and storing in the storage means 10a provided in the user terminal, as described in the first embodiment and the second embodiment, Used at the terminal.

When browsing on the Internet or the like, a browsing menu is displayed (S305). When there are multiple content data files (here, translation of comics is taken as an example) (S306) ), And which one to browse (S307). In addition, if it is possible to select from multiple languages such as English-Japanese translation and Japanese-English translation, these choices are made (S308 to S311). The information stored in the server system 20 is as follows, for example, using a system that can translate Japanese into English using Japanese manga as a material.

The image information storage means 15 includes character information such as dialogue of Japanese manga arranged in the balloon character information field of the pre-translation language for each frame of each manga page, each dialogue, etc. Image information is stored for each page of the comic.

The translated language character information storage means 17 stores, in correspondence with the above-mentioned dialogues, English translations of Japanese manga dialogues, etc., and textual information such as texts, etc. of each manga page, It is stored for each line or the like. It is also possible to store information such as points and notes for translation and other similar expressions.

The translated speech information database 16 stores audio files of English translations such as dialogues of Japanese comics, and text information such as text for each frame on each page of the comics, each speech, etc. You can also do that.

Here, the image data storage means including the character information of the language before translation, the character information storage means of the language after translation, and the voice information storage means of the language after translation are tentatively stored. The database structure can take various forms as long as it can extract data. These can be realized by adopting a hierarchical structure of folders or a relational database. Also, the translation of comics is an example here, and can take various forms described in the first embodiment and the second embodiment described above.

Also, in the first embodiment, the second embodiment, or the third embodiment described here, the information stored in each of the storage means 15, 16, and 17 is a claim. Like the invention described in the range 17, it is possible to store by the following means, and it is preferable to use the present invention in order to store a lot of contents.

The storage unit including the image information storage unit, the voice information storage unit, the information processing unit, and the character information storage unit includes a data capturing unit that captures data such as a print having image information and balloon character information as image information. Be provided. The data capturing means is, specifically, a scanner device or the like, and the read data is stored as an image file.

Here, comics (comics) are used as the original material that becomes the content such as printed matter having image information and character information.

First, the writer who publishes the manga and the author who wrote the manga or Z obtain the rights such as copyrights for use in the system of the present invention.

Next, the comics for which the right has been acquired are stored by scanning or the like in accordance with the database or the storage area having a hierarchical structure.

The first data processing means for storing the processed image information in the image information storage means sets a balloon character information field in a balloon-shaped graphic portion such as each speech included in the captured image information. The character information to be arranged is input and stored as character information data to be arranged in the balloon character information field. Alternatively, it is stored as image information data including character information.

The character information (for example, Japanese speech in the pre-translation language) placed in the balloon character information field can be converted into character information by reading image information with OCR software. Next, the second data processing means inputs the character information and voice information associated with the character information included in the character information field, converts them to the translated language if necessary, converts the The information is stored in the information storage means. The second data processing means includes a process of associating each balloon character information field included in the image information stored in the image information storage device, a process of inputting character information or voice information, and the above-described hierarchical structure or database. This is a program that performs processing for storing data files in a storage area such as a structure.

The process of inputting character information or voice information is performed by automatic translation software or the like based on the character information read by OCR or the input character information from the character part of the image information read by the data capture unit. It is possible to adopt a process of storing the translated information (for example, English) as character information (text data or the like) in the character information storage means and storing the information in the voice information storage means using a reading software or the like.

Then, the files stored in the character information storage means, the image information storage means, and the audio information storage means are clicked with a mouse or the like on the user's website using a CGI program or another program. In response to commands (requests), the system can automatically generate files such as HTML files, XML files, Flash (trademark) movie files, etc., and create a system that obtains the desired language translation results.

FIG. 18 is a flowchart showing an example of the above processing.

The data created as described above is stored in a server system or the like, or is stored in an information storage medium such as a CD-ROM, and is loaded into a changer. It can be accessed and used from. Alternatively, an information storage medium such as the CD-ROM described above can be obtained by purchase or the like, and the user can use it on a terminal.

Next, returning to FIG. 17, returning to the processing flow of the present embodiment, the description will be made. In S312, the image file of the comic page is displayed. This is the screen shown in Fig. 4, for example, with a manga displayed on the left side of the screen. This speech balloon character information field of the comic, that is, the Japanese part of the dialogue is clicked with the mouse (S313), and the dialogue part of the clicked comic is displayed on the right character information field 20a. To be displayed. In other words, the control means of the user terminal detects the user's action by clicking the Japanese portion of the dialogue to be translated into English on the left side of the screen with a mouse (S31). 4), the information is transmitted through the computer network, and the application described later is started.

The application receives a file transmission request by a command such as a user's click (S315), judges the speech character information selected (clicked, etc.) by the user (S316), and With reference to each storage means 15, 16, 16 and 17, search and extract the information corresponding to the transmission request from the stored information, and include such information or hyperlink information to the information Combine HTML files etc. In the above case, the application stored in the application server is used for the given condition, that is, the condition of the corresponding page, frame, dialogue, etc. In accordance with instruction information such as a request, access is made to step 10 in the memory s, data matching the conditions and the like are combined into, for example, an HTML format file, and transmitted to the user terminal.

Then, the English-translated dialog is displayed on the right side of the screen.

In addition, sentences translated into English are played back as voices like narration via a browser plug-in software for playing back the voice information file of the hyperlink.

These processes are realized by, for example, the following application programs.

A simple program is realized by using a CGI (Common Gateway Interface) program using Perl or C language. However, if the number of users, access frequency, database capacity, etc. become large, V It is desirable to construct

Today, server software that generates the contents of html files is also being implemented. Such software has a function to query the database and the server in advance, and if the content of the database query using SQL is embedded in an HTML file with its own extension, the server sends back the page. At this time, replace the contents of the HTML file with the contents of the result of the inquiry. Because the contents of the database query and the appearance of the displayed page can be described in a single file, service development is easier than with the CGI program.

Also, a script environment operating on a server can be used. Microsoft's ASP (Active Server Age) is a server-side script (written in VB Script, JS cript, etc.) and an Active X control (including JaVaplet). It is a platform for executing

Although described in detail above, the present invention is not limited to the embodiment described above, and it goes without saying that various modifications can be made within the scope of the invention described in the claims. .

Further, in the present specification, means does not necessarily mean physical means, but also includes a case where the function of each means is realized by software.

Further, the function of one means or step may be realized by two or more physical means, and the functions of two or more means or steps may be realized by one physical means.

Further, as another embodiment of the present invention, as in the invention according to claim 15, the system further includes an authentication unit for logging in from a user terminal, and an access number of the user. A usage information storage unit capable of storing the number of uses of the translation system, other information, information on the user, and the like, and calculating a usage fee by referring to the information stored in the usage information storage unit. Means for calculating usage fees, and each of the above-mentioned number of accesses, etc. There is a system that can calculate usage charges and manage charges according to species usage information.

Here, the information on the user may be a method in which the user inputs an e-mail address or the like, or may be information automatically acquired by the system of the present invention.

Alternatively, the information of the user may be obtained in advance through member registration on the Web site, and the system of the present invention may perform authentication by inputting an ID and a password.

Usage information of the access number or the like of the user, also _c are stored for each individual user, the unit price of the usage fee, the access execution of the translation, the voice acquisition, there have the use time unit and usage information amount unit It can be set for each and the like. By multiplying these, the usage fee calculating means calculates and stores the usage fee for each user.

Further, as another embodiment of the present invention, as in the invention according to claim 16, the system further includes information on a right holder such as a copyright of text information and image information of a language before translation. And a license 'fee calculation means to be paid to the right holder, and a fee to be returned to the right holder according to the usage fee calculated by the usage fee calculation means. There is a system that can calculate and manage data. .

In this case, the unit price of the fee to be returned to the copyright holder can be set for each unit of access, execution of translation, acquisition of voice, or unit of use time or amount of use information. By multiplying these by the usage information of each copyrighted work, the license fee calculating means calculates and stores the license fee of each copyright and other right holder.

(Fourth embodiment)

Next, an information storage medium according to a fourth embodiment of the present invention will be described ₍ a computer-readable information storage medium according to this embodiment can store data constructed by the system described above). Something A CD-ROM is preferable. In addition, DVD, FD (floppy disk), MO, hard disk, and any other storage media are included. In addition, portable information terminals such as mobile phones, personal digital assistants, e-book players, and sound playback players can be loaded into portable information terminals that have been remarkably developed in recent years, and small information that can be loaded into computer devices such as personal computers. Storage media (various memories, etc.) are included.

The information storage medium includes at least image information storage means for storing image information, voice information storage means for storing voice information, and information processing means for displaying character information and voice information at a user terminal. If necessary, character information storage means for storing character information is provided.

Image files displayed on the user terminal, such as image files inserted into HTML files or XML files stored in the storage means, image files, and files such as slideshows including a plurality of image files. Has one or a plurality of balloon-like figures surrounded by a linear figure having an arbitrary shape and thickness at an arbitrary position in the image. In the image file displayed on the user terminal, one or more character information display fields for displaying character information are arranged.

Next, in the balloon-shaped figure arranged in the image information, image information including a balloon character at predetermined coordinates in the image information indicated by the balloon-shaped figure, or balloon character information is displayed. A balloon character information field is provided for performing the processing, and the image information including the balloon character or the balloon character information is associated with a predetermined voice information file stored in the voice information storage unit. In addition, it is possible to set in advance the association with a predetermined character information file stored in the character information storage means.

These configurations are omitted because they are common to the various embodiments described in detail above. In addition, as described in Claim 20, when performing a charging process or a user registration process, such as when downloading via a computer network or when registering a trial version user, the authentication permission information is set. If the information is stored in the authentication information storage means and made reproducible, it can be used only by the users who have downloaded it properly, or the trial version can be downloaded from a CD-ROM such as a magazine appendix. In order to distribute and obtain the official version, it is possible to take the form of registering the user online and obtaining the authentication permission information.

Next, in the invention according to claim 19, the information storage medium is loaded in the user terminal 1 which is a portable information terminal such as a mobile phone, a portable information terminal, an electronic book player, and a sound reproduction player. A small information storage medium that is possible and that can be loaded into a computer device such as a personal computer is adopted.

The image information storage unit, the voice information storage unit, the image information stored in the character information storage unit, the audio information, or the character information included in the information storage medium can be output or reproduced by a portable information terminal. Files are included.

Alternatively, a portable information terminal file that can be output or reproduced by the portable information terminal is stored separately from the computer device file.

FIG. 19 shows an example of a basic process using the information storage medium of the present embodiment.

First, an information storage medium is loaded into a user terminal 1 such as a portable information terminal (S400). The information processing means is activated (S401).

When the user terminal is a computer device such as a personal computer, all the data formats of images, sounds, and characters can be displayed, so that the same processing as in the above-described embodiments is performed. That is, for example, the process proceeds to S104 in FIG.

In the case of mobile information terminals, there are various terminals such as mobile phones, mobile information terminals, e-book players, and sound playback players. The process proceeds to the process of determining whether the data file can be displayed and played back.However, the determination may be performed automatically, or may be performed by the user through a menu display operation. .

It is determined whether or not an image file (page) can be displayed (S403), and if so, the image file is displayed (S404). The image file may be a file for a computer device or an image file for a portable information terminal.

If the image file cannot be displayed (S 403), it is determined whether or not character information display is possible (S 405), and if possible, the character information arranged in the balloon is displayed ( S406).

If character information cannot be displayed, audio information is output (S407). The reproduction of the audio information may be performed automatically, and the user operates and moves back and forth while reproducing the individual audio information corresponding to the balloon character information by an operation. May be.

Next, when the image file or the character information is displayed (S404, S406), select the balloon character information field in the image file (S408), or select the character information. (S409), and the control means detects the user's operation (S410, S411). _{0 The} detected information is received by the information processing means (S412). It is determined whether the balloon character information or character information has been selected (S 4 13), and the corresponding voice information is extracted (S 4 1 3).

5), and output (S4 17). In addition, the corresponding character information is extracted (S 4 1

6), and display (S4 18). Industrial applicability

As described above, according to the present invention, when playing back content, etc., attention is paid to the graphics of speech balloons used particularly in comics and the like, and the speech balloons are set to be associated with textual information and voice information. Provide a system that allows users to connect to a computer terminal or the Internet from a terminal, synthesize images, characters, sounds, etc., and perform various playback, output, and display. be able to.

In particular, by combining images and text with the corresponding translated text and audio, it is possible to provide a system that enables interactive translation and learning. It is possible to provide an information processing system for translation, which is capable of outputting character information, voice information, and the like in a language after translation according to a user's selection of image information and character information online. C Use of content such as comics and other images with text and information for language translation and education systems will not only provide new teaching materials for English conversation, etc. While having fun, users can learn conversations etc. in a way that is rich in interactivity and entertainment according to the will of the user.

Further, according to the present invention, in the system of the present invention, when a copyrighted work such as a comic is used, a consent of a copyright holder or the like is obtained, and a profit is also returned to the right holder. In addition, secondary use of comics and the like is possible, and the content of this system can be enhanced and provided.

Further, according to the present invention, an information storage medium storing data and a program for realizing the above-described system is provided, and in particular, an information storage medium usable in both a computer device and a portable information terminal is provided. be able to.

Claims

The scope of the claims

1. A computer provided with an input unit, a control unit, a display unit, an output unit, a storage unit, and the like.

Image information storage means for storing image information;

Voice information storage means for storing voice information;

If necessary, a character information storage means for storing character information is provided, and an image file, an image file, a plurality of image files inserted in an HTML file, an XML file, or the like stored in the image information storage means. An image file displayed on a user terminal such as a file such as a slide show including an aisle includes an image file, which is enclosed in a linear figure having an arbitrary shape and thickness at an arbitrary position in the image. A plurality of balloon-shaped figures are arranged, and the image file displayed on the user terminal is provided with one or a plurality of character information display fields for displaying character information as necessary. In the balloon-shaped figure arranged in the image information, image information including a balloon character at a predetermined coordinate in the image information indicated by the balloon-shaped figure is displayed. A speech character information field for arranging the speech information is provided, and the image information including the speech character is associated with a predetermined speech information file stored in the speech information storage means. It is possible to set in advance the association with a predetermined character information file stored in

Depending on the user's action, such as placing the cursor in the balloon character information field displayed in the balloon character information field placed in the displayed balloon-shaped figure, placing the cursor, and clicking, etc. Said memory hand A process in which the information processing means provided in the stage detects the operation of the user;

If the association with the predetermined character information file has been set in advance, the character information display field includes the character whose location is indicated by the link information added to the balloon character information field. C) transmitting information to the user terminal and displaying the information on display means provided in the user terminal. C.

2. A computer provided with an input unit, a control unit, a display unit, an output unit, a storage unit, and the like.

Image information storage means for storing image information;

At least information processing means for displaying character information and voice information on the user terminal is provided.

If necessary, a character information storage unit for storing character information is provided, and an image file, an image file, a plurality of image files inserted into an HTML file, an XML file, or the like stored in the image information storage unit. An image file displayed on the user terminal, such as a file such as a slide show including an aisle, is placed in an arbitrary place in the image, surrounded by a linear figure having an arbitrary shape and thickness. Or, a plurality of balloon-shaped figures are arranged. In the image file displayed on the user terminal, one or a plurality of character information display fields for displaying character information as necessary are arranged, In the balloon-shaped figure arranged in the image file, a balloon character information field for displaying balloon character information on predetermined coordinates in image information indicated by the balloon-shaped figure is arranged. The balloon character information is associated with a predetermined voice information file stored in the voice information storage unit, and is linked to a predetermined character information file stored in the character information storage unit. It is possible to set the association in advance,

The voice information indicating the location in the link information added to the balloon-shaped character information field arranged in the balloon-shaped figure is output by voice reproduction means provided in the user terminal, When the process of performing the reproduction and the association with the predetermined character information file are set in advance, the character information display field indicates the location in the link information added to the balloon character information field. Displaying the character information by a display means of the user terminal. The information processing system uses a balloon-shaped figure.

3. In the invention described in claims 1 or 2,

If the association with the predetermined character information file has been set in advance, the character information indicating the location in the link information added to the balloon character information field is displayed in the character information display field. Processing for displaying the image information including by the display means provided in the user terminal. Information processing using balloon-shaped figures, characterized by performing>

4. A computer provided with an input unit, a control unit, a display unit, an output unit, a storage unit, and the like.

Image information storage means for storing image information;

Voice information storage means for storing voice information;

If necessary, a character information storage unit for storing character information is provided, and an image file, an image file, a plurality of image files inserted into an HTML file, an XML file, or the like stored in the image information storage unit. The image file displayed on the user terminal such as a file such as a slide show including an aisle contains text information or text information consisting of an arbitrary shape and size, etc., at an arbitrary location in the image. One or more character information display fields for arranging image information are arranged,

According to the user's actions such as placing a cursor in a character information field or placing a cursor in a character information field and clicking in a language before translation, the above memory device A process in which the information processing means provided in the stage detects the operation of the user;

A process of transmitting the voice information as the translated language to the user terminal, which indicates the location in the link information added to the character information field, to a user terminal, and outputting and reproducing the voice information by voice reproducing means provided in the user terminal; When,

If the association with the predetermined character information file has been set in advance, the location is indicated in the link information added to the character information display field. A language translation information processing system for transmitting to a user terminal and displaying the information on display means provided in the user terminal.

5. In the invention described in claims 1-4,

For the user's actions such as placing the cursor on the balloon character information displayed in the balloon character information field indicated in the pre-translation language placed in the displayed balloon-shaped figure, or placing the cursor and clicking on it A process in which the information processing means provided in the storage means detects the operation of the user;

Transmitting the voice information, which is the translated language, to the user terminal, where the location is indicated by the link information added to the balloon-shaped character information field arranged in the balloon-shaped figure; A process of outputting and reproducing by an audio reproducing means provided in the terminal;

If the association with the predetermined character information file has been set in advance, the location is indicated in the link information added to the balloon character information field in the character information display field. language The balloon-shaped figure according to any one of claims 1 to 4, wherein the character information is transmitted to the user terminal and displayed by a display means provided in the user terminal. Language Translation Information Processing System

6. In the invention described in claim 5,

The post-translation speech information stored in the speech information storage means includes: post-translation language speech information including a plurality of post-translation languages corresponding to the pre-translation language character information included in the balloon character information field. Is included,

When the association with the predetermined character information file has been set in advance, the location is indicated in the link information added to the balloon character information field in the character information display field. 6. A language using a balloon-shaped graphic according to claim 5, wherein the character information that is the translated language is transmitted to the user terminal and is displayed by a display means included in the user terminal. Translation information processing system.

7. In the invention described in claims 4 to 6,

Blowing in the pre-translation language placed in the displayed balloon In the balloon character information displayed in the balloon character information field, balloon character information including a plurality of pre-translation languages is prepared in advance, and an image file including a balloon-shaped figure is stored in the image information storage means. And

A process in which a user selects from which of a plurality of pre-translation languages the pre-translation language character information is translated;

Use such as placing the cursor on the balloon character information displayed in the balloon character information field, which is displayed in the balloon character information field, placed in the displayed balloon-shaped figure, or by placing the cursor and clicking A process in which the information processing means provided in the storage means detects an action of the user in accordance with an action of the user;

If the association with the predetermined character information file has been set in advance, the location is indicated in the link information added to the balloon character information field in the character information display field. 7. A speech balloon according to any one of claims 4 to 6, characterized in that text information in a language is transmitted to the user terminal and displayed on display means provided in the user terminal. A language translation information processing system using figures.

8. In the invention described in claims 1 to 7,

Image files displayed on the user terminal such as image files inserted in HTML files or XML files stored in the image information storage means, image files, slideshow files including a plurality of image files, etc. Are multiple image files inserted in multiple HTML files and XML files, and multiple image files consisting of multiple pages. Data files, such as slide shows, or slide shows that include multiple image files that make up multiple pages,

The user terminal can select and display a desired page from a plurality of pages,

9. In the invention according to claim 8, the data file stored in the storage means is a story-like content data such as a cartoon,

9. The method according to claim 8, wherein a process of reproducing voice information associated with the selected balloon-shaped figure of the selected page using the balloon-shaped figure and outputting character information is possible. Content Reproduction Information Processing System Using Balloon Shapes

10. The invention according to claim 9, wherein the storage means stores a plurality of content data having a story, such as a manga, and a user selects one of the plurality of content data. 10. The content reproduction information processing system using a balloon-shaped figure according to claim 9, wherein the content reproduction information can be displayed and reproduced.

11. In the invention described in claims 1 to 10,

The audio information indicating the location in the link information added to the balloon-shaped character information field arranged in the balloon-shaped graphic includes a plurality of pieces of audio 'information having different audio playback speeds. Stored in the voice information storage means,

The speech balloon according to any one of claims 1 to 10, wherein a voice playback speed can be selected by setting in advance by a user. Information processing using figure-like figures ·

1 2. In the invention described in claims 1 to 11,

By setting in advance by the user, whether to automatically play voice information and text information, whether to turn pages automatically, select the number of times of voice playback, whether to display text information, The information processing system using a balloon-shaped graphic according to any one of claims 1 to 11, wherein a reproduction control method such as setting of a reproduction order of content data can be selected.

13. In the invention described in claims 6 or 7 or 11 or 12, the selection of the language before translation, the selection of the language after translation, the selection of the audio playback speed, or the selection of the playback control method is based on XML or the like. The settings selected by the user are stored in the storage means as setting files using the tag information, and are automatically played back according to the audio playback speed or playback control method selected with reference to the setting files during playback. An information processing system using the balloon-shaped figure according to any one of claims 6 or 7, or 11 or 12.

14. In the invention described in claims 1 to 13,

Image information storage means for storing image information;

If necessary, a character information storage means for storing character information is provided, and in accordance with a request from the accessing user terminal, a data file including a balloon-shaped figure is transmitted to the user terminal and displayed,

Place the cursor in the balloon character information field displayed in the balloon character information field placed in the displayed balloon-shaped figure. A process in which the information processing means provided in the storage means detects an action of the user in accordance with a user's action such as placing and clicking a sol;

If the association with the predetermined character information file has been set in advance, the character information display field includes the character information indicating the location in the link information added to the balloon character information field. And transmitting the data to the user terminal and displaying the data on the display means of the user terminal. The online processing using the balloon-shaped figure according to any one of claims 1 to 13, Information processing system.

15.The invention according to claim 14, wherein the system further comprises an authentication unit for authenticating a user who accesses the server system from a user terminal via a computer network;

15. The online information processing system using a balloon-shaped graphic according to claim 14, wherein a usage fee can be calculated and charge management can be performed according to the various types of usage information.

16. The invention according to claim 14, wherein the system further comprises a right holder information storage means for storing information on the right holder such as the copyright of the character information and image information of the language before translation,

The license fee paid to the right holder is provided, and a fee for returning to the right holder is calculated by the usage fee calculating unit. An online information processing system using a balloon-shaped figure according to claim 14, characterized in that it is calculated and managed in accordance with the usage fee and the like obtained.

17. In the invention described in claims 1 to 16,

A data capturing means for capturing, as image information, data such as a printed matter having image information and balloon character information;

An information processing system using the balloon-shaped figure according to any one of claims 1 to 16, characterized by comprising:

1 8. Image information storage means for storing image information,

Voice information storage means for storing voice information;

If necessary, a character information storage means for storing character information is provided, and an image file, an image file, and a plurality of image files inserted in an HTML file, an XML file, or the like stored in the storage means are provided. The image file displayed on the user terminal, such as a file such as a slide show, contains one or more files that are enclosed at any place in the image by a linear figure of any shape and thickness. Balloon shape is placed,

In the image file displayed on the user terminal, one or a plurality of character information display fields for displaying character information as necessary are arranged, and the balloon-shaped figure arranged in the image information is provided. The said balloon An image information including a balloon character or a balloon character information field for displaying balloon character information at predetermined coordinates in the image information represented by a striped figure, and an image information including a balloon character or a balloon character The information is associated with a predetermined voice information file stored in the voice information storage means, and is associated with a predetermined text information file stored in the text information storage means in advance. A computer-readable information storage medium, which can be set.

1 9. In the invention described in claim 18,

The information storage medium described above is a small-sized information storage medium that can be loaded into a portable information terminal such as a mobile phone, a portable information terminal, an electronic book player, or a sound reproduction player, and that can be loaded into a computer device such as a personal computer. So,

The image information storage means, the voice information storage means, the image information, the sound information, or the character information stored in the character information storage means provided in the information storage medium can be output or reproduced on a portable information terminal. Claim 17 characterized by that a file for a form information terminal that can be output or reproduced on a portable information terminal is stored separately from a file for a computer device, and that the file for a portable information terminal is stored separately from the file for a computer device. Computer-readable information storage medium.

20. In the invention described in claims 18 or 19,

When downloading over a computer network or registering a trial version of the user, the authentication permission information is stored in the above-mentioned authentication information storage means when the accounting process or the user registration process is performed. The computer-readable information storage medium according to claim 18, wherein the information storage medium is enabled.