JP6089874B2 - Multilingual simultaneous playback system - Google Patents

Description

  The present invention relates to a method for simultaneously reproducing multi-languages that allows a plurality of viewers to simultaneously view and listen to the same video or music on a single device while viewing and listening in the language according to the request of each viewer. And the system.

  Traditionally, many video playback devices have been installed for guidance and publicity in places where many people with various native languages gather, such as airports, stations, museum facilities, and international conference halls. Also, in these devices, multilingual audio is often played and subtitles are displayed. However, although major languages such as English and Chinese have been supported, there are few examples that support other languages, and the barrier-free effect due to multilingualization is not necessarily sufficient.

  There is already known a system that allows viewers to arbitrarily select the language of audio and subtitles displayed on the screen at the same time as displaying images such as guidance, publicity, or content of products that interest them. ing. In Patent Document 1, a plurality of video files and language-specific audio prepared for each video file are distributed by adding audio or subtitle text in an arbitrary language to a video selected by a user via a communication network. A video distribution system for accumulating files is shown.

  Further, in Patent Document 2, an operation guidance device that uses a DVD disc having multilingual reproduction information, for example, that satisfies the DVD video standard, and provides detailed guidance on the operation of a device or the like in a language requested by a user simultaneously with video. Is disclosed.

  In Patent Document 3, for example, audio information that is a sub-content that describes the contents of an exhibit is synchronized with the reproduction of video information that is the main content of the exhibit, for example, for a user who has approached the vicinity of the exhibit. There is disclosed an exhibit guide system which is distributed to an information terminal carried by a user, and the sub-content to be distributed can select a language desired by the user.

JP 2004-215126 A JP 2001-6259 A JP 2008-97362 A

  However, in the method disclosed in Patent Document 1, one receiver is required for each group of viewers who select the same language. In a public place or an environment where many viewers view simultaneously, Not suitable. The method disclosed in Patent Document 2 is also premised on the existence of one receiver for each viewer. Furthermore, when both add new languages after system development and video production, the system and video have to be recreated, and there has been a problem of lack of extensibility. Moreover, in the system in Patent Document 3, the information terminal possessed by the viewer is a dedicated device and the distribution area is limited, so that there is a problem that the viewer has to go to the vicinity of the device.

  The present invention has been made in order to solve the above-described points. An object of the present invention is to allow a plurality of viewers to view the same video content at the same time using a single playback device. It is intended to provide a multi-language simultaneous playback system that enables viewing in a language desired by a user and that allows easy addition of a new language even after production of video content.

  In order to solve the above problems, a first invention of the present application includes a content playback device that plays back video content including an audio signal in which a digital watermark is embedded, and a viewer terminal device that is connected to the sub-content server device via a network. , Wherein the viewer terminal device includes a communication means for transmitting / receiving information to / from the sub-content server device, and a digital watermarked audio signal reproduced by the content reproduction device. Watermark detecting means for detecting a digital watermark including content type and time information, and transmitting the information extracted from the digital watermark and predetermined control information to the sub-content server device via the network. Sub-content acquisition means for acquiring sub-content from the device, and reproducing the sub-content A sub-content reproduction unit, wherein the sub-content server device is extracted from a transmission / reception unit that transmits / receives information to / from a viewer terminal device and a digital watermark received from the viewer terminal device via a network Digital watermark signal analyzing means for selecting sub-contents identified from the information and predetermined control information, sub-content distribution means for transmitting data relating to the selected sub-contents to the viewer terminal device via the network, and each of the means A multilingual simultaneous playback system comprising: a control means for controlling As a result, the viewer detects the digital watermark identifier from the audio signal with the digital watermark that is simultaneously reproduced using the viewer terminal device while watching the video content of the content reproduction device, and transmits it to the sub-content server device. The sub-content can be received from the sub-content server device.

  According to a second aspect of the invention, the viewer terminal device detects a digital watermark identifier by acquiring a communication means for transmitting / receiving information to / from the sub-content server device and an audio signal with a digital watermark reproduced by the content reproduction device. A sub-content acquisition unit that transmits an identifier of the digital watermark and predetermined control information to the sub-content server device through a network, and acquires sub-content and synchronization information related to the content from the sub-content server device; A sub-content reproducing unit that reproduces sub-contents, wherein the sub-content server device receives and transmits from the viewer terminal device via a network, and a transmission / reception unit that transmits and receives information to and from the viewer terminal device. A sub-computer specified from the digital watermark identifier and predetermined control information A digital watermark signal analyzing means for selecting synchronization information related to content and content, and a sub-content distribution means for transmitting synchronization information related to the selected sub-content and content to a viewer terminal device via a network. The multi-language simultaneous playback system according to the first invention is characterized by being configured by these. Accordingly, the viewer terminal device can reproduce the sub content even when the sub content is divided into a plurality of files.

  A third invention is the multi-language simultaneous playback system according to the first or second invention, wherein the sub-content playback unit of the viewer terminal device plays back the sub-content that is an audio signal. Thereby, the viewer can listen to sub-contents such as multilingual audio simultaneously with the content video.

  A fourth invention is the multi-language simultaneous playback system according to the first or second invention, wherein the sub-content playback means of the viewer terminal device plays back the sub-content which is text information. Accordingly, sub-contents such as explanations in multilingual subtitles can be displayed in synchronization with the content video on the display unit of the viewer terminal device.

  According to the present invention, a viewer corresponding to the main content being played back by the content playback device using a viewer terminal device (hereinafter referred to as a viewer terminal) such as a mobile phone possessed by the viewer is desired. Content (audio, video, or text information) can be downloaded from a sub-content server device (hereinafter referred to as a sub-content server) and can be played back on a viewer terminal.

It is a figure which shows the whole structure of the system of this embodiment. It is a functional block diagram explaining the whole structure of this embodiment. It is a functional block diagram explaining each device in the present invention in more detail. 2 is a diagram illustrating a hardware configuration of a viewer terminal 200. FIG. It is a block diagram explaining the flow of the process of the whole system which concerns on one Embodiment of this invention. 6 is a flowchart illustrating an example of a flow of sub-content acquisition / playback processing according to an embodiment of the present invention. It is a figure which shows the data structure of the main content and subcontent which concern on one Embodiment of this invention. It is a block diagram explaining the flow of the process of the whole system which concerns on another embodiment of this invention. 6 is a flowchart illustrating an example of a flow of sub-content acquisition / playback processing according to another embodiment of the present invention. It is a figure which shows an example of a synchronous table. It is a figure which shows an example of a content table. It is a figure which shows an example of the data structure of the main content and subcontent based on another embodiment of this invention. It is a figure which shows an example of the data structure of the main content and subcontent based on another embodiment of this invention.

  Hereinafter, the configuration of a system according to an embodiment of the present invention will be described in more detail with reference to the drawings.

  FIG. 1 is a diagram showing how the system of this embodiment is used. The viewer views the reproduced video and audio in front of a content reproduction device that reproduces the video and audio recorded on a DVD or the like. A digital watermark is recorded in the audio signal, and the application of the viewer terminal recognizes the digital watermark of the audio signal captured from the built-in microphone of the viewer terminal, thereby allowing the viewer to communicate with the viewer through a device such as an earphone. For example, it is possible to provide audio in a desired language.

  FIG. 2 is a functional block diagram for explaining the overall configuration of the present embodiment in more detail. The viewer terminal 200 transmits and receives information between the sub-content server 100 and the application distribution server 300 via the network 500. The content reproduction device 400 transmits an audio signal to the viewer terminal 200 offline.

  Although the network 500 is shown in a simplified manner, it is a combination of a plurality of networks via a gateway such as the Internet, and is composed of a LAN (Local Area Network), a telephone line, a mobile communication network, etc. The access point is provided to enable transmission / reception of information between the viewer terminal 200, the sub-content server 100, and the application distribution server 300 using wired or wireless.

  FIG. 3 is a functional block diagram illustrating the sub-content server 100, the viewer terminal 200, and the application distribution server 300 of the present invention in more detail. The application distribution server 300 is an HTTP (Hyper Text Transfer Protocol) server, and is transmitted to the viewer terminal 200 via a web page existing on the application distribution server 300 using a WWW (World Wide Web) via a network such as the Internet. It is a server that distributes applications that operate on the viewer terminal 200. The application distribution server 300 includes a communication unit 310 that performs transmission and reception with the viewer terminal 200 via the network 500, receives a request from the viewer terminal 200, and displays an application requested from the application database 320 according to a predetermined procedure. 200.

  The viewer terminal 200 transmits information extracted from the digital watermark and predetermined control information to a sub-content server 100 that is an HTTP (HyperText Transfer Protocol) server via a network such as the Internet, and the sub-content server 100. A portable device such as a smartphone that receives sub-content and synchronization information provided to the viewer from the mobile phone, reproduces the received sub-content and displays it on the terminal display, or outputs it as an audio signal to the MicroUSB terminal. is there.

  The viewer terminal 200 also includes a communication unit 210 that transmits and receives data to and from the sub-content server 100 and the application distribution server 300 via the network 500. The viewer terminal 200 itself listens to a digital watermarked audio signal that is reproduced by the content reproduction device 400. A digital watermark identifier that is acquired from a built-in microphone, is transmitted to the sub-content server 100 together with predetermined control information to detect a digital watermark identifier, and sub-content data is acquired from the sub-content server 100 Sub-content acquisition means 230 and sub-content reproduction means 240 for reproducing the acquired sub-content data are provided.

  FIG. 4 is a diagram illustrating a hardware configuration of the viewer terminal 200. The viewer terminal 200 is a computer system as hardware. The viewer terminal 200 includes a control unit 201, a storage unit 202, a peripheral device I / F unit 203, an input unit 204, a display unit 205, and a communication unit 206, which are connected via a bus 209. Note that the hardware configuration shown in FIG. 4 is merely an example, and various configurations can be adopted depending on purposes.

  The control unit 201 includes a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and the like. The CPU calls a program stored in the storage unit 202, ROM, recording medium, etc. to a work memory area on the RAM, executes it, drives and controls each device connected via the bus 209, and performs processing performed by the computer. Realize. The ROM is a non-volatile memory, and permanently stores programs such as a boot program and BIOS for the viewer terminal 200, data, and the like. The RAM is a volatile memory, and temporarily stores a program, data, and the like loaded from the storage unit 202, ROM, recording medium, and the like, and includes a work area used by the control unit 201 for performing various processes.

  The storage unit 202 is preferably a compact storage medium such as SSD (Solid State Drive), and stores a program executed by the control unit 201, data necessary for program execution, an OS (operating system), and the like. Regarding the program, there are a control program corresponding to an OS (operating system), a program code that functions as the digital watermark detection means 220, a program code that functions as the sub-content acquisition means 230, and a program code that functions as the sub-content playback means 240. Stored. These program codes are read by the control unit 201 as necessary, transferred to the RAM, and read and executed by the CPU to function as various means.

The peripheral device I / F unit 203 is a port for connecting the peripheral device to the viewer terminal 200, and the computer via the peripheral device I / F unit 203 allows the microphone 211 and the earphone terminal (for example, MicroUSB terminal) 212. Send and receive data to and from peripheral devices.

  The input unit 204 inputs data, and has an input device such as a touch panel, for example, and can perform operation instructions, operation instructions, data input, and the like to the viewer terminal 200 via the input unit 204. it can. The display unit 205 is a display device such as a liquid crystal panel.

  The communication unit 206 includes a communication control device, a communication port, and the like. One is a wired or wireless communication interface that mediates communication between the viewer terminal 200 and the network 500, and other computers are connected via the network 500. In particular, a communication unit that performs communication control with the sub-content server 100 is provided. The bus 209 is a path that mediates transmission / reception of control signals, data signals, and the like between the devices.

  Although not particularly illustrated, the sub-content server 100 and the application distribution server 300 also have a hardware configuration in accordance with the viewer terminal 200, and include an input unit such as a display unit, a storage unit, and a touch panel. The communication unit includes a communication control device, a communication port, and the like, and includes a communication unit that performs communication control with another computer via the network 500.

  The content playback apparatus 400 is a device set including a DVD or the like that plays back content made up of video and audio, and a large display and a speaker, preferably 50 inches or more in size, connected thereto. The content reproduction device 400 is not connected to the sub content server 100, the application distribution server 300, and the viewer terminal 200, and reproduces content independently.

  Next, the flow of processing of the present embodiment will be described in detail with reference to the drawings including the functions of the respective devices.

  FIG. 5 is a block diagram illustrating the flow of processing of the entire system according to one embodiment of the present invention. First, when the viewer desires to use the multilingual simultaneous playback system according to the present invention, the multilingual simultaneous playback application is acquired by the application distribution server 300 via the network 500 through the communication unit 220 using the viewer terminal 200. Send request to request. When the application distribution server 300 receives this request through the communication unit 310, the application distribution server 300 selects the application from the application database 330 through the control unit 320, and transmits it to the viewer terminal 200 via the network 500 through the transmission / reception unit 310. Is transmitted (S01). This process is normally performed only when the application is used for the first time.

  After receiving the multilingual simultaneous playback application, the viewer activates the application using the viewer terminal 200 (S02). When the application is activated, an input request is made to select the language to be used, so the language to be used is selected by clicking the touch panel (S03). When the usage language is selected, the sub-content to be acquired and reproduced by the viewer terminal 200 is determined. The viewer terminal 200 acquires a digital watermarked audio signal to be played back by the content playback device 400, and acquires and plays back the sub-content (S04).

  FIG. 6 is a flowchart for explaining an example of the flow of sub-content acquisition / playback processing (S04 in FIG. 5). Here, it is assumed that the digital watermark data includes a content name and an elapsed time from the beginning. When the multilingual simultaneous playback application of the viewer terminal 200 extracts the digital watermarked audio signal (S061), the digital watermark data is evaluated (S062). Here, when the digital watermark data cannot be obtained, the process returns to the extraction of the audio signal with the digital watermark again. Further, when the digital watermark data can be acquired and the data is predetermined information indicating the end (for example, END), the processing is ended. If it is normal digital watermark data (content name and identifier indicating elapsed time from the beginning), it is evaluated whether or not the sub-content corresponding to the content and the language used has already been read (S063).

  Here, if the sub-content has not yet been read, for example, when this routine is entered for the first time, the detected digital watermark data (content name and identifier indicating elapsed time from the beginning) and predetermined control information are used. A usage language is transmitted from the communication unit 210 to the sub-content server 100 via the network 500, and a request for acquisition of the sub-content is made. When the transmission / reception means 110 of the sub-content server 100 receives this data, it activates the digital watermark signal analysis means 130 through the control means 120, and the main content is obtained from the received digital watermark data (content name and identifier indicating elapsed time from the beginning). And the sub-content to be transmitted to the viewer terminal 200 is specified from the received usage language. Here, the sub-content distribution unit 140 is activated, and the sub-content to be transmitted is transmitted to the viewer terminal 200 through the network 500 through the transmission / reception unit 110 (S065).

  When receiving the sub-content, the viewer terminal 200 stores that the sub-content has been read, and stores the received sub-content data in the sub-content storage unit 290.

  FIG. 7 is an example of sub-content data. Here, a plurality of language-specific sub-contents having the same playback time length as the main content are stored in the sub-content database. Each sub-content is composed of a single file, and audio data is recorded from the top. Here, the main content includes a digital watermark at a timing of once every few seconds, for example, with a single segment such as a phrase (in this example, an identifier indicating the time from the beginning is inserted as a digital watermark).

  Returning to FIG. 6, when the next digital watermarked audio signal is extracted (S061), the digital watermark data is evaluated (S062). When normal digital watermark data (content name and identifier indicating elapsed time from the beginning) is detected, since the sub-content has already been read (S063), the “elapsed time from the beginning” is determined from the digital watermark data. What is necessary is just to extract and reproduce | regenerate the audio | voice data of a corresponding time part (S064). For example, when the sub-content of sub-content 1 (English) in FIG. 7 is read and the detected digital watermark data is “0′22” ”, immediately after“ 0′22 ”” in the sub-content. The voice data may be extracted and played back.

  Next, another embodiment of the present invention will be introduced. According to this embodiment, the sub content data may be divided into a plurality of files, and the reproduction time of the main content and the sub content need not be the same length. FIG. 8 is a block diagram illustrating the flow of processing of the entire system according to another embodiment of the present invention. As described in the previous embodiment, when the application is used for the first time, the multilingual simultaneous playback application is acquired from the application distribution server 300 and downloaded to the viewer terminal 200 (S11).

  After receiving the multilingual simultaneous playback application, the viewer activates the application using the viewer terminal 200 (S12). When the application is activated, an input request is made to select the language to be used, so the language to be used is selected by clicking the touch panel (S13). When the selection of the use language is completed, the sub-content to be acquired and reproduced by the viewer terminal 200 is determined. The viewer terminal 200 acquires a digital watermarked audio signal that is reproduced by the content reproduction apparatus 400 (S14).

  FIG. 9 is a flowchart for explaining an example of the flow of sub-content acquisition / playback processing (S14 in FIG. 8). Here, it is assumed that the digital watermark data includes a content name and a digital watermark identifier (ID). When the multilingual simultaneous playback application of the viewer terminal 200 extracts a digital watermarked audio signal (S161), it evaluates whether it is a digital watermark (S162). Here, when the digital watermark identifier (ID) cannot be acquired, the process returns to digital watermark extraction again. In addition, when the digital watermark identifier (ID) can be acquired and the ID is predetermined information indicating the end (for example, 9999), the processing is ended. If the ID is a normal ID, it is evaluated whether the synchronization information of the content has already been read (S163).

  Here, when the synchronization information of the content has not yet been read, for example, when this routine is entered for the first time, the detected digital watermark identifier (ID), the ID of the language used and the synchronization information as predetermined control information An acquisition request is transmitted from the communication unit 210 to the sub-content server 100 via the network 500, and the acquisition of the synchronization table and the content table that are the synchronization information of the content is requested. When the transmission / reception means 110 of the sub-content server 100 receives this data, it activates the digital watermark signal analysis means 130 through the control means 120, determines the main content from the detected digital watermark ID, and sends it to the viewer terminal 200 from the ID of the language used. Identify the synchronization table and content table to be sent. Here, the sub-content distribution unit 140 is activated, and the synchronization table and the content table to be transmitted are transmitted to the viewer terminal 200 via the network 500 through the transmission / reception unit 110 (S164). As shown here, one piece of synchronization information (synchronization table and content table) exists for one language (one sub-content) of one content.

  When the viewer terminal 200 receives the synchronization table and the content table, the viewer terminal 200 stores that they have been read, and stores the received synchronization table and content table data in the sub-content storage unit 290.

  FIG. 10 shows an example of the contents of the synchronization table. Here, for all digital watermark IDs of a certain main content, the main content synchronization position indicating which elapsed time of the main content the ID indicates and the elapsed time of the sub-content file corresponding to each language A sub-content synchronization position indicating whether the sub-content synchronization position is present, a sub-content physical position indicating where the sub-content synchronization position physically exists (here, the number of bytes from the beginning of the file), and This is an example of a download request flag that prompts reading of the sub-content file.

  FIG. 11 shows an example of the contents of the content table. Here, the language code corresponding to the language, the sub-content file name constituting the sub-content, the reproduction start position describing the reproduction start time, the reproduction end position describing the reproduction end time, and the sub-content file are downloaded. This is an example in which a downloaded flag indicating whether or not has been recorded as one record. The downloaded flag is assigned by the application program, and all the flags are set to OFF immediately after downloading the content table.

  Returning to FIG. 9, the viewer terminal 200 acquires the synchronization position on the sub-content from the received digital watermark identifier (ID) and the synchronization table. For example, if the received digital watermark ID is “xyz0051”, it can be seen from FIG. 10 that the synchronization position on the sub-content is “09′15” ”, and the physical position is 55503 bytes from the top of the sub-content file. (S165).

  Next, from the synchronization position on this sub-content and the content table, it can be determined in which sub-content file this synchronization position exists. For example, if the synchronization position is “09′15”, it can be seen from FIG. 11 that this synchronization position is “English01.wav” (S166).

  When the sub-content file to be reproduced and the synchronization position are determined as described above, it is determined whether the sub-content file has already been read with reference to the downloaded flag in the content table. Further, it is determined whether or not a download request flag corresponding to the received digital watermark ID is set with reference to the synchronization table (S167).

  Here, if the sub-content file has not been read yet, or if a download request flag corresponding to the received digital watermark ID is set, the sub-content server 100 is detected with the detected digital watermark and predetermined control information. A usage language and a sub-content file acquisition request are transmitted from the communication unit 210 to the sub-content server 100 via the network 500 to request acquisition of the sub-content file. When the transmission / reception means 110 of the sub-content server 100 receives this data, it activates the digital watermark signal analysis means 130 through the control means 120, determines the main content from the received digital watermark data (content name and digital watermark ID), and receives it. The sub-content file to be transmitted to the viewer terminal 200 is specified from the used language and the synchronization information held by itself. Here, the sub-content distribution unit 140 is activated, and the sub-content file to be transmitted is transmitted to the viewer terminal 200 through the network 500 through the transmission / reception unit 110 (S169). Here, since the sub-content file name to be acquired on the viewer terminal side is already known, the sub-content file name and sub-content file to be acquired instead of the detected digital watermark data are sent to the sub-content server 100. An acquisition request may be transmitted.

  When the viewer terminal 200 receives the sub-content file, the viewer terminal 200 records that the sub-content file has been read in the content table (sets the downloaded flag in the sub-content file field to ON), and receives the data of the received sub-content file. Is stored in the sub-content storage unit 290.

  FIG. 12 is a diagram showing an example of the data configuration of main content and sub-content according to another embodiment of the present invention. Here, with respect to the main content, the sub-content corresponding to a certain language is divided into a plurality of sub-content files. Also in this case, by referring to the synchronization table and the content table for the received digital watermark ID, it is possible to specify from which position in the sub-content file it should be reproduced.

  Furthermore, as shown in FIG. 13A, the information in the sub-content may be composed of only a sound part, for example. Further, as shown in FIG. 13B, one sub-content may be divided into sub-content files composed of only a plurality of sound parts.

  Returning to FIG. 9, for example, if the digital watermark ID received in S161 is “xyz0052”, the synchronization information has already been read, so the synchronization position of the sub-content is acquired (S165). Therefore, referring to the synchronization table of FIG. 10, the sub-content synchronization position for this ID is “9'40”, the physical position is 58044 bytes from the beginning of the file, and a download request flag is set. Next, referring to the content table of FIG. 11 in order to determine the sub-content file, this synchronization position “9′40” ”exists in the sub-content file“ English01.wav ”, and this sub-content is determined from the downloaded flag. It can be seen that the file has already been downloaded (S166).

  Returning to FIG. 9, since the sub-content file at this synchronization position has already been downloaded, but the download request flag is set (S167), the next sub-content file “English02.wav” is downloaded, and the corresponding digital watermark ID is supported. What is necessary is just to reproduce | regenerate the audio | voice data of a time part (S169). It should be noted that the next sub-content file download is expected to take a certain amount of time, so it is desirable that the download request flag is set at a position with some time margin.

  As described above, according to another embodiment of the present invention, the amount of data that can be transmitted to the viewer terminal at a time due to the large capacity of the sub-content to be reproduced or the communication speed by using the synchronization information. Even if the data is limited, it is possible to reduce the amount of data transmitted at one time by dividing the file or compressing the data.

  According to the present embodiment, a content playback apparatus that plays back video content including an audio signal in which a digital watermark is embedded, a digital watermark is read from the played back audio, and multi-language sub-contents are downloaded and played back from a predetermined position. The viewer terminal application provides a multilingual simultaneous playback system that allows a large number of viewers to view the same video content at the same time, while allowing each viewer to view in a desired language.

  The present invention is not limited to the above-described embodiments, and various modifications and applications are possible within the scope of the claims. For example, the sub-content may be video or text information instead of audio, and the main content may be only an audio signal or only a digital watermark signal. Further, in the synchronization table for synchronizing the main content and the sub-content, if the sub-content synchronization position is unnecessary if the sub-content physical position is known, this information may not be necessary. In particular, when the sub-content is text information, the sub-content synchronization position is not necessary. It should be noted that the present invention can be similarly implemented even if the viewer terminal is a terminal such as a tablet personal computer or a notebook personal computer.

100 Sub-content server 110 Transmission / reception means 120 Control means 130 Digital watermark signal analysis means 140 Sub-content distribution means 180 Sub-content data 190 Synchronization information 200 Viewer terminal 210 Communication means 220 Digital watermark detection means 230 Sub-content acquisition means 240 Sub-content reproduction means 300 Application Distribution Server 400 Content Playback Device 500 Network

Claims (4)

A multilingual simultaneous playback system configured to include a content playback device that plays back video content including an audio signal embedded with a digital watermark, and a viewer terminal device connected to the sub-content server device via a network,
The viewer terminal device
A communication means for transmitting / receiving information to / from the sub-content server apparatus; and a digital watermark detection means for detecting a digital watermark including a digital content type and time information by acquiring a digital watermarked audio signal reproduced by the content reproduction apparatus; ,
Sub-content acquisition means for acquiring information extracted from the digital watermark and predetermined control information through the network to the sub-content server device and acquiring the sub-content from the sub-content server device;
A sub-contents reproducing means for reproducing the sub-contents,
The sub-content server device
A transmission / reception means for transmitting / receiving information to / from a viewer terminal device;
Digital watermark signal analysis means for selecting sub-contents identified from information extracted from digital watermark received from the viewer terminal device through the network and predetermined control information;
Sub-content distribution means for transmitting data relating to the selected sub-content to the viewer terminal device via the network;
A multilingual simultaneous reproduction system comprising: a control unit that controls each of the units described above. The viewer terminal device
A communication means for transmitting / receiving information to / from the sub-content server device, a digital watermark detection means for detecting a digital watermark identifier by acquiring a digital watermarked audio signal reproduced by the content reproduction device,
Sub-content acquisition means for transmitting the identifier of the electronic watermark and predetermined control information to the sub-content server device through the network, and acquiring the sub-content and the synchronization information related to the content from the sub-content server device;
A sub-contents reproducing means for reproducing the sub-contents,
The sub-content server device
A transmission / reception means for transmitting / receiving information to / from a viewer terminal device;
A digital watermark signal analyzing means for selecting sub-content specified from the digital watermark identifier received from the viewer terminal device through the network and predetermined control information and synchronization information related to the content;
The sub-content distribution unit that transmits the selected sub-content and the synchronization information related to the content to the viewer terminal device through the network, and the apparatus includes the sub-content distribution unit. Multi-language simultaneous playback system. The sub-contents reproducing means of the viewer terminal device is
The multilingual simultaneous reproduction system according to claim 1 or 2, wherein the sub-content which is an audio signal is reproduced. The sub-contents reproducing means of the viewer terminal device is
The multilingual simultaneous reproduction system according to claim 1 or 2, wherein the sub-content which is text information is reproduced.

Images