JP6596450B2 - Computer, content reproduction method and program - Google Patents

Description

  The present invention relates to a computer, a content reproduction method, and a program for reproducing audio data for reproducing audio and drawing data for drawing on a background image in order to view content.

  In recent years, various contents have been provided to users with the spread of public network such as the Internet. Such contents tend to have a large file capacity because audio and video are synthesized as one file and become one content. Therefore, when downloading and reproducing a plurality of contents, there has been a problem that it takes time to download.

  In order to solve such a problem, a system is disclosed that controls reproduction of video data and audio data divided into a plurality of parts based on a comparison result between the current time and a designated reproduction start date and time ( Patent Document 1).

JP, 2006-127370, A

  However, in the configuration of Patent Document 1, although it is possible to suppress the time required for downloading, it is difficult to reproduce video data and audio data at the same timing. In particular, when playback control is performed using the timer of the terminal itself to be played back, synchronization may be lost due to the processing capability of the terminal.

  An object of the present invention is to provide a computer, a content reproduction method, and a program capable of improving convenience and suppressing synchronization deviation when reproducing content.

  The present invention provides the following solutions.

The present invention is a computer (for example, a reproduction terminal 100 described later) that reproduces audio data for reproducing audio and drawing data for drawing on a background image in order to view content.
Data storage means in which the audio data and the drawing data are stored in separate files;
Current time acquisition means for acquiring the current time from a computer providing standard time;
The time difference between the acquired current time and the previously acquired time is calculated, and from the value obtained by dividing the time difference by the time of one frame, the number of frames of drawing data to be reproduced during the time difference is calculated. Frame calculation means using a stacked time variable that adds the remainder to the time difference before calculation,
In accordance with the current number of frames and the calculated number of frames, a frame number adjusting unit that adjusts the number of frames for reproducing the drawing data by adding the number of frames for the delay to the current number of frames when a delay occurs, (For example, a playback terminal 100 to be described later).

According to the present invention, in order to view content, a computer that reproduces audio data for reproducing audio and drawing data for drawing on a background image, the audio data and the drawing data are individually stored. The current time is obtained from a computer stored in a file and providing standard time, the time difference between the obtained current time and the last obtained time is calculated, and the time difference is calculated from the value obtained by dividing the time difference by the time of one frame . Calculate the number of frames of drawing data to be reproduced in the meantime, and use the accumulated time variable to add the remainder of the division to the time difference before calculation. Delay occurs according to the current number of frames and the calculated number of frames. Sometimes, the number of frames for the delay is added to the current number of frames to adjust the number of frames for reproducing the drawing data.

  Although the present invention is a computer category, the same operations and effects corresponding to the category are exhibited in other categories such as a content reproduction method and a program.

  According to the present invention, it is possible to provide a computer, a content reproduction method, and a program capable of improving convenience and suppressing synchronization deviation during content reproduction.

FIG. 1 is a diagram showing an outline of the content reproduction system 1. FIG. 2 is an overall configuration diagram of the content reproduction system 1. FIG. 3 is a functional block diagram of the computer 10 and the playback terminal 100. FIG. 4 is a flowchart of content registration processing executed by the computer 10. FIG. 5 is a flowchart of the content acquisition process executed by the playback terminal 100. FIG. 6 is a flowchart of content reproduction processing executed by the reproduction terminal 100. FIG. 7 is a flowchart of content playback processing executed by the playback terminal 100. FIG. 8 is a flowchart of content playback processing executed by the playback terminal 100.

  Hereinafter, the best mode for carrying out the present invention will be described with reference to the drawings. This is merely an example, and the technical scope of the present invention is not limited to this.

[Outline of Content Playback System 1]
An outline of a preferred embodiment of the present invention will be described with reference to FIG. FIG. 1 is a diagram for explaining an outline of a content reproduction system 1 which is a preferred embodiment of the present invention. The content playback system 1 includes a server-side computer 10 and a playback terminal 100 that provide content, and plays back audio data for playing music and drawing data for drawing on a background image in order to view the content. Computer system.

  In FIG. 1, the number of computers 10 and playback terminals 100 is not limited to one and may be plural. Each process described below may be realized by either or both of the computer 10 and the playback terminal 100.

  The computer 10 is a computer device capable of data communication with the playback terminal 100 and capable of executing various processes.

  The playback terminal 100 is a computer device capable of data communication with the computer 10. The playback terminal 100 is, for example, an electronic product such as a netbook terminal, a slate terminal, an electronic book terminal, a portable music player, a smart glass, a head-mounted display, etc. in addition to a mobile phone, a portable information terminal, a tablet terminal, and a personal computer. Wearable terminals and other items.

  First, the computer 10 acquires content from another computer, terminal device, or the like (not shown) (step S01). The content is, for example, a moving image composed of audio and video.

  The computer 10 divides the acquired content into audio data, drawing data, and background image data, and stores them as separate files (step S02). The computer 10 recognizes the content by voice and extracts voice data. The computer 10 extracts data of a predetermined extension (such as jpg, gif, png, bmp) and a predetermined data format included in the content as background image data. The computer 10 recognizes the pen drawing (graphics, annotations, emphasis, mathematical formulas, etc.) in the content and the drawing drawn by the CANVAS of the browser, and draws the coordinates, time, color, thickness, etc., the drawing, etc. of the pen drawing. Extract as data. Accordingly, the drawing data includes at least the coordinates, time, color, and thickness of the drawn points and figures. The computer 10 also stores the number of frames from which each data is to be reproduced.

  Here, the CANVAS tag of the browser is one of HTML elements that are the display language of the homepage. By specifying this as a tag in HTML, it is possible to draw a figure or the like using JavaScript. Components and functions. By using <CANVAS>, it is possible to display a figure using only standard HTML or JavaScript. Therefore, using this CANVAS tag, drawing data is drawn (reproduced) every time by the browser function.

  Note that the computer 10 may be configured to divide content into audio data and drawing data. In this case, the background data described above may be included in the drawing data. Moreover, the playback terminal 100 may execute each process described above. In this case, the playback terminal 100 may execute each process executed by the computer 10.

  The playback terminal 100 acquires content from the computer 10 (step S03). The playback terminal 100 separately receives audio data, drawing data, and background image data as content. The playback terminal 100 may acquire the audio data, the drawing data, and the background image data sequentially as a divided file, or may acquire the audio data, the drawing data, and the background image data as they are. .

  The playback terminal 100 stores the audio data, drawing data, and background image data as separate files (step S04). At this time, the reproduction terminal 100 also stores the number of frames for reproducing the audio data, the drawing data, and the background image data.

  The playback terminal 100 receives a content playback instruction from the user (step S05). The playback terminal 100 receives a content playback instruction using a predetermined browser. In particular, the reproduction of the drawing data is realized by using the browser CANVAS.

  The playback terminal 100 acquires the current time from a computer (not shown) that provides standard time (step S06). The playback terminal 100 uses, for example, UTC (Coordinated Universal Time) as standard time. The playback terminal 100 acquires the standard time every 30 milliseconds, for example.

  The playback terminal 100 calculates the time difference between the acquired current time and the previously acquired time, and calculates the number of frames of drawing data to be played during this time difference (step S07). For example, when the time difference is 57 milliseconds, the playback terminal 100 calculates the number of frames of drawing data to be played back as 5 because one frame is 10 milliseconds.

  The playback terminal 100 adjusts the number of frames for playing back the drawing data according to the calculated number of frames (step S08). For example, when the calculated number of frames is five, the reproduction terminal 100 adjusts the number of frames for reproducing drawing data based on whether the current number of frames is larger or smaller than five. Specifically, since one frame is 10 milliseconds, in the case of 57 milliseconds, 5 frames and the remainder are 7. When the number of frames is smaller than 5, the playback terminal 100 adjusts by adding 5 frames to the current frame and advancing the frame by 5. On the other hand, when the number of frames is larger than 5, the number of frames is not added. Further, the playback terminal 100 adjusts the remainder by adding the remainder 7 to the next time difference using the accumulated time variable.

  The playback terminal 100 plays back the content based on the adjusted number of frames (step S09). The playback terminal 100 emits a sound based on the sound data, plays the drawing data using the browser CANVAS, and displays the background image based on the background image data.

  The above is the outline of the content reproduction system 1.

[System Configuration of Content Playback System 1]
Based on FIG. 2, the system configuration of the content reproduction system 1 which is a preferred embodiment of the present invention will be described. FIG. 2 is a diagram showing a system configuration of the content reproduction system 1 which is a preferred embodiment of the present invention. The content reproduction system 1 is composed of a computer 10, a reproduction terminal 100, and a public line network (Internet network, third generation, fourth generation communication network, etc.) 5, and audio data for reproducing audio in order to view content. And a computer system for reproducing drawing data for drawing on a background image.

  In FIG. 1, the number of computers 10 and playback terminals 100 is not limited to one and can be changed as appropriate. Further, the computer 10 and the playback terminal 100 are not limited to actual devices, and may be virtual devices. Each process described below may be realized by either or both of the computer 10 and the playback terminal 100.

  The computer 10 is the above-described computer device having the functions described below.

  The playback terminal 100 is the above-described computer device having the functions described below.

[Description of each function]
Based on FIG. 3, the function of the content reproduction system 1 which is a preferred embodiment of the present invention will be described. FIG. 3 is a functional block diagram of the computer 10 and the playback terminal 100.

  The computer 10 includes a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), and the like as the control unit 11, and the communication unit 12 can communicate with other devices (cameras and sensors). For example, a device compatible with WiFi (Wireless Fidelity) compliant with IEEE802.11. The computer 10 also includes a data storage unit such as a hard disk, a semiconductor memory, a recording medium, or a memory card as the storage unit 13. Further, the computer 10 includes various devices such as a device that performs speech recognition, a device that extracts speech, a device that extracts drawing data, and a device that extracts background image data as the processing unit 14.

  In the computer 10, the content acquisition module 20 is realized in cooperation with the communication unit 12 by the control unit 11 reading a predetermined program. Further, in the computer 10, the control unit 11 reads a predetermined program, thereby realizing the storage module 30 in cooperation with the storage unit 13. Further, in the computer 10, the content processing module 40 is realized in cooperation with the processing unit 14 by the control unit 11 reading a predetermined program.

  As with the computer 10, the playback terminal 100 includes a CPU, RAM, ROM, etc. as the control unit 110, a communication unit 120, a device for enabling communication with other devices, and a storage unit 130. A data storage unit is provided. In addition, the playback terminal 100 includes, as the input / output unit 140, a display unit that outputs and displays data and images controlled by the control unit 110, a touch panel that accepts input, a keyboard, an input unit such as a mouse, and the like.

  In the playback terminal 100, the content acquisition module 150 and the standard time acquisition module 151 are realized in cooperation with the communication unit 120 by the control unit 110 reading a predetermined program. In the playback terminal 100, the control unit 110 reads a predetermined program, thereby realizing the storage module 160 in cooperation with the storage unit 130. Also. In the playback terminal 100, the control unit 110 reads a predetermined program, thereby realizing the input receiving module 170 and the content playback module 171 in cooperation with the input / output unit 140.

[Content registration process]
A content registration process executed by the content reproduction system 1 will be described with reference to FIG. FIG. 4 is a flowchart of content registration processing executed by the computer 10. Processing executed by each module described above will be described together with this processing. Note that this processing may be executed by the playback terminal 100 instead of the computer 10. In this case, each module of the playback terminal 100 may execute the process executed by each module of the computer 10.

  The content acquisition module 20 acquires content from other computers, terminal devices, etc. (not shown) (step S10). In step S <b> 10, the content acquisition module 20 acquires a moving image including audio and video as content.

  The content processing module 40 extracts audio data, drawing data, and background image data from the content, respectively, and creates separate files (step S11). In step S11, the content processing module 40 analyzes the speech included in the content and extracts the acoustic features. The extracted acoustic features and the acoustic features of the words described in the recognition dictionary Are compared to extract voice data. The content processing module 40 further divides drawing data and background image data from data obtained as a result of deleting audio data from the content. For example, the content processing module 40 divides data of a predetermined extension (such as jpg, gif, png, bmp) included in the content as background image data. For example, the content processing module 40 recognizes pen drawing (graphics, annotations, emphasis, mathematical formulas, etc.) in the content, and divides the coordinates, time, color, thickness, etc. of the pen drawing as drawing data. Further, the content processing module 40 recognizes a figure to be drawn by CANVAS of the browser, and divides the coordinates, time, color, thickness, etc. of the figure as drawing data. The content processing module 40 creates individual files using the divided audio data, drawing data, and background image data as audio files, drawing files, and background image files.

  Note that the content processing module 40 may extract audio data, drawing data, and background image data by a method other than the above-described example. The content processing module 40 may store drawing data and background image data as drawing data. In this case, drawing data and background image data may be used as drawing data for processing to be described later.

  The storage module 30 stores each created individual file (step S12). In step S12, the storage module 30 stores the number of frames to be reproduced together.

  The content registration process has been described above.

[Content acquisition processing]
Based on FIG. 5, a content acquisition process executed by the content reproduction system 1 will be described. FIG. 5 is a diagram illustrating a flowchart of content acquisition processing executed by the playback terminal 100. Processing executed by each module described above will be described together with this processing.

  The input reception module 170 determines whether or not a content reproduction instruction has been received from the user (step S20). In step S20, the input reception module 170 determines whether or not a predetermined browser activation and content reproduction input have been received.

  In step S20, when the input reception module 170 determines that the input is not received (NO in step S20), the process is repeated.

  On the other hand, in step S20, when the input reception module 170 determines that it has been received (step S20 YES), the content acquisition module 150 acquires content from the computer 10 (step S21). In step S21, the content acquisition module 150 acquires the designated content. The content acquisition module 150 acquires the audio data, drawing data, and background image data of the content as individual files. At this time, the content acquisition module 150 may sequentially acquire the content as a divided file obtained by dividing the content into a plurality of files, or may acquire the content as it is. That is, a divided audio file obtained by dividing each of an audio file, a drawing file, and a background image file into a predetermined number, a divided drawing file, and a divided background image file may be sequentially acquired according to the number of frames, The drawing file and the background image file may be acquired as they are.

  The storage module 160 stores the acquired audio file, drawing file, and background image file as individual files (step S22). In step S22, the storage module 160 also stores the number of frames to be reproduced.

  The content reproduction module 171 performs content reproduction processing described later (step S23). In step S23, the content reproduction module 171 reproduces content based on the audio file, the drawing file, and the background image file. In the content reproduction process, the content reproduction module 171 adjusts the number of frames for reproducing drawing data. The content reproduction module 171 realizes reproduction of drawing data using a browser CANVAS. The content reproduction module 171 reproduces audio data and background image data in accordance with the number of frames to be reproduced. The content reproduction module 171 emits sound based on the audio data, reproduces the drawing data using the browser CANVAS, and displays the background image based on the background image data.

  Note that the content reproduction module 171 may adjust the number of frames to be reproduced in the content reproduction process for either or both of the audio data and the background image data.

  The content acquisition process has been described above.

  A content reproduction process executed by the content reproduction system 1 will be described with reference to FIGS. 6, 7, and 8 are flowcharts of content playback processing executed by the playback terminal 100. Processing executed by each module described above will be described together with this processing. This process is a process for adjusting the reproduction timing of the drawing data. As described above, the playback timing of either or both of the audio data and the background image data may be adjusted by this processing.

  The content reproduction module 171 initializes the frame variable (step S30). In step S30, the content playback module 171 initializes the playback frame variable of the content as “0”.

  The content reproduction module 171 initializes the accumulation variable (step S31). In step S31, the content reproduction module 171 initializes the stacking variable as “0”.

  The content reproduction module 171 substitutes the initial setting for the old time (step S32). In the present embodiment, the initial setting is, for example, “12: 50: 0100”, and in the following description, it is assumed that the value is “100 milliseconds”, which is a value equal to or less than a second.

  The content reproduction module 171 executes the following loop logic until the content reproduction is completed.

  The standard time acquisition module 151 acquires standard time (step S33). In step S33, the standard time acquisition module 151 acquires UTC as standard time from a computer not shown. The standard time acquisition module 151 executes the standard time acquisition at a predetermined time interval (for example, at an interval of 30 milliseconds). That is, this loop logic is performed at a time interval that satisfies this predetermined time interval. The standard time is, for example, “12: 50: 0157”, and in the following description, it is assumed that the value is “157 milliseconds” which is a value equal to or less than a second.

  The content reproduction module 171 calculates the elapsed time from the old time to the standard time (current time) (step S34). In step S34, the content reproduction module 171 calculates the elapsed time “57 milliseconds” by calculating the difference between the standard time “157 milliseconds” and the old time “100 milliseconds”. Note that, at the timing when the reproduction is started, the standard time is acquired before the reproduction process is executed, and the acquired standard time may be used.

  The content reproduction module 171 adds the calculated elapsed time to the accumulated variable (step S35). In step S35, the content reproduction module 171 adds the elapsed time “57 milliseconds” to the accumulated variable “0” to calculate a new accumulated variable “57 milliseconds”.

  The content reproduction module 171 converts the calculated new accumulated variable into the number of frames (step S36). In step S36, the content reproduction module 171 determines that this elapsed time (time difference) from the value obtained by dividing the new accumulated variable “57 milliseconds” by the number of frames per unit time because one frame is 10 milliseconds. The number of frames of drawing data to be reproduced during is calculated. The content reproduction module 171 calculates the number of frames “5” and the remainder “7”. The content reproduction module 171 converts the new accumulated variable “57 milliseconds” into the number of frames “5” and the remainder “7”.

  The content reproduction module 171 adds the converted number of frames to the frame variable (step S37). In step S37, the content reproduction module 171 adds the converted number of frames “5” to the frame variable “0” described above to calculate the frame variable “5”.

  The content reproduction module 171 substitutes the converted remainder for the accumulated variable (step S38). In step S38, the content reproduction module 171 assigns the converted remainder “7” to the accumulation variable to calculate the accumulation variable “7”.

  The content playback module 171 acquires the current time from the playback player (step S39). In step S39, the content reproduction module 171 obtains the content (audio data, drawing data, and background image data) from a reproduction player that reproduces the content. The content playback module 171 acquires the current playback time of the playback player.

  The content reproduction module 171 determines whether or not the frame variable is delayed (step S40). In step S40, based on the calculated frame variable “5” and the number of frames of the current playback time, whether or not there is a delay is determined based on whether or not the current number of frames is greater than this frame variable “5”. to decide.

  In step S40, if the content reproduction module 171 determines that there is no delay (NO in step S40), that is, if the current number of frames is larger than the frame variable “5”, the process of step S42 described later is executed. To do.

  On the other hand, if the content reproduction module 171 determines in step S40 that it is delayed (YES in step S40), that is, if the current number of frames is smaller than the frame variable “5”, the content reproduction module 171 The number of frames corresponding to the delay is added to the current number of frames (step S41). In step S41, the content reproduction module 171 adds the frame variable “5” to the current number of frames. At this time, the content reproduction module 171 displays the drawing data between the added frame variables on the reproduction player.

  Further, the content reproduction module 171 adjusts the remainder in addition to the next time difference. That is, the content reproduction module 171 adds the remaining accumulated time variable to the elapsed time in the next loop, and adjusts the remainder. That is, if the sum of the remainder of the elapsed time in the next loop and the remainder of the current time exceeds 10, the content reproduction module 171 converts it into a frame variable.

  In step S41, when the variable frame is larger than the current number of frames, the content reproduction module 171 does not execute the process of adding to the frame variable in the process of step S37 described above in the next loop, and after step S38. Execute the process.

  The content reproduction module 171 substitutes the new time for the old time (step S42). In step S42, the content reproduction module 171 substitutes the new time based on the frame variable added with the delay amount for the old time.

  The content playback module 171 repeats the playback loop process described above until the playback of the content ends.

  The above is the reproduction process.

  The means and functions described above are realized by a computer (including a CPU, an information processing apparatus, and various terminals) reading and executing a predetermined program. The program is provided, for example, in a form (SaaS: Software as a Service) provided from a computer via a network. The program is provided in a form recorded on a computer-readable recording medium such as a flexible disk, CD (CD-ROM, etc.), DVD (DVD-ROM, DVD-RAM, etc.). In this case, the computer reads the program from the recording medium, transfers it to the internal storage device or the external storage device, stores it, and executes it. The program may be recorded in advance in a storage device (recording medium) such as a magnetic disk, an optical disk, or a magneto-optical disk, and provided from the storage device to a computer via a communication line.

  As mentioned above, although embodiment of this invention was described, this invention is not limited to these embodiment mentioned above. The effects described in the embodiments of the present invention are only the most preferable effects resulting from the present invention, and the effects of the present invention are limited to those described in the embodiments of the present invention. is not.

  1 content playback system, 10 computers, 100 playback terminals

Claims (4)

A computer that reproduces audio data for reproducing audio and drawing data for drawing on a background image in order to view content,
Data storage means in which the audio data and the drawing data are stored in separate files;
Current time acquisition means for acquiring the current time from a computer providing standard time;
The time difference between the acquired current time and the previously acquired time is calculated, and from the value obtained by dividing the time difference by the time of one frame, the number of frames of drawing data to be reproduced during the time difference is calculated. Frame calculation means using a stacked time variable that adds the remainder to the time difference before calculation,
In accordance with the current number of frames and the calculated number of frames, a frame number adjusting unit that adjusts the number of frames for reproducing the drawing data by adding the number of frames for the delay to the current number of frames when a delay occurs, A computer comprising:   The computer according to claim 1, wherein reproduction of the drawing data is realized using a CANVAS tag of a browser. A content reproduction method executed by a computer that reproduces audio data for reproducing audio and drawing data for drawing on a background image in order to view content,
The audio data and the drawing data are stored in separate files;
Obtaining the current time from a computer that provides standard time;
The time difference between the acquired current time and the previously acquired time is calculated, and from the value obtained by dividing the time difference by the time of one frame, the number of frames of drawing data to be reproduced during the time difference is calculated. Using a stacked time variable that adds the remainder of the time to the time difference before calculation;
A step of adjusting the number of frames for reproducing the drawing data by adding the number of frames corresponding to the delay to the current number of frames when a delay occurs according to the calculated number of frames; Playback method. In order to view content, a computer that reproduces audio data for reproducing audio and drawing data for drawing on a background image,
A step in which the audio data and the drawing data are stored in separate files;
Obtaining the current time from a computer providing standard time;
The time difference between the acquired current time and the previously acquired time is calculated, and from the value obtained by dividing the time difference by the time of one frame, the number of frames of drawing data to be reproduced during the time difference is calculated. Using a stacking time variable to add the remainder of to the time difference before calculation,
In order to execute the step of adjusting the number of frames for reproducing the drawing data by adding the number of frames for the delay to the current number of frames when a delay occurs according to the current number of frames and the calculated number of frames Computer readable programs.