WO2013151087A1

WO2013151087A1 - Data processing device and data processing program

Info

Publication number: WO2013151087A1
Application number: PCT/JP2013/060179
Authority: WO
Inventors: 八木　健; 幹也田中; 慶木谷; 敏赤坂; 貴次青山; 一仁荘司; 諭石山
Original assignee: 株式会社ニコン
Priority date: 2012-04-06
Filing date: 2013-04-03
Publication date: 2013-10-10

Abstract

This data processing device is provided with: an input unit for inputting multimedia container files and vibration data files; a file manipulation unit for adding vibration data contained in the vibration data files to the multimedia container files; and an output unit for outputting the multimedia container files to which the vibration data has been added.

Description

Data processing apparatus and data processing program

The present invention relates to a data processing apparatus and a data processing program.
This application claims priority based on Japanese Patent Application No. 2012-87311 for which it applied on April 6, 2012, and uses the content here.

An apparatus as disclosed in Patent Document 1 may be provided with a vibration unit for generating vibration. As such a vibration part, for example, an eccentric motor is known.

JP 2010-086089 A

However, in such a device, the vibrating part simply vibrates at a predetermined timing. For this reason, there is a problem that it is not possible to obtain a production effect due to vibrations such as generating vibrations suitable for images and sounds.

An object of an aspect of the present invention is to provide a data processing apparatus and a data processing program capable of processing vibration data for obtaining a rendering effect by vibration.

One aspect of the present invention includes an input unit that inputs a multimedia container file and a vibration data file, a file operation unit that adds vibration data included in the vibration data file to the multimedia container file, and the vibration data. Is a data processing apparatus including an output unit that outputs a multimedia container file to which is added.

Another aspect of the present invention includes an input unit for inputting a multimedia container file including vibration data, a file operation unit for extracting the vibration data from the multimedia container file, and a vibration data file from the extracted vibration data. A data processing apparatus including an output unit for outputting.

According to another aspect of the present invention, an input unit for inputting a multimedia container file including vibration data and generation of vibration based on the vibration data in synchronization with reproduction of image data or audio data included in the multimedia container file The data processing device includes a data operation unit that outputs data to be generated and a vibration generation unit that generates vibration based on the data that generates the vibration output from the data operation unit.

According to the aspect of the present invention, it is possible to obtain an effect that vibration data for obtaining a rendering effect by vibration and image data or audio data can be stored in one multimedia container file.

It is a block diagram which shows the structure of one Embodiment of this invention. It is an external view which shows the structure of the vibration generation part shown in FIG. It is explanatory drawing which shows the example of the apparent exercise | movement from which a user can obtain a movement feeling. It is explanatory drawing which shows the file structure of the VIB file shown in FIG. It is explanatory drawing which shows the file structure of the VIB file shown in FIG. It is explanatory drawing which shows the file structure of the VIB file shown in FIG. It is a flowchart which shows operation | movement of the data processor shown in FIG. It is a figure which shows an example of a user operation screen. It is a flowchart which shows operation | movement of the data processor shown in FIG.

Hereinafter, a vibration data processing apparatus according to an embodiment of the present invention will be described with reference to the drawings. In the following description, a production effect obtained by the user by generating a vibration and a sense of localization and a sense of movement is referred to as a “vibration effect”. Here, the sense of orientation is phantom sensation, that is, “when two points of the skin are vibrated (stimulated) at the same time, it feels as if at a specific position between the two points. Sense that the user feels that there is no localization. In addition, the sense of movement is an apparent movement, that is, “when two points on the skin are vibrated (stimulated) with a phase difference and an output difference, the localization of vibration is moving. The feeling that the user feels ".

Hereinafter, PCM (Pulse Code Modulation) data as uncompressed waveform data that has been processed so as to obtain a vibration effect is referred to as “VPCM data”. In addition, for example, AAC (Advanced Audio Coding) data as compressed waveform data that has been processed so as to obtain a vibration effect is referred to as “VAAC data”. Note that tone bursts may be included in these waveform data so that vibration effects can be obtained more.

Hereinafter, data including waveform data (for example, VPCM data, VAAC data) processed so as to obtain a vibration effect in accordance with a predetermined format (described later with reference to FIGS. 4 to 6). Is referred to as “vibration data (VIB data)”. Here, VIB is an abbreviation that indicates vibration. Hereinafter, a file including vibration data is referred to as a “vibration file (VIB file, vibration data file)”.

FIG. 1 is a block diagram showing the configuration of the embodiment. In this figure, reference numeral 1 denotes a data processing device that processes vibration data, audio data, and image data. Reference numeral 2 denotes an operation unit including a keyboard, a pointing device, a controller, and the like for a user to perform an input operation. Reference numeral 3 denotes a display unit configured by a display device. Reference numeral 4 denotes an audio output unit composed of a speaker that emits audio. Reference numeral 5 denotes a vibration generating unit that generates vibration in order for the user to obtain an effect produced by vibration. Reference numeral 6 denotes an MOV file that is one of the multimedia container files, and includes at least one of image data and audio data.

Reference numeral 7 denotes an MOV file in which vibration data is added to one or both of image data and audio data to form one multimedia container file.
Reference numeral 8 denotes a VIB file in which vibration data is recorded. Reference numeral 9 denotes an MOV file generated by the data processing apparatus 1 by adding the vibration data recorded in the VIB file 8 to the MOV file 6 that does not include vibration data. Reference numeral 10 denotes a VIB file generated by extracting only vibration data from the MOV file 7 including vibration data.

Reference numeral 11 denotes an application execution unit that inputs the

MOV files

6 and 7 and the VIB file 8 and executes the application. Reference numeral 12 denotes an input unit for inputting data by reading a file. Reference numeral 13 denotes a vibration processing unit that processes vibration data among the data input from the input unit 12. Reference numeral 131 denotes a file operation unit that performs file operations such as addition and extraction of vibration data with respect to the MOV file. Reference numeral 132 denotes a data operation unit that performs data operations such as extracting vibration data from the MOV file and outputting data for generating vibration.

Numeral 14 is an audio processing unit that processes audio data among the data input in the input unit 12. Reference numeral 15 denotes an image processing unit that processes image data among the data input from the input unit 12. Reference numeral 16 denotes an output unit for writing data to the MOV file 9 and the VIB file 10.

Next, the configuration of the vibration generating unit 5 shown in FIG. 1 will be described with reference to FIG. FIG. 2 is an external view showing the configuration of the vibration generator 5 shown in FIG. For example, the vibration generating unit 5 includes a vibrator (vibration device) 51 (FL: front left), a vibrator 52 (FR: front right), a vibrator 53 (RL: rear left), and a vibrator at four corners of the casing. 54 (RR: rear right). The vibration generating unit 5 vibrates the casing of the vibration generating unit 5 by vibrating the vibrators 51 to 54 based on VPCM data or VAAC data.

FIG. 3 is a diagram illustrating an example of an apparent exercise that allows the user to obtain a sense of movement. Here, for the sake of explanation, a coordinate system (x, y) = (− 1.0 to +1.0, −1.0 to +1.0) with the center of the casing of the vibration generating unit 5 as the origin is defined. .
For example, the vibrator 51 (channel 1) is arranged at coordinates (−0.9, +0.9). For example, the vibrator 52 (channel 2) is arranged at coordinates (+0.9, +0.9). For example, the vibrator 53 (channel 3) is arranged at coordinates (−0.9, −0.9). For example, the vibrator 54 (channel 4) is arranged at coordinates (+0.9, −0.9).
The type of each vibration unit may be a voice coil motor (VCM) or a speaker.

In FIG. 3, as an example, the user feels that the vibration is moving linearly from the start point coordinates (+0.4, +0.2) to the end point coordinates (−0.3, −0.55). Obtainable. Hereinafter, when the vibration localization moves from the start point coordinates to the end point coordinates, information including the start point coordinates and the end point coordinates is referred to as “vector information”. Here, the start point coordinates and the end point coordinates may be expressed by relative coordinates. The vector information may include information indicating a vibration time (ringing time [ms]) for which the vibration unit continues to vibrate.

Next, the vibration file (VIB file) shown in FIG. 1 will be described with reference to FIGS. 4, 5 and 6 are views showing the file structure of the VIB file. Here, although divided into three drawings, the actual VIB file is composed of one file. The VIB file is a file having a chunk structure. The VIB file indicates information indicating a list type “INFO”, information indicating a creation date, information indicating a file owner, information indicating a file creator, information indicating a title (title), and a comment. Information.

The VIB file includes information indicating the list type “vibl”, information indicating the version of the format of the VIB data, information indicating the creation time (creation time), information indicating the update date, and update time Information indicating (update time), information indicating an encoding tool (such as a tool name), information indicating a genre (for example, video, audio, haptic, game), and information indicating a manufacturer code of the VIB file , Information indicating protection information (protection presence / absence), information indicating vibration time (reproduction time), vibrator device information (for example, identification information of the manufacturer of the vibration unit), and information indicating vibration ch (channel) allocation Information indicating a vibration frequency band (single frequency, wide band), information indicating a user comment, and GPS (Global Posit) oning System) is included and information (geotagging).

The VIB file also includes information indicating the version of the VIB file (VIB data) itself and the type of waveform data (for example, PCM format) included in the data area (the area indicated by “ckData” in FIG. 6). Information indicating VPCM, VAAC having AAC format), information indicating the number of vibration channels (channels), information indicating a sampling rate (sampling frequency of waveform data), and sampling bits (quantization bits of waveform data) Information).

Hereinafter, information indicating a creation date, information indicating a creation time (creation time), information indicating an update date, information indicating an update time (update time), information indicating a format version of VIB data, , Information indicating the version of the VIB file (VIB data) itself, information indicating the type of waveform data included in the data area, information indicating the number of vibration channels, information indicating the sampling rate, and sampling bits Information indicating file owner, information indicating file creator, information indicating title, information indicating comment, information indicating encoding tool, information indicating genre, and manufacturer code of VIB file Information indicating protection information, information indicating vibration time, vibrator device information, information indicating vibration ch allocation, and vibration circumference And information indicating the number band, and the information indicating the user comments, together with GPS information (geotagging), referred to as "VIB information".

In addition, hereinafter, information indicating the arrangement (coordinates) of the vibrator in the vibration generating unit 5 (see FIGS. 2 and 3), information indicating the number of vibration ch (channels), and a sampling rate (sampling frequency of PCM data) ) And information indicating sampling bits (the number of quantization bits of PCM data) are collectively referred to as “configuration information”. Here, the number of vibration channels (channels), the sampling rate, and the sampling bit are determined based on the VIB information.

Next, a processing operation in which the data processing apparatus 1 shown in FIG. 1 performs file operations on the MOV file and the VIB file will be described with reference to FIG. First, the application execution unit 11 displays a user operation screen on the display unit 3 (step S1). The user operation screen displayed here is, for example, the screen shown in FIG. FIG. 8 is a diagram illustrating an example of a user operation screen. The user operation screen includes a radio button for selecting a processing function, and the user operates the operation unit 2 to select one of the processing functions.

The selectable processing functions include the following two functions.
The first function is to add vibration data of a VIB file (VIB file 8 in the example shown in FIG. 1) to an MOV file (MOV file 6 in the example shown in FIG. 1) that does not include existing vibration data. Then, it is an “add” function for generating a new MOV file (MOV file 9 in the example shown in FIG. 1).
The second function is an “extraction” function that extracts vibration data from the MOV file including the vibration data (MOV file 7 in the example of FIG. 1) and writes it to the VIB file (VIB file 10 in the example shown in FIG. 1). is there.

Further, the user operation screen is provided with an input file name input field for inputting an input file name and an output file name input field for inputting an output file name.
When the function of the selected process is “add”, the file name of the MOV file 6 that does not include vibration data and the file name of the VIB file 8 are input to the input file name input field as input file names. Further, the file name of the new MOV file 9 is input to the output file name input field as the output file name.

On the other hand, when the function of the selected process is “extract”, the file name of the MOV file 7 including the vibration data is input as an input file name in the input file name input field. Further, the file name of the new VIB file 10 is input to the output file name input field as the output file name.

Then, when the user presses a “start” button in the user operation screen, the application execution unit 11 reads input information (selected function, input / output file name) in the user operation screen (step S2).

Next, the application execution unit 11 determines whether or not the selected function is “addition” (step S3).
As a result of this determination, if the function is an additional function, the application execution unit 11 instructs the input unit 12 to specify the MOV file specified by the MOV file name input in the input file name input field (here, the MOV file 6). ) Is read. In response, the input unit 12 reads the MOV file 6 (step S4).
Subsequently, the application execution unit 11 instructs the input unit 12 to read the VIB file (here, the VIB file 8) specified by the VIB file name input in the input file name input field. In response to this, the input unit 12 reads the vibration data of the VIB file 8 (step S5).

Next, when the file reading is completed, the application execution unit 11 instructs the file operation unit 131 to add vibration data to the MOV file. In response to this, the file operation unit 131 inputs the data of the MOV file 6 and the VIB file 8 input at the input unit 12, and adds vibration data to the MOV file (step S6).
At this time, the file operation unit 131 adds vibration data to the user area of the MOV file. As a result, even an application that cannot recognize vibration data can be processed using only image data or audio data in the MOV file to which the vibration data is added, so that the compatibility of the MOV file is maintained. be able to.

Next, the application execution unit 11 instructs the output unit 16 to output a new MOV file. In response to this, the output unit 16 inputs the MOV file to which the vibration data is added in the file operation unit 131, generates the output file input in the output file name input field, and outputs the output file (here, the MOV file). The data of the MOV file with the vibration data added is written in the file 9) (step S7). Thereby, the MOV file 9 in which the vibration data is written in the user area of the MOV file is generated.

Next, when the selected function is not “Add” (when “Eject” is selected), the application execution unit 11 uses the MOV file name input in the input file name input field to the input unit 12. An instruction to read the specified MOV file (here, MOV file 7) is issued. In response to this, the input unit 12 reads the MOV file 7 (step S8).

Next, when the reading of the file is completed, the application execution unit 11 instructs the file operation unit 131 to extract vibration data from the MOV file. In response to this, the file operation unit 131 inputs the data of the MOV file 7 input at the input unit 12 and extracts only the vibration data (step S9). At this time, the file operation unit 131 extracts vibration data from the user area of the MOV file.

Next, the application execution unit 11 instructs the output unit 16 to output a new VIB file. In response to this, the output unit 16 inputs the vibration data extracted from the MOV file in the file operation unit 131 and converts it into data in the VIB file format. And the output part 16 produces | generates the output file input into the output file name input column, and writes the vibration data of a VIB file format in the output file (here VIB file 10) (step S10). Thereby, the VIB file 10 is generated.

As described above, since vibration data written in the VIB file format can be added to an MOV file composed of either or both of image data and sound data not including vibration data, image data and sound data And vibration data can be stored in one multimedia container file (MOV file). It is also possible to extract only vibration data from a multimedia container file (MOV file) including vibration data.

Next, with reference to FIG. 9, an operation of playing back a multimedia container file (MOV file) in which vibration data is written in the user area will be described. FIG. 9 is a flowchart showing an operation in which the data processing apparatus 1 shown in FIG. 1 reproduces a multimedia container file (MOV file) in which vibration data is written in the user area.

First, when the user operates the operation unit 2 to perform playback of the MOV file 7 including vibration data, the application execution unit 11 instructs the input unit 12 to read the MOV file. . In response to this, the input unit 12 reads the MOV file 7 including the vibration data that has been reproduced (step S11).
Then, the application execution unit 11 instructs the vibration processing unit 13, the sound processing unit 14, and the image processing unit 15 to execute the reproduction process. In response to this, the image processing unit 15 takes out the image data from the data of the MOV file 7 input at the input unit 12, converts it to the format displayed on the display unit 3, and displays the display unit via the output unit 16. The image is reproduced by outputting to 3 (step S12).

In synchronism with this image reproduction, the audio processing unit 14 extracts audio data from the data of the MOV file 7 input by the input unit 12. Then, it is converted into a format that can be output from the audio output unit 4, and output to the audio output unit 4 via the output unit 16, thereby reproducing audio in synchronization with image reproduction (step S13).

Further, in synchronization with the image reproduction, the data operation unit 132 of the vibration processing unit 13 extracts vibration data from the data of the MOV file 7 input at the input unit 12. Then, it is converted into a format capable of generating vibration in the vibration generating unit 5 and output to the vibration generating unit 5 via the output unit 16 to generate vibration in synchronization with image reproduction (step S14). .
The vibration data controls the localization of vibration and movement of the vibration source in accordance with image reproduction. For example, the vibration localization and the position of the vibration source are controlled in accordance with the movement of the object in the image. As a result, when a car is displayed in the image, by controlling the position and amount of movement of the vibration source according to the movement of the car in the image, it is possible to create a scene where the user is on the spot. Can do.

Next, the application execution unit 11 determines whether or not the end of the MOV file 7 has been read (step S15). If the end is not the end, the next data is read and the image, sound, and vibration are reproduced in synchronization. repeat. On the other hand, if the end of the MOV file 7 has been read, the process ends.

As described above, since the image, the sound, and the vibration can be reproduced in synchronization with each other, it is possible to obtain the effect of the vibration in addition to the image and the sound. In addition, the user can feel more reality with respect to the image and the sound than before due to the effect.

In the above description, an example in which a MOV file is used as an example of a multimedia container file has been described. However, a multimedia container file to which vibration data is added is not limited to an MOV file, and other existing It may be a multimedia container file. That is, as long as a user area can be provided and vibration data can be written in this user area, the multimedia container file can be anything.

As described above, since the image, sound and vibration are stored in one multimedia container file, the vibration data can be reproduced in synchronization with the image data and the sound data, and the effect of vibration can be obtained. be able to. In addition, the user can feel more reality with respect to the image and the sound than before due to the effect. In addition, since vibration data is stored in an existing multimedia container file, images and sound can be played back using existing playback technology, and the configuration of the data processing apparatus can be simplified. .

A program for realizing the function of the processing unit in FIG. 1 is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read into a computer system and executed to execute file operation processing and Data manipulation processing may be performed.
Here, the “computer system” includes an OS and hardware such as peripheral devices. The “computer system” includes a WWW system having a homepage providing environment (or display environment). The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM or a CD-ROM, and a hard disk incorporated in a computer system. Further, the “computer-readable recording medium” refers to a volatile memory (RAM) in a computer system that becomes a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. In addition, those holding programs for a certain period of time are also included.

The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, what is called a difference file (difference program) may be sufficient.

As described above, the embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes a design and the like within the scope not departing from the gist of the present invention.

DESCRIPTION OF SYMBOLS 1 ... Data processing apparatus, 11 ... Application execution part, 131 ... File operation part, 132 ... Data operation part, 3 ... Display part, 4 ... Audio | voice output part, ... Vibration generator, 6, 7, 9 ... MOV file, 8, 10 ... VIB file.

Claims

An input unit for inputting a multimedia container file and a vibration data file;
A file operation unit for adding vibration data included in the vibration data file to the multimedia container file;
An output unit that outputs a multimedia container file to which the vibration data is added.
The data processing apparatus according to claim 1, wherein the vibration data is added to a user area provided in advance in the multimedia container file.
An input unit for inputting a multimedia container file including vibration data;
A file operation unit for extracting the vibration data from the multimedia container file;
An output unit that outputs a vibration data file from the extracted vibration data.
An input unit for inputting a multimedia container file including vibration data;
A data operation unit that outputs data that generates vibration based on the vibration data in synchronization with reproduction of image data or audio data included in the multimedia container file;
A data processing apparatus comprising: a vibration generating unit that generates vibration based on the data that generates the vibration output from the data operation unit.
On the computer,
An input step for inputting a multimedia container file and a vibration data file;
A file operation step of adding vibration data included in the vibration data file to the multimedia container file;
An output step of outputting a multimedia container file to which the vibration data is added.
The data processing program according to claim 5, wherein the vibration data is added to a user area provided in advance in the multimedia container file.
On the computer,
An input step for inputting a multimedia container file including vibration data;
A file operation step of retrieving the vibration data from the multimedia container file;
A data processing program for executing an output step of outputting a vibration data file from the extracted vibration data.
On the computer,
An input step for inputting a multimedia container file including vibration data;
A data operation step of outputting data that generates vibration based on the vibration data in synchronization with reproduction of image data or audio data included in the multimedia container file;
A data processing program for executing a vibration generation step for generating vibration based on the data for generating vibration generated by the data operation step.