JP2000341642A - Synchronization processor for digital picture and sound data - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a synchronization processor capable of moving and deforming an illumination on a digital picture synchronously with sound data, and realizing reproduction regardlessly of the difference of the performance of computers without requiring any special set or technique. SOLUTION: The illumination to be moved and deformed synchronously with sound data is virtually prepared on a movie by using a synchronization control function 1440 for performing real time management control data a database 1500 including a control table for controlling synchronization. Thus, the movie in which the illumination on a digital picture is moved and deformed synchronously with the sound data can be prepared without requiring any special set or technique. Moreover, the proper timing of synchronization is calculated as necessary by using the synchronization control table 1532 so that the difference of the qualities of movies due to the difference of the performance of computers can be reduced, and the fine adjustment or partial change of the illumination can easily be realized.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an apparatus for synchronizing digital images and audio data, and more particularly, to a synchronous processing apparatus for reproducing digital images and audio data, regardless of the difference in performance between computers for reproducing digital images and audio data. The present invention relates to a digital image and audio data synchronization processing device that enables moving and deforming illumination on a digital image.

[0002]

2. Description of the Related Art As a conventional technique for creating digital data that moves and deforms lighting on an image in synchronization with audio data, a scene in which lighting is moved and deformed in synchronization with audio data is actually used as a video camera or the like. 2. Description of the Related Art There is a known prior art in which an image is captured using a computer and the captured data is converted into a data format that can be reproduced on a computer. Generally, photographing a scene in which a portion to be illuminated is changed while synchronizing with audio requires a dedicated set and staff with special skills.

In addition, since the data amount of a playback movie created by the above-described conventional technique is large, a process of compressing the data is required. Compressing the data causes quality problems in the movie, such as the gradation of the illumination being coarsened depending on the performance of the computer, dropping frames violently, and sound skipping. Further, in the above-described conventional technology, it is difficult to finely adjust or partially change the illumination of a movie once created, and it is necessary to start shooting again depending on the amount of change.

[0004] As a prior art relating to a method of synchronizing digital image and audio data, for example, Japanese Patent No. 2607 is disclosed.
A technology described in, for example, Japanese Patent Application No. 561 is known. This prior art relates to speech synchronization animation, which synchronizes the behavior of the lips of a human character with voice in units of one character, and includes the time from the start to the end of the pronunciation of the character and the start and end of the animation corresponding to the character. The processing is performed on the assumption that the time until the time is the same. For this reason, this prior art has a quality problem such as an unnatural interval being inserted into the sound when the animation time is delayed due to the performance of the computer.

[0005] As another conventional technique, for example, a technique described in Japanese Patent Application Laid-Open No. 10-243351 is known. This prior art relates to a video reproducing apparatus, which separates a movie that integrates audio and video into audio and video, detects unvoiced sound portions and portions where the video is stationary, and fast-forwards the audio or video. In this way, the audio and the moving image are synchronized. However, while this method according to the prior art is effective for fine-tuning the synchronization, in order to prepare a movie in which audio and video are integrated in advance, a special set as described above and a special set are used. In need of skilled staff.

[0006]

The prior art described above is
There is a problem that a special set or technique is required for shooting, and further, there is a problem that the quality of a movie is deteriorated due to a difference in computer performance, and it is difficult to change lighting.

SUMMARY OF THE INVENTION An object of the present invention is to solve the above-mentioned problems of the prior art, and to create a movie that moves and deforms lighting on a digital image in synchronization with audio data without requiring a special set or technique, and Another object of the present invention is to provide a synchronous processing apparatus for digital image and audio data, which can reduce a difference in quality of a movie due to a difference in performance between the digital images and audio data.

[0008]

According to the present invention, an object of the present invention is to provide an apparatus for synchronizing a digital image and audio data, which represents illumination on a digital image in synchronization with audio data. Control means having a real-time management control function for synchronizing with the audio data, image data to illuminate, audio data to synchronize with the illumination,
A synchronization control table that defines the synchronization timing and the position and size of the lighting, and image data, audio data,
A database in which a playback content control table defining the correspondence between the synchronization control tables is registered, wherein the synchronization control means moves in synchronization with audio data based on information in the table of the database. This is achieved by virtually generating illumination for the digital image to be deformed.

Further, the object is that the synchronization control means comprises:
Calculating the appropriate movement of the lighting on the digital image, the time of the start of the deformation, and the real-time management control function monitors a timer for synchronizing the lighting on the digital image with the audio data. Is achieved by

According to the present invention, a movie that moves and deforms lighting on a digital image in synchronization with audio data can be created without requiring a special set or technique by providing the above-described configuration. The difference in movie quality due to the difference in computer performance is small, and the lighting can be easily adjusted.

[0011]

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram showing an embodiment of a synchronous processing apparatus for digital image and audio data according to the present invention.

FIG. 1 is a block diagram showing a system configuration of a synchronous processing apparatus for digital image and audio data according to an embodiment of the present invention, FIG. 2 is a diagram for explaining a configuration example of a reproduction content control table, and FIG. FIG. 4 is a diagram illustrating an example of a configuration of a control table, FIG. 4 is a diagram illustrating an example of an interface screen, FIG. 5 is a flowchart illustrating a processing operation of a control process according to an embodiment of the present invention, and FIG. And FIG. 7 is a diagram illustrating a synchronization control table and an illumination mask control value calculated in the control process. 1 to 3, 1100 is a user input unit, 1200 is a screen display unit, 1300 is a sound output unit, 1400 is a content control unit, 1410 is an input control function, 1420 is a screen display function, 1430 is a sound reproduction function, and 1440. Is a synchronization control function, 1441 is a real-time management function, 1500 is a synchronization control database, 1510 is image data, 1520 is audio data, 1530 is a control table, 1531 is a playback content control table, and 1532
Is a synchronization control table.

A system for realizing the synchronization control of the present invention shown in FIG. 1 includes a user input unit 1100 for receiving a user's input, and a screen display unit 12 for displaying a still image or a moving image.
00, an audio output unit 1300 that outputs audio data,
A content control unit 1400 for controlling selection of reproduction content, synchronization of digital images and audio, and a synchronization control database 15 including data and tables required for synchronization control
00 is provided. In the above description, the user input unit 1100 is an input device such as a mouse and a keyboard, and the screen display unit 1200 is an output device such as a CRT and a display using a liquid crystal device. Also, the audio output unit 13
Reference numeral 00 denotes an output device such as a speaker.

The content control section 1400 includes an input control function 1410, a screen display function 1420, a sound reproduction function 1430, and a synchronization control function 1440. Then, the input control function 1410 controls the user input unit 1
The input content received from the control unit 100 is determined, and a request is made to the synchronization control function 1440 for selection of a reproduction content, reproduction / stop of audio, and the like. The screen display function 1420 receives a display request from the synchronization control function 1440, and displays an image on the screen display unit 1200 using a screen display function of an operating system (hereinafter, referred to as an OS). Audio playback function 143
0, upon receiving a sound reproduction request from the synchronization control function 1440, the sound output unit 1300 using the sound reproduction function of the OS.
Then, when a request to stop the sound reproduction is received, the sound reproduction is stopped. Furthermore, synchronization control function 1
Reference numeral 440 denotes a timing at which the illumination mask is moved / deformed by controlling an illumination mask for virtually creating the illumination and monitoring a timer for synchronizing the audio data with the illumination using the real-time management function 1411. Measure.

The synchronization control database 1500 includes image data 1510 as a background image, audio data 1520,
The control table 1530 is stored. The control table 1530 is the playback content control table 1
531 and a synchronization control table 1532.

[0016] The reproduction content control table 1531 includes:
As shown in FIG. 2, a selection key 2100 representing the user's content selection, a playback content name 2200, an image data name 2300, an audio data name 2400, a synchronization control table name 2500, and a standard value 26 of the movie tempo
00 and the standard value 27 of the amount of change when moving the illumination mask
It is configured to have information corresponding to the content, which is set to 00. The tempo 2600 of the movie is specified by the number of frames to be displayed per second (frames / second), and the amount of change 2700 when moving the illumination mask is specified by the amount of movement of one frame (pixels / frame). Then, the standard value 2600 of the movie tempo and the standard value 2700 of the change amount when the illumination mask is moved are determined by moving the illumination mask in synchronization with the sound.
It is used to calculate the optimal movement / deformation start time of the illumination mask for deformation.

In the synchronization control table 1532, one content registered in the reproduction content control table 1531 is assigned one by one. Then, as shown in FIG. 3, each synchronization control table 1532 includes a synchronization point number 3100, an arrival time 3200 of the synchronization point, a center coordinate position 3300 of the illumination mask,
It is configured by storing information of each synchronization point as a set of the size 3400 of the illumination mask.

Next, in the embodiment of the present invention configured as described above, the synchronization control process using the synchronization control function 1440 and the synchronization control database 1500 will be described with reference to the flowcharts shown in FIGS. This will be described with reference to the example interface screen shown. First, with reference to FIG. 5, a control process up to the start of audio reproduction will be described.

(1) First, the synchronization control function 1440 uses the playback content control table 1531 to generate an interface for selecting content, and the screen display function 1
420 is requested to display it. Screen display function 1420
Displays a content selection interface screen according to a request from the synchronization control function 1440. The content selection interface screen includes, for example, a content name list 4100, a display 4200 for confirming the content, and a button 4300 for selecting the content, as shown in FIG. 5
001).

(2) When the user performs an input for designating a content name on the content selection interface screen shown in FIG. 4A, the content name selected by the user is changed to a selection key by the input control function 1410 of the content control unit. And is notified to the synchronization control function 1440. Here, it is assumed that the selection key K has been notified to the synchronization control function 1440 (steps 5002 and 5003).

(3) Next, the synchronization control function 1440 requests the screen display function 1420 to display an interface for reproducing sound as shown in FIG. 4B, and the screen display function 1420 In response to the request from 1440, the audio reproduction interface is displayed (step 5).
004).

(4) Subsequently, the synchronization control function 1440
The image data (Pic
[K]) is requested to the screen display function 1420 to display the background image of the illumination mask.
The image data (Pic [K]) is displayed on the screen display unit 1200 (step 5005).

(5) When a voice playback command is received from the user via the voice playback interface screen shown in FIG. 4B, the synchronization control table Cnt [K] assigned to the selection key K is read, and the synchronization control table is read. (Cnt
[K]), the center position (X
[0], Y [0]), width Width [0], height Height
[0] is set as the illumination mask, and the screen display function 1420 is requested to display the illumination mask. Screen display function 1
420, according to a request from the synchronization control function 1440,
Display the illumination mask at the start position (Step 500)
6 to 5008).

(6) Next, the synchronization control function 1440 requests the screen display function 1420 to display an interface screen for stopping audio, as shown in FIG.
The screen display function 1420 follows the request in FIG.
The interface screen for stopping the voice as shown in (5) is displayed (5009).

(7) Then, the synchronization control function 1440
The synchronization point Cpoint where the current illumination mask is located is initialized, the audio data (Sound [K]) is read, and the reproduction is started. At the same time, the synchronization control function 1440 requests the real time management function 1441 to start timer monitoring,
The real time management function 1441 starts monitoring the timer. Synchronous control during audio reproduction is performed in an audio reproduction loop (steps 5010 to 5012).

Next, with reference to the flow shown in FIG. 6, a description will be given of the synchronization control process in the audio reproduction loop executed subsequently to the above.

(1) When the audio reproduction loop is started, the synchronization control function 1440 checks whether or not the audio data is being reproduced. Returning to step 5004, an audio reproduction interface screen is displayed. If the audio is being reproduced, the current synchronization point number Cpoint is compared with the final synchronization point number (END) (step 600).
1, 6002).

(2) In the comparison in step 6002, the current synchronization point number Cpoint is changed to the last synchronization point number (E
If it is equal to ND), the illumination mask is not moved or deformed, and the current position and size are maintained.

(3) In the comparison in step 6002, the current synchronization point number Cpoint is changed to the final point number (EN
D), the synchronization control function 1440 requests the real-time management function 1441 to acquire the elapsed time from the start of the sound reproduction, and receives the notification of the elapsed time Ctime from the real-time management function 1441. By comparing the elapsed time Ctime with the movement start time to the synchronization point, selection of a synchronization point to which the illumination mask is moved starts. To select the destination of the illumination mask, first, 1 is added to the counter of the synchronization point number, and the current synchronization point Cpo is added.
Preceding travel time from int to the next synchronization point point
This is performed by calculating pre [point] (steps 6003 to 6005).

The preceding moving time of the synchronization point pre
Can be calculated as follows using the standard value 2600 of the movie tempo defined in the playback content control table 1531 and the standard value 2700 of the change amount when the illumination mask is moved. That is, assuming that the standard value of the movie tempo is tmpo (frames / second) and the amount of change when moving the illumination mask is Δp (pixels / frame), the number of pixels moved per second is calculated as tmpo × Δp. And the preceding travel time pre is (number of pixels moved to the destination synchronization point) / (tmpo ×
Δp).

(4) Synchronization point in step 6005
After calculating the preceding travel time pre [point] to the point, the elapsed time Ctime and the arrival time t [point] of the synchronization point point
Minus the preceding travel time pre [point] (comparison with the travel start time to the synchronization point point, and the elapsed time C
If the time is smaller than the movement start time to the synchronization point point, the synchronization control function 1440 determines that the time to start the movement to the synchronization point point has not been reached, and does not move or deform the illumination mask. The position and size are maintained (step 6006).

(5) If the elapsed time Ctime is greater than or equal to the movement start time to the synchronization point point in the comparison in step 6006, it is determined that preparation for reproducing the sound after the synchronization point point point is necessary. . Then, in order to narrow down the synchronization point of the movement destination, the preceding movement time pre [point + 1] to the synchronization point point + 1 is calculated, and the movement start time to the synchronization point point and the movement start time to the synchronization point point + 1 are compared. (Steps 6007 and 6008).

(6) Synchronization point point and synchronization point
The difference in point arrival time from point +1 is small, and
When the distance between the synchronization point point and the synchronization point point + 1 is large, the movement start time to the synchronization point point + 1 may be earlier than the movement start time to the synchronization point point. Since this causes a synchronization deviation, if the movement start time to the synchronization point point is longer than the movement start time to the synchronization point point +1 in the comparison in step 6008, the movement is performed according to the arrival time at the synchronization point. Select the previous synchronization point. That is, in this case, the selection of the destination synchronization point based on the arrival time at the synchronization point is determined by the elapsed time Ctime
Is compared with the arrival time t [point] at the synchronization point point, the elapsed time Ctime is greater than or equal to the arrival time t [point] at the synchronization point point, and the synchronization point poin
If t is not the final synchronization point END, it is determined that the synchronization has shifted, and in order to adjust the synchronization shift,
One is added to the synchronization point number counter point, and the elapsed time Ctime and the arrival time t [poin
[t], and the process proceeds to move to the synchronization point after the synchronization point point (steps 6020 and 6021).

(7) In the comparison in step 6020, the elapsed time Ctime is equal to the arrival time t [point
]) Or if the synchronization point point is the last synchronization point END, the synchronization point poin
It is determined that it is necessary to prepare to reproduce the sound of t, and t [po
int] −Calculate the change amount Δp ′ that can reach the synchronization point point by Ctime. The change amount Δp ′ is (the number of pixels moved to the destination synchronization point) / ｛(t [po
int] -Ctime) × Δp} (step 6022).

(8) Next, the calculated change amount Δp ′ is compared with a limit value Δp_max of the change amount for making the movement of the illumination mask look smooth, and the calculated change amount Δp ′ is changed to the limit value Δp_max of the change amount. Exceeds the limit value Δp_max
Changes the value of the change amount Δp ′ so as to move the illumination mask. As a result, a delay occurs in the arrival time at the synchronization point in order to smoothly move the illumination mask. However, since a part of the illumination mask has reached the point before the arrival time, the user has a large delay. The user does not feel any out of synchronization (steps 6023 and 6024).

(9) If the change amount Δp ′ is smaller than or equal to the limit value Δp_max of the change amount in the comparison in step 6023, the calculated change amount Δp ′ is compared with the standard value Δp of the change amount. If the change amount Δp 'is smaller,
The value of the change amount Δp ′ is changed so that the illumination mask is moved at the standard value Δp of the change amount (steps 6025 and 602).
6).

(10) After the adjustment of the variation Δp ′ is completed, that is, after the processing of steps 2024 and 2026 or in step 2025, the variation Δp ′ is changed to the standard value Δp of the variation.
If so, the synchronization control function 1440 sets the illumination mask to the center position X [point], Y [point
]), Request the screen display function 1420 to move / deform to width Width [point] and height Height [point] (step 6027).

(11) The screen display function 1420 displays an image obtained by moving the illumination mask by Δp ′ pixels at a time.
00 is sequentially displayed. Then, after the movement of the illumination mask is completed, the current synchronization point Cpoint is updated, and the process returns from step 6001 to the process from step 6001 after waiting for the user input process in the audio reproduction loop (steps 6011 and 6012).

(12) In the comparison in step 6008, the movement start time to the synchronization point point is the synchronization point point +
If the time is less than or equal to the movement start time to 1, the elapsed time Ctime is compared with the movement start time to the synchronization point point + 1, and if the elapsed time Ctime is less than or equal to the movement start time to the synchronization point point + 1, The synchronization control function 1440 determines that it is necessary to reproduce the sound at the synchronization point point, and sets the illumination mask to the center position X.
[Point], Y [point]), request the screen display function 1420 to move / deform to width Width [point] and height Height [point] (steps 6009, 601)
0).

(13) The screen display function 1420 displays an image obtained by moving the illumination mask by Δp pixels at a time on the screen display unit 12.
00 is sequentially displayed. Then, after the movement of the illumination mask is completed, the current synchronization point Cpoint is updated (step 6011).

(14) In step 6009, the elapsed time Cti
If me is greater than the movement start time to the synchronization point point +1, the synchronization point point
It is determined that it is necessary to prepare to reproduce the audio of the part after +1 and 1 is added to the counter point of the synchronization point number, and the synchronization point point is compared with the final synchronization point END. If the synchronization point point is not the final synchronization point END in this comparison, step 6007
Returning to the processing from, the processing for narrowing down the destination synchronization point is continued. If the synchronization point point is equal to the final synchronization point END, the synchronization point is set as the destination motive point (step 6014).

(15) After the processing in step 6011, the current synchronization point number Cpoint
Is equal to the final synchronization point number (END), the illumination mask is not moved or deformed, and the current position and size are maintained, or in the comparison in step 6006, the elapsed time Ctime moves to the synchronization point point. If it is shorter than the start time, does not move or deform the illumination mask, and keeps the current position and size, the user performs input processing during the audio reproduction loop, and a command to stop the audio in the audio reproduction loop is issued by the user. When input, the voice and the timer are paused, and the interface screen shown in FIG. 4B is displayed. Also, when receiving a user input instructing to resume sound reproduction during the pause of the sound, the interface screen shown in FIG. 4C is displayed, and the sound reproduction and the timer monitoring are resumed (step 6012).

Next, the movement / deformation process of the illumination mask will be specifically described using the values of the synchronization control table shown in FIG. An example of the values in the synchronization control table shown in FIG.
Synchronization point 1 of the synchronization control table Cnt001 shown in FIG.
The number of moving pixels 710 is calculated using the values of
It shows the result of calculating 0, the preceding movement time 7200, and the movement start time 7300.

In the example shown in FIG. 7, if the current synchronization point number is 1, the illumination mask is set at the center position (470, 200), 70 pixels wide and 350 pixels high. Then, as can be seen from the example of FIG. 7, the illumination mask controlled by the synchronization control table Cnt001 moves only in the horizontal direction because the Y coordinate at all synchronization points is constant.

As a result, the number of moving pixels 7100 from the synchronization point 1 to the synchronization point N can be calculated as | X coordinate of synchronization point n + 1−X coordinate of synchronization point n |.

If the standard value of the tempo is 25 frames / second and the standard value of the change is 2 pixels / frame, the change per second is 50 pixels, and the number of moving pixels / 50 is equal to the preceding moving time 7200. Become. Then, the movement start time 7300 is obtained by subtracting the preceding movement time from the arrival time at the synchronization point.

In the above description, depending on the performance of the computer, the illumination mask cannot be moved at the standard tempo, and the movement of the illumination mask may be delayed.
In the embodiment of the present invention, even when a delay occurs in the movement of the illumination mask, since the synchronization point is moved in advance,
When the sound of the destination synchronization point is played, the head of the synchronization point is included in the illumination mask, and the synchronization between the illumination and the sound can be maintained.

The synchronization control function 1440 determines the destination of the synchronization point by comparing the elapsed time from the start of audio playback with the movement start time of each synchronization point. Then, if the current synchronization point number is 1 and the elapsed time acquired in step 6003 is 6 seconds (Ctime = 6), the synchronization control function 1440 determines in step 6006 that the elapsed time Ctime and the movement of the synchronization point 2 have been changed. Compared with the start time, the elapsed time Ctime <the movement start time of the synchronization point 2 (7.1 seconds), it is determined that the time to move to the synchronization point 2 has not been reached, and the movement / deformation of the synchronization point is performed. Is not performed, and the center position and the size of the synchronization point 1 are maintained.

Also, the current synchronization point number is 1, and the elapsed time acquired in step 6003 is 8 seconds (Ctime =
8), when the elapsed time Ctime is compared with the movement start time of the synchronization point 2 in step 6006, the elapsed time Ctime> the movement start time of the synchronization point 2 (7.1 seconds). It is determined that preparation for reproducing the sound after the synchronization point 2 is necessary, and step 600 is performed to further narrow down the synchronization point of the movement destination.
Proceed to 7.

The synchronization control function 1440 has the synchronization point 3
, The elapsed time Ctime is compared with the movement start time of the synchronization point 3 in step 6009, and the elapsed time Ctime <the movement start time of the synchronization point 3 (10.5)
Second), it is determined that the time to start moving to the synchronization point 3 has not been reached. As a result, synchronization point 2
Center position (430, 200), width 60 pixels,
The illumination mask is moved and deformed to a height of 350 pixels.

The current synchronization point number is 1, and the elapsed time acquired in step 6003 is 12 seconds (Ctime = 12)
When the elapsed time Ctime is compared with the movement start time of the synchronization point 2 in step 6006, the elapsed time Ctime
Since time> the movement start time of the synchronization point 2 (7.1 seconds), the synchronization control function 1440 determines that preparation for reproducing the sound after the synchronization point 2 is necessary, and further determines the synchronization point of the movement destination. Step 6007 to refine
Proceed to.

The synchronization control function 1440 has the synchronization point 3
After calculating the preceding travel time, the elapsed time Ctime and the movement start time of the synchronization point 3 are compared in step 6009. Since the elapsed time Ctime> the movement start time of the synchronization point 3 (10.5 seconds), the synchronization point It is determined that it is necessary to prepare for reproduction of the third and subsequent sounds, the pointer of the synchronization point number is updated to further narrow down the synchronization point of the movement destination, and the preceding movement time of the synchronization point 4 is calculated in step 6007.

The synchronization control function 1440 has the synchronization point 4
After the calculation of the preceding movement time, the elapsed time Ctime is compared with the movement start time of the synchronization point 4 at step 6009. Since the elapsed time Ctime <the movement deformation time of the synchronization point 4 (13.8 seconds), the synchronization point It is determined that the time to start moving to 4 has not been reached. As a result, the synchronization control function 1440 determines, via the synchronization point 2, the center position (400, 200) and the width 40 of the synchronization point 3.
The illumination mask is moved and deformed to pixels and 350 pixels in height.

The synchronization control function 1440 has the synchronization point 1
When moving the illumination mask from to the synchronization point 3,
The movement start time is calculated in consideration of moving the illumination mask to the synchronization point 2 and moving the illumination mask via the synchronization point 2 only by not suspending the illumination mask at the synchronization point 2. . In the embodiment of the present invention, since such control is performed, the user does not feel a large synchronization deviation.

The above-described synchronization processing apparatus according to the embodiment of the present invention can partially change the position and size of the focal point of the illumination only by changing the numerical value of the synchronization control table.
Further, by replacing the illumination mask, it is possible to produce effects due to changes in various modes, such as changing the color of light or changing the color of characters.

[0056]

According to the above-described embodiment of the present invention, the illumination that moves and deforms in synchronization with the audio data is virtually created, so that the illumination on the still image is moved and deformed in synchronization with the audio data. You can create a movie that does not require a special shooting set or technology, and calculate the appropriate synchronization timing as needed using the synchronization control table, so you can synchronize with the audio data and stop When playing a movie in which the lighting on the picture is moved or deformed, the difference in quality due to the difference in computer performance can be reduced, and the lighting of the movie can be easily adjusted.

[0057]

As described above, according to the present invention, a movie for moving and deforming illumination on a digital image in synchronization with audio data can be created without any special set or technique, and the performance of the computer can be improved. The difference in movie quality due to the difference can be reduced, and the lighting can be easily adjusted.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a system configuration of a digital image and audio data synchronization processing apparatus according to an embodiment of the present invention.

FIG. 2 is a diagram illustrating a configuration example of a playback content control table.

FIG. 3 is a diagram illustrating a configuration example of a synchronization control table.

FIG. 4 is a diagram illustrating an example of an interface screen.

FIG. 5 is a flowchart illustrating a processing operation of a control process according to the embodiment of the present invention.

FIG. 6 is a flowchart illustrating a processing operation of a control process of an audio reproduction loop according to an embodiment of the present invention.

FIG. 7 is a diagram illustrating a synchronization control table and an illumination mask control value calculated in a control process.

[Explanation of symbols]

 1100 User input unit 1200 Screen display unit 1300 Audio output unit 1400 Content control unit 1410 Input control function 1420 Screen display function 1430 Audio reproduction function 1440 Synchronization control function 1441 Real time management function 1500 Synchronization control database 1510 Image data 1520 Audio data 1530 Control table 1531 playback content control table 1532 synchronization control table

 ──────────────────────────────────────────────────続 き Continued on the front page F term (reference) 5B050 AA09 BA06 BA08 EA12 EA24 FA10 5C053 FA14 FA21 GB10 GB11 HA27 JA12 KA00 LA06 LA11

Claims (4)

[Claims] 1. A real-time management control for synchronizing illumination and audio data on a digital image in a digital image and audio data synchronization processing device for expressing illumination on a digital image in synchronization with audio data. Synchronous control means having a function, image data to be illuminated, audio data to be synchronized with the illumination, a synchronization control table defining synchronization timing and position and size to be illuminated, and image data, audio data, and the synchronous control A database in which a reproduction content control table defining a correspondence between tables is registered, wherein the synchronization control means moves and deforms in synchronization with audio data based on information in the table of the database. Synchronizing apparatus for digital image and audio data, wherein illumination for the digital image is virtually generated . 2. The synchronization processing device for digital image and audio data according to claim 1, wherein said synchronization control means calculates an appropriate movement of illumination on the digital image and a time of starting deformation. . 3. The synchronization of digital image and audio data according to claim 2, wherein the real-time management control function monitors a timer for synchronizing illumination on the digital image with audio data. Processing equipment. 4. The digital image and audio data according to claim 1, 2 or 3, wherein the digital image and audio data subjected to the synchronization processing are output to a screen display unit and an audio output unit. And synchronous processing device.