JPS62137670A - Picture effect control device - Google Patents

Abstract

PURPOSE:To add a deformation effect corresponding to the feature of a sound at a prescribed part on a picture by mapping a flat plane picture on a deformable curved surface generated based on a sound detection signal corresponding to the feature of the sound. CONSTITUTION:A sound detection signal generation circuit 3 detects the feature of a generated sound, and sends a sound detection signal DET corresponding to the feature of the sound to a deformable curved surface data generation device 10. The device 10 reads out a deformation parameter data DPR corresponding to the sound detection signal DET out of deformation parameter data DPR stored at a deformation parameter data storage device 16, and forms the deformable curved surface where a picture part indicated by the DPR is deformed cubically. And by mapping the flat plane picture represented by a flat plane picture data VDIN at a cubical deformable curved surface with a picture conversion device 4, the flat plane picture can be deformed with the deformable curved surface.

Description

[Detailed description of the invention] The present invention will be explained in the following order.

A: Industrial application field B: Outline of the invention C: Conventional technology: Problems to be solved by the invention E: Means for solving the problem (Figs. 1, 2) ) G Embodiment (Figures 1 to 9) H Effects of the invention A Industrial field of use The present invention relates to an image effect control device, for example, a special effects device for broadcasting, for configuring still images or moving images. This is suitable for use in devices that add special effects to images by deforming them.

B. Summary of the Invention The present invention provides an image effect control device that adds a deformation effect to a planar image represented by planar image data. By mapping a flat image onto a curved surface, a deformation effect corresponding to the characteristics of the sound can be added to a predetermined portion of the image.

C. PRIOR TECHNOLOGY In conventional special effects apparatuses for broadcasting, when changing an image, an operator manually operates an operator such as a lever to obtain an operation mode for adding an effect.

However, in music promotional videos, etc.
When music is the main focus and a screen is created to match the sound of the music, the operator must manually operate the controls when performing special effects that transform the images that make up the screen. In addition, it was difficult to accurately match the timing of inserting effects to the progress of the song. Therefore, it is necessary to perform complicated operations such as adding special effects on images to match the music through trial and error, and then repeatedly playing back and checking the performance sounds and images.

However, a part of the image can be modified in detail (for example, timbre, pitch, strength, etc.) by adjusting the sound elements of the song (for example, timbre, pitch, strength, etc.) at one or more locations.
The deformed side? Such processing could not be achieved by conventional special effects devices.

D Problems to be Solved by the Invention The present invention has been made in consideration of the above points. This paper attempts to propose an image effect control device that can transform images in accordance with changes in sound by transforming the images.

E Means for Solving the Problem In order to solve this problem, in the present invention, an image generating means 2A for generating planar image data V D + N;
2B, 2C..., a sound detection signal forming means 3 that sends out a sound detection signal DET corresponding to the sound characteristics, and a deformation parameter data storage means that stores the deformation parameter data DPR corresponding to the sound characteristics. 16, the sound detection signal DET is received from the sound detection signal forming means 3, the corresponding deformation parameter data DPR is read out from the deformation parameter data storage means 16, and the image portion specified by the deformation parameter data DPR is three-dimensionally deformed. A deformed surface data creation means 1O creates deformed control data DEF representing the deformed surface formed by the deformed surface, and a plane given by two image generating means, 2B, 2C, etc., on the deformed surface represented by the deformed control data DEF. An image converting means 4 for mapping the image data VD, .

The F action sound detection signal forming means 3 detects the characteristics of the generated sound and sends a sound detection signal DET corresponding to the characteristics of the sound to the deformed surface data generating means 10.

The deformed surface data creation means 10 uses the deformed parameter data DP stored in the deformed parameter data storage means 16.
Of R, the deformation parameter data DPR corresponding to the sound detection signal DET is read out, and the deformation parameter data DPR corresponding to the sound detection signal DET is read out.
A deformed curved surface is formed by three-dimensionally deforming the image portion designated by PR.

The image converting means 4 maps the planar image represented by the planar image data VD+n onto this three-dimensional deformation curved surface, and thus deforms the planar image by the deformation curved surface, thereby making it possible to obtain a display image with added image effects. .

Thus, it is possible to realize an image effect control device that controls the effect based on the characteristics of the sound.

G Embodiment A detailed explanation will be given below with reference to the drawings as an embodiment in which the present invention is applied to a special effects device for broadcasting.

In FIG. 1, ■ indicates an image effect control device as a whole, and includes a plurality of image information input devices 2A, 2B, 2C, etc., each of which includes, for example, a video tape recorder (VTR), etc.
The plane image data VD+s sent from the image human power device selected based on the selection signal SEL generated by the sound detection signal forming circuit 3 is input to the image conversion device 4, and this image Converted image data V of the image conversion bag W4 whose contents can be displayed and confirmed on the monitor 5
Douy is displayed on the display device 6.

The input sound signals AD, . . . reproduced by the image information input devices 2A, 2B, 2C, . It will be done. The sound detection signal forming circuit 3 extracts the characteristics of a reproduced sound (for example, a performance sound), and sends out a sound detection signal DET based on the sound element. For example, it is possible to extract the occurrence of a bass drum sound based on the volume and pitch of the input sound signal ADIN+, and also extract the occurrence of a bass drum sound based on the volume and pitch of the input sound signal ADIN+.
It is extracted that a cymbal sound has occurred based on the pulse-like peak waves mixed in N+, and then the input sound signal ADIN is extracted.
Recognizes when + contains voice (that is, human voice).

In this way, the sound detection signal forming circuit 3 is configured to detect, for example, the second
As shown in the figure, a drum player (hereinafter referred to as a drama) 6
When the performance image of C is played back together with the sound, bass drum 6 is selected based on the characteristics of the sound ¥ and pitch of the human sound signal ADIN+.
Detects that A has been hit. For example, input sound signal A
When a pulse-like peak waveform occurs in DIN+, it is determined based on this characteristic that the drama 6C has hit the cymbal 6B. Further, the sound detection signal forming circuit 3 includes an input sound signal ADIH.
When + contains speech, recognize the vowel or consonant produced.

In addition to this, the sound detection signal forming circuit 3 includes a microphone 7
The input sound signal AD1Nffi is received from the input sound signal AD1Nffi, and a sound detection operation is performed based on this input sound signal AD+sz.

The sound detection signal DET from the sound detection signal forming circuit 3 is provided to the deformed surface data creation device 10. The deformed surface data creation device 10 includes image information input devices 2A, 2B, 2C...
Planar image data V supplied to the image conversion device 4 from ...
This is to create a deformed surface that partially deforms DIN, and creates parameter data for a three-dimensional deformed surface using parameter signals input manually using the parameter input means 11, and stores the deformed parameter data in the deformed parameter data storage device. 16.

The parameter input means 11 includes a trackball 12, a plurality of levers 13, and a mouse 14.
input parameter signals regarding the viewpoint from which the shape after deformation is viewed, and manually input parameter signals regarding the deformation area VCF representing the range to be deformed and the deformation vector V%'' representing the deformation direction and magnitude using the lever 13; Furthermore, the mouse 14 is used to manually input a parameter signal for the point of action CP," which indicates the position where the deformation is to be performed.
(as indicated by the cursor represented on the display screen)
.

In this way, the deformation parameter data stored in the deformation parameter data storage device 16 is, for example, when displaying the performance of the drama 6C as described above with reference to FIG. ,
Transformation parameter data that allows specifying and transforming an image portion of the drama 6C is stored.

This deformation parameter data is created in the deformed curved surface data creation device 10 by the method described below based on the parameters input from the parameter input means 11.

The deformation parameter data DPR stored in the deformation parameter data storage device 16 is stored in the deformation surface data creation device IO.
The zero image human power device η↓ sent to the image conversion device 4 through the image information human power device 2A, 2B, 2C...
By performing a conversion operation on the planar image data VD4 supplied from ・using the deformation control data DEF representing a three-dimensional curved surface, an image similar to that of pasting a planar image (Fig. 2) on the deformation curved surface represented by the deformation control data DEF is obtained. Image data V
(this is called mapping), the converted image data VDou is provided to a display device 6 made of, for example, a cathode ray tube (CRT), so that the display screen DS
A stereoscopic image 21 that looks three-dimensional is displayed on P.

The deformed surface data creation device 10 includes parameter input means 11
Based on the deformation parameter data DPR input by , as shown in FIG.
A position vector representing the point of action cp,'' is specified, and the deformation operation is executed only within the range of the deformation region VCF that includes the point of action cp,''.

As a result of the calculation, as shown in FIG. 4, a three-dimensional image 21 can be displayed on the display screen DSP of the display device 6 as if the deformed curved surface after deformation is viewed from an arbitrarily determined viewpoint position.

The deformation of the curved surface in the deformation region VCF is performed by recursively calculating T'A using a conversion formula expressed by the following recursive formula %(1).

In equation (1), P8' is a position vector representing each point of the curved surface after deformation formed in three-dimensional space, and this position vector P,' is the position vector of the corresponding point on the curved surface before deformation. It is represented by the sum of p i-+ 1'' and a deformation amount vector U''' from the position vector P, -1'' before the deformation.

This deformation amount vector U0 is expressed as the following formula (% formula %), vector field function Fi (P,, ”,
It is expressed as a position vector obtained by multiplying CP,') by a deformation vector Vi''.

Here, the deformation vector Vi" is defined as the point of action CP," when the point of action cp," is specified on the surface to be deformed SOR.
The direction and magnitude of the deformation to be applied to the deformation target surface SOR at , "is expressed as a vector quantity, and the point of action CP," of the surface SOR is thereby lifted by that magnitude in the direction of the deformation vector V %. It will undergo deformation such that

Also, the vector field function Fi (Pi−,”, CPt
") is the deformation area V determined including the point of action cp,"
It represents the distribution of relative deformation rates that determines how much deformation is given to each point P,−,'' of CF (its size can be specified manually by setting parameters). This relative deformation rate distribution has values only inside the deformation region VCF, and when going to the periphery,
It has a distribution of scalar quantities that "converges to 0" or "converges to 0".

Therefore, the deformation amount vector U'' does not represent the deformation amount at each point of the deformation area VCF, but is a position vector, whose direction is parallel to the deformation vectors ■ and *, and whose magnitude is
It has a multiplication value (scalar quantity) of the magnitude of the deformation vector V4'' and the relative deformation rate distribution represented by the vector field function F.

In this way, the deformation of the curved surface of the deformation region VCF occurs at the point of action CP,
” occurs in the direction and magnitude of the deformation vector ■, ., and a line from this point of action CP, ” to the periphery (according to the direction of the deformation vector ■, “, and corresponds to a change in the deformation rate of the vector field function F, occurs with varying magnitude.

Here, as the vector field function F, for example, as shown in Fig. 5, if a function such as a Gaussian distribution function that gradually converges symmetrically to 0 as it goes outward from the center point is assigned. The deformation amount vector U''' has the maximum value in the direction of the deformation vector v,'' at the position of the point of action CP,', and has the direction of the vector V% as it goes from the point of action CP,'' toward the outer periphery, and The size gradually increases to 04
This results in a deformed surface that converges.

In this way, the deformation amount vector U' is obtained by one conversion operation, and this is determined by the position ff- before deformation and the vector Pi
-1゛ is added to obtain the post-transformation position edge 1~Pl''', and every time the transformation operation is performed in the same way, (
1) By calculating the recurrence formula expressed by the equation, a position vector representing the deformed surface is repeatedly and recursively calculated based on the position vector before deformation.

The position vector PH” representing the final deformed curved surface obtained as a result of repeating such recursive calculations N times is the deformed surface SO before the start of deformation, as expressed by the following equation.
As a position vector obtained by adding the sum of the deformation l vectors U'' (i.e., the total deformation amount) sequentially obtained by N times of deformation calculations (i = 1 to N) to the position vector representing the point p% on R. Desired.

Thus, according to equation (3), the operator deforms the surface S
When sequentially performing N deformation operations from point Po9 on OR,
Each time, the point of action cp,'' is specified for the curved surface before deformation (for example, specified using the coordinates of the deformation target surface SOR), and the operator selects the portion of the curved surface p +-+'' before deformation that is to be deformed. It can be specified arbitrarily based on the judgment of In addition, by respecifying the vector field function F and the parameter □-E that determines the deformation vectors 1 and 1, the size of the deformation region VCF, the deformation rate distribution of the deformation surface, and the direction of deformation can be similarly determined by the operator. You can reset the settings as you like.

In this way, each time the operator performs a single deformation operation, the operator gradually applies deformation to the undeformed curved surface at a desired position, in a desired direction, and with a desired magnitude. It can be stacked chemically.

The image effect control device 1 shown in FIG.
Deformation control data DEF that converts the coordinate position of each point on R into a position vector p% representing a point in three-dimensional space.
is generated in the deformed surface data creation device 10, and the image converting device 4 is subjected to conversion control based on this deformation control data DEF.

As a result, the image conversion device 4 converts the image information manual bag ff2A onto the deformation curved surface represented by the deformation control data DEF.
, 2B, 2C .

Here, as the image conversion device 4, Japanese Patent Application Laid-Open No. 58-19975
The method disclosed in the above publication can be applied, and the planar image data VDIN is mapped onto the deformed curved surface indicated by the deformation control data DEF in the following procedure.

That is, first, the deformation target surface SOR on the xy coordinates where the planar image data VD+s is expressed is divided into minute regions ER of the necessary size, and for one representative point P of each minute region ER, the deformation control data DEF is divided. The coordinates are transformed onto the deformed curved surface by executing a deformation operation based on .

In this way, a transformed image by the representative point P is formed on the deformed curved surface, but the image converting device 4 then converts the transformed image from the four surrounding representative points P around each transformed representative point P. A parallelogram shadow area of a predetermined size including the center representative point P+t is formed (with sides extending in the direction connecting mutually adjacent representative points), and the position of each pixel included in this parallelogram shadow area is interpolated. Obtain by calculation.

This interpolation calculation determines the position of a pixel in the parallelogram shadow area after transformation based on the positional relationship between the corresponding minute area ER on the deformation target surface SOR and each pixel included therein.

Once the position of each pixel within the parallelogram shadow region on the deformation curved surface is determined, the image conversion device 4 assigns image information possessed by the corresponding pixel on the deformation target surface SOR to the pixel on the deformation curved surface. . As a result, the planar image drawn on the deformation target surface SOR is converted into an image using the deformation control data DEF, and a three-dimensional display image mapped onto the deformation curved surface is displayed on the display screen DSP on the display device 6. If
That will happen.

In this way, the planar image data V on the deformation target surface SOR
Image conversion processing is performed on D using the representative point P, and
By performing the interpolation calculation process based on the representative point PR after the conversion, the conversion process of the image conversion device 4 can be practically performed in real time.

In this way, the operator obtains the deformation control data D corresponding to one or more image parts that are to be transformed.
When EF is repeatedly reset by the parameter input means 11, the deformed image data VDoLIT represented by the planar image data VDIN is converted by the deformation control data DEF in the image conversion device 4 and displayed on the display device 6. Ru. At this time, the image conversion device 4 executes calculations for mapping the repeatedly set data onto the deformed curved surface in real time, and can display a gradually deformed image on the display device 6. As a result, the planar image data VD1. The user can interactively modify and modify the image little by little while viewing the display on the display device 6.

In the case of this embodiment, the deformed surface data creation device 10 uses a Gaussian distribution function as the vector field function Ft, and is capable of specifying a circular or elliptical region as the deformed region VCF.

In this case, the vector field function F is expressed as
, Yi), and the radii in the X direction and the X direction are α and β1.
), as shown in Figure 5, it becomes a Gaussian distribution function in the X direction and in the
direction only).

Therefore, the deformation amount vector U0 is calculated by the following formula (0 formula %).
, the parameters of the point of action cp,'' are expressed as coordinates (Xi
, Y, ), and the X-direction and X-direction radii α and β2 are set as parameters of the deformation region VCF, as well as the parameters of the deformation vector v,'. Thus, the operator has a point of action cp. (Xi, Y
, ), the point of application CP+'' (Xt
, Yr), the deformation rate smoothly converges to 0 as it draws a Gaussian distribution curve toward the periphery with the deformation vector V,′ as the center. You can create deformed surfaces.

Thus, among the curved surfaces represented by the position vector Pi-+" before deformation, for the local deformation region VCF centered on the point of action CP,4 on the deformation target surface SOR, a Gaussian curve is applied in the direction of the deformation vector ■,1. It is possible to obtain a deformed curved surface represented by the position vector P8' that exhibits a smooth curved surface as shown by the distribution function.

In this embodiment, when the operator inputs a parameter using the parameter input means 11, the deformed surface data creation device 10 processes the deformed parameter data DPR in accordance with the processing procedure shown in FIG. The processing result is stored in the deformation parameter data storage device 16 with an address corresponding to the sound detection signal DET, and thereby the deformation parameter data DPR corresponding to the content is stored in the deformation parameter data storage device 16 according to the sound detection signal DET. It is arranged so that it can be read out from the value Tl 16.

That is, after starting the processing procedure in step SP1, the central processing unit (CPU) provided in the deformed surface data creation device 10 represents the deformation target surface SOR (consisting of planar image data VD+s) in step SP2. The position vector Po' is stored in the curved surface data memory 15 provided in the image conversion device 4 (FIG. 7(A)).
Set to .

Subsequently, the CPU moves to the next step SP3 and inputs the parameters α8, β, , x, , y, , V, II set by the operator using the lever 13 and mouse 14.

In the next step SP4, the CPU of the deformed surface data creation device 10 controls the track pole 12 by the operator.
After taking in the viewpoint position data input from , the process moves to step SP5. This step SP5 executes the above-mentioned calculations regarding equations (4) and (5). Here, the deformed surface data creation device 10 sends the deformation control data DEF to the image conversion device 4, and outputs the position vectors p+-, ” before deformation.
The values set in the surface data memory 15 are used as (x, y), and each parameter α1, β8, X, ,
Y, , Vi'' and the deformation calculation is performed using the settings in step SP a 4::.

Next, in step SPG, the CPU of the deformed surface data creation device 10 uses the image conversion device 4 to convert the planar image data VDIN onto the deformed surface represented by the deformed position vectors P,' calculated in step SP5. is subjected to mapping processing and displayed on the display device 6.

In this state, the CP of the deformed surface data creation bag W10
U causes the display to m)J, and in the next step SP7, the operator checks the display on the display device 6 to confirm whether the degree of deformation meets the operator's requirements.

Thereafter, the CPU of the deformed surface data creation device 10 moves to the next step SP8 and determines whether or not the operator has input a confirmation signal.

If a negative result is obtained here, the CPU of the deformed surface data creation device 10 returns to the above-described step SP3 and returns to the state of waiting for the setting of new parameters.

At this time, the operator resets the new parameters in steps SP3 and SF3, recalculates the transformation formula in steps SP5 and SF3, and then displays an image based on the calculation results in the display bag fi6. Then, in step SP8, the operator is again asked to judge whether the deformation is as requested.

Thus, the CPU of the deformed surface data creation device 10 executes steps 5P3-3P4-3P5-3P6-3P7-3P8.
- Repeat the loop LOOPI of SF3 until the operator is able to deform it to meet his/her requirements, thereby resetting the position of the point of action CP□, the size of the deformation area VCF, the direction and height of the deformation vector v/0 I can do it.

Eventually, when the operator is satisfied with his/her setting operation and manually inputs a setting end signal to the deformed surface data creation device 10, the CPU moves to the next step SP9 and stores the set data α1, β+, Xt, Yt, V%. command list memory 4 provided in the deformed surface data creation device 10
6 (FIG. 7(B)), after storing it in the parameter memory area N=1 corresponding to the first setting operation, move to step 5PIO, add "+1" to the number of operations i, and then (
i=2), move to step 5 PII.

This step 5PII is a step for confirming whether or not the operator has finished the deformation operation. If the operator has not inputted an operation end command, the CPU of the deformed surface data creation bag WlO returns a negative result in step 5PII. As a result, the process returns to step SP3 through the loop LOOP2, and waits for the second transformation operation (i=2) by the operator.

In this state, the operator starts the first operation with a new intention.
For the curved surface created by the screen transformation operation,
A second deformation operation of the curved surface can be performed. In this way, the operator can again perform a transformation operation that meets his/her needs regarding the application point CP, which is different from the application point CP, ``, which was transformed by the first transformation operation.

In this second deformation operation, the deformed surface data creation device 10 calculates, in step SP5, a deformed image viewed from the viewpoint position set in step SP4, based on the parameter data taken in in step SP3, and After displaying on the display device 6 at SP6, a series of transformation operations are executed to store the input transformation parameter data DPR in the command list memory 46 at step SP9.

Thereafter, in the same manner, the CPU of the deformed surface data creation bag WIO executes the deformation process described above and displays the processing result on the display device 6 each time the operator performs a new deformation operation, and then commands the deformation parameter data DPR. The operation of storing in the list memory 46 is repeated.

Eventually, when the operator finishes all the deformation processes, the deformed surface data creation device 7Z10 responds to this by obtaining a positive result in steps 5PII to 5P12, where the deformation created by N times of deformation processes is performed. The parameter data DPR is stored in the modified parameter data storage device 1 with an address corresponding to the sound detection signal DET.
6, the program moves to step 5P13 and ends.

In the case of this embodiment, when the sound detection signal forming circuit 3 detects a bass drum sound, the sound detection signal DET is used as a deformation parameter that causes the entire surface of the bass drum 6A in the stereoscopic image 21 to be deformed so as to jump forward. Data DPR
is read out from the deformation parameter data storage device 16.

Further, when the sound detection signal forming circuit 3 detects a cymbal sound, the sound detection signal DET generates the deformation parameter data DPR that causes the arm of the drama 6C in the stereoscopic image 21 to deform as if it hits the cymbal 6B. The information is read out from the storage device 16.

In the above configuration, the sound detection signal forming circuit 3 generates a selection signal SEL that sequentially selects the image information input devices 2A, 2B, 2C, . . . in that order and sends out the planar image data VD+s. do. The timing at which the selection signal SEL is switched is set to the timing at which a predetermined sound is generated in the human-powered sound signal ADIN+ and the sound detection signal forming circuit 3 detects this.

For example, when two image information input devices are in operation mode,
If the image of the plane image data VD+s is a performance image of a drama as described above with reference to FIG. 2, the plane image data V D I N is written to the curved surface data memory 15,
The curved surface data representing this planar image can be transformed three-dimensionally while being partially designated by the transformation control data DEF.

The image information input device 2A inputs planar image data VDIN.
An input sound signal ADIN+ corresponding to the content of is sent out, which is converted into sound by the speaker device 8 and is also provided to the sound detection signal forming circuit 3.

In this state, when a bass drum sound occurs in the input sound signal ADIN+, the sound detection signal forming circuit 3 sends the sound detection signal DET to the deformed surface data creation device IO based on the characteristics of the volume and pitch.

At this time, the deformation surface data creation device 10 generates deformation data DP for deforming the image portion of the bass drum 6A on the display screen.
R is read from the deformation parameter data storage device 16 and sent to the image conversion device 4 as deformation control data DEF.

The image conversion device 4 converts the deformation data DE into the flat image data VDIN written in the curved surface data memory 15.
Transformed image data vt)ou by adding three-dimensional deformation by F
It is sent to the display device 6 as t. As a result, the display device 6 displays an image in which the bass drum 6A appears to be protruding forward, for example.

Here, the speaker device 8 is operated for a time corresponding to the time delay from the time when the sound detection signal forming circuit 3 detects the sound of the bass drum 6A until the image of the transformed bass drum 6A is displayed on the display device 6. The bass drum sound is generated at a delayed timing, and thus the bass drum sound is generated at the timing when the lotus drum 6A jumps forward on the display screen of the display device 6.

For example, if a peak waveform occurs in the input sound signal ADIN+, the sound detection signal forming circuit 3 generates the sound detection signal DET at that timing and provides it to the deformed surface data creation device 10, thereby generating the deformed surface data creation bag surface 1o. The deformation parameter data DPR representing a series of actions such as the arm of the drama 6C hitting the cymbal 6B on the display screen is sequentially read from the deformation parameter data storage device 16 and supplied to the image conversion device 4 as deformation control data DEF.

At this time, the image conversion device 4 transforms the planar image data VDIN written in the curved surface data memory l5 using the deformation control data DEF, so that an action similar to that of the drama 6C hitting the cymbal 6B on the display screen of the display device 6 is generated. will be displayed. At this time as well, the speaker device 8 generates a cymbal sound after a delay of a predetermined delay time (same as in the case of the bass drum), and thus at the timing when the drama 6C hits the cymbal 6B on the display screen of the display device 6. A cymbal sound is generated.

According to the above-described embodiment, the portion to be deformed and the degree of deformation in the image are detected based on the sound characteristics of the human-powered sound signal ADIN+, and special effects are added by deforming the two-dimensional image data VD, N. By doing so, the timing at which special effects are added to the image is determined by the input sound signal ADIN.
You can easily adjust to the change in the + sound.

8 and 9 show an embodiment in which the present invention is applied to a judgment information display device. A judgment information display device is a device having a judgment ability, such as a computer, and displays judgment results on a display device (usually a CRT). This system notifies the operator of the results of the computer's judgment by displaying the result on the computer screen.

This judgment information display device focuses on the fact that a variety of information can be conveyed to the other party by slight changes in the parts of the human body, such as the face, hands, shoulders, etc., and displays devices such as computers on the display device. A system has been proposed in which the facial expressions of body parts are displayed in accordance with the judgment results (Japanese Patent No. 11/1986 60439882).

In the case of this embodiment, the judgment result of the computer or other equipment is a sound composed of an arrangement of vowels or consonants such as "a", "i", etc., and this is the sound that is input by the microphone 7. The signal is converted into a signal AD+Nz and input to the sound detection signal forming circuit 3.

The sound detection signal forming circuit 3 outputs the sounds "a", "i", etc.
. . , and provides the corresponding sound detection signal DET to the deformed surface data creation device 10. For example, as shown in Part 8, the deformed surface data creation device 10 generates the sounds "a", "i", etc. based on the face image FDI with the mouth closed.
Movement of facial expressions up to face image aFD2 when pronouncing ... (
Mainly, movements around the mouth) are displayed as an animation image, for example.

In this embodiment, the deformation parameter data storage device 16
is the data of the standard face image FDI and the data of the final face image FD2 when each vowel and consonant is pronounced (in the case of Fig. 8, this face image corresponds to the sound "a"). and data representing changing interpolation coefficients are stored in the deformation parameter data storage device Z16.

In this configuration, when the human powered signal A D + s□ corresponding to, for example, the voice "A" is input to the sound detection signal forming circuit 3 through the microphone 7, the deformed surface data creation device IO converts the deformed parameter data arrangement device 16 Facial image data FDI and FD2 are read out from FD!, and data representing changing interpolation coefficients are read out to create facial image data FD!
Deformation control data DEF is created by interpolating and FD2 using interpolation coefficients that gradually change.

As a result, as shown in FIG. 8, at time 1. Deformation control data DEF corresponding to the reference face image FDI is obtained in
After that, the expression gradually changes from reading until time t2 in the facial image F.
Animated face image data that changes to D2 is supplied to the image conversion device 4 as deformation control data DEF.

In this embodiment, the image conversion device 4 has planar image data V.
Data containing a standard face image as D+N is written in the curved surface data memory 15, and by three-dimensionally deforming this standard face image data using deformation control data DEF, the data shown in FIG. 9 is obtained. The face images FDI to FD2 can be displayed three-dimensionally on the display screen of the display device 6 as shown in FIG.

According to the configuration of this embodiment, the microphone 7 can display a facial image with a corresponding expression on the display device 6 based on the sound given from a device having judgment ability such as a computer.

In the embodiments shown in FIGS. 8 and 9, we have described the case where an animated image with a moving facial expression is used as the facial image, but even if a still image is used instead, the same effect as in the case described above can be obtained. Obtainable.

Furthermore, in the above-described embodiments, a case has been described in which the present invention is applied to a special effects device for broadcasting and a judgment information display device, but the present invention is not limited to this. It can be widely applied to control devices.

In addition, in the above, the human sound signals ADIN+ and ADI
A case has been described in which a source that generates a sound signal is provided as the NK, and the characteristics of the sound signal generated from the sound signal generation source are extracted in the sound detection signal forming circuit 3. However, the present invention is not limited to this. When a device that can create sounds, such as a speech synthesizer, is used as a signal generation source, the characteristic signals used in synthesizing the sounds may be used to be applied to the deformed surface data creation device 10. , the same effect as in the above case can be obtained.

Effects of the Invention As described above, according to the present invention, a two-dimensional image is transformed three-dimensionally based on the characteristics of the sound. Accordingly, it is possible to obtain an image effect control device that can easily add the following effects.

[Brief explanation of drawings]

FIG. 1 is a block diagram showing an embodiment of the image effect control device according to the present invention, FIG. 2 is a route map showing a converted image on a display device, and FIG. 3 is used to explain the surface to be transformed in FIG. 1. The route map, Figure 4, is displayed on display device 1i! 6 Route map showing the display above,
FIG. 5 is a route map for explaining the vector field function F used for calculations in the deformed surface data creation device, FIG. 6 is a flowchart showing the surface deformation processing procedure of the deformed surface data creation device, and FIG. 7 is a deformed surface A route map showing the memory provided in the data creation device 10; FIG. 8 is a route map showing changes in facial images in animation display mode;
FIG. 9 is a perspective view showing a three-dimensional facial image displayed on the display device 6. FIG. 1... Image effect control device, 2A, 2B, 2c.
... Image information input device, 3 ... Sound detection signal forming circuit, 4 ... Image conversion device, 6 ...
...Display device, 8...Speaker device, 10...
. . . Deformation surface data creation device, 11. . . Parameter input means, 16. Old . . . Deformation parameter data storage device.

Claims (1)

[Scope of Claims] Image generating means for generating planar image data; sound detection signal forming means for transmitting a sound detection signal corresponding to sound characteristics; and a modification for storing deformation parameter data corresponding to the sound characteristics. Parameter data storage means receives the sound detection signal from the sound detection signal formation means, reads out deformation parameter data corresponding thereto from the deformation parameter data storage means, and processes the image portion specified by the deformation parameter data. a deformed surface data creation means for creating deformation control data representing a deformed curved surface that has been three-dimensionally deformed; and an image conversion means for mapping planar image data given from the image generation means onto the deformed curved surface represented by the deformation control data. An image effect control device comprising: