JP2003173455A - Program edition method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To freely set a pose and action of a character appearing on a program in a program edition method using a CG animation production technique, a sound synthesizing technique, and an animation production technique of a computer. <P>SOLUTION: The pose can be designated with a selected angle or a position of a joint, and separately from the selected joint, one joint serving as a base point is decided. In this way, a movement distance of the joint can be limited not to move unnaturally, and the position of a joint existing between the selected joint and the joint as the base point is calculated automatically. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention is designed to generate a program on a computer used as a program editing apparatus used for television broadcast program production, video program production, program simulator, presentation, demonstration and the like. The present invention relates to a method of editing program information for describing contents.

[0002]

2. Description of the Related Art As a conventional technique, Japanese Patent Laid-Open No. 2001-1
As in the video data editing method described in Japanese Patent Publication No. 95604, the window display displayed on the monitor is set by the GUI (graphical user interface) operation function to set various function of program information, and the content of the program to be produced. It describes the program information according to. Based on the described program information, the computer's CG (computer graphics) animation function, voice synthesis function, and moving image generation function are controlled to output a video / audio signal generated as a program from the computer. To be done.

In particular, as a function for generating a video signal for displaying a character whose shape and shape are changed by performing various actions, such as a human character appearing in a program, a character operation function is used. There is a character setting function, and a motion setting function is further used. By this operation setting function, for example, as shown in the window display described in FIG. 14 of the above-mentioned publication, by operating to specify one operation from a predetermined operation list, for example, "sit" Is specified, program information for displaying the character in the program to sit down is generated by a previously specified motion.

[0004]

As the operation setting function according to the above-mentioned conventional technique, one of the predetermined operation lists is selected.
By operating to specify one motion, only the program information for displaying the character in the program to move with the motion specified in advance is generated.
By a motion not specified in advance, for example, the direction or angle of the joint connecting the shoulder or arm of the character is set to a direction and angle not specified in advance, that is, an arbitrary direction or angle, A video signal for displaying a character cannot be generated.

[0005]

In order to solve the above-mentioned problems, the present invention provides a program editing apparatus for generating and editing program information for generating a program by interactive operation with a computer on a monitor screen. In the editing method, a program is edited to specify the pose of the character by specifying the position of the joint of the character appearing in the program.

Further, the present invention is a program editing method in a program editing device for generating and editing program information for generating a program by a dialogue with a computer on a monitor screen, in a joint of a character appearing in the program. By designating the angle, the program is edited to designate the pose of the character.

It should be noted that it may be possible to edit the pose so that the joint position does not become unnatural by providing a biological limitation on the movement distance or direction of the joint.

[0008]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below. FIG. 18 is a diagram showing a block configuration example of the program editing apparatus according to the present invention. In this figure, 10
3 is a CPU (computer). The program information is generated by the CPU 103, and the generated program information is recorded in the memory 119 or the HDD 102.
In addition, the CPU 103 uses the memory 119 or the HDD 10
The program information recorded in No. 2 is received via the signal bus 119, and the received program information is edited. Further, it is possible to edit the program information input from the outside through the input / output interface 117. The program information input from the outside through the input / output interface 117 is stored in the memory 11
9 or the HDD 102.

The CPU 103 is an input / output interface 1
17, the program information can be edited based on input operation signals from the keyboard input device 113 and the mouse 116. The keyboard input device 113 and the mouse 116 output an input operation signal when the operator operates them. The operator is a CPU
The operation may be performed while visually confirming the operation image displayed on the monitor 112 from the video interface 110 from the screen 103. By doing so, the screen of the monitor 112, which is a peripheral device, is displayed by the software executed by the computer 103. A GUI (Graphical User Interface) for substituting functions such as push buttons
Is displayed. By inputting information to the computer 103 from the GUI using the input device 113 such as a keyboard and a mouse, the operation of the device can be interactively performed.

The CPU 103 also outputs a video signal representing the content of the program based on the program information to the input / output interface 11
7, or output the video signal to the monitor 112 via the video interface 110. The monitor 112 displays the content of the program as a video based on the video signal.

FIG. 1 is a diagram showing an example of a functional configuration relating to generation and editing of program information included in the television program editing apparatus. In this figure, a processing function unit realized by software operating on the computer 103 is roughly divided into a program generation unit 104 and a program information generation unit 105. The program generation unit 104 generates a program by interpreting the program information 101 generated by the program information generation unit 105.

Here, the program information 101 is TVML (TV
It is assumed that it is written in the Program Making Language). TVML language specification 1.0 (h
http: // www. strl. nhk. or. jp /
TVML / Japanese J03. In the rule of (html), one event is described as follows. Event type: Command name (arg = 1 = data
1, arg2 = data2 ,. ． ． 11) There are 11 event types, and there are multiple commands for each. The command arguments are in any order.

The sequencer unit 106 is based on the program information 101 and has a CG animation generation unit 107 and a voice synthesis unit 10.
8. The TV program is sequentially reproduced by controlling the moving image generation unit 109. CG animation generation unit 107
Generates CG animation of emerging sets, characters, props, etc. The voice synthesizing unit 108 outputs a voice signal by generating a speaking voice of an appearing character or a narrator or voice data stored in the storage device (HDD) 102 in advance by controlling the speaker 111.
The speech synthesizer 108 may be an external device for speech synthesis connected to the computer 103 by RS-232C or the like. In that case, it is necessary to input the synthesized voice signal to the computer 103. The moving image generation unit 109 generates a moving image from the moving image data that has been edited in advance and stored in the storage device 102.

The image data generated by the CG animation generation unit 107 and the moving image generation unit 109 is converted into a video signal by the video interface 110. The signal output from the video interface 110 is a monitor 112.
Is output as a video. The storage device 102 is C
The G character modeling data, studio set data, moving image data, audio data, program information 101 and the like are stored. The storage device 102 may be a hard disk, a magneto-optical disk, a flexible disk, a computer via a transmission network, or the like.

Reference numeral 114 is pose information, which is information defining the posture of the character. The pause information 114 is read or edited by the program information generation unit 105 and described in the program information 101. After that, the program generation unit 114 outputs the CG animation.

Reference numeral 115 is animation information, which is information defining the motion of the character. The animation information 115 is read by the program information generation unit 105, described in the program information 101, and then output as a CG animation by the program generation unit 114.

Next, the program information generation unit 105 will be specifically described. The program information generation unit 105 uses the program information 10
It is software that operates on the computer 103 to create 1.

FIG. 2 shows an example of the screen of the program information generating section. Two
Reference numeral 01 is a main screen of the program information generation unit 105. A menu bar 202 displays a menu for activating a preparation window 205 for inputting information in the information display window 203. Reference numeral 203 denotes a program information display window, which is a window for displaying program information in text. A preview window 204 is a window for displaying the program information displayed as text in the information display window 203 as a program through the program generation unit 204.

As a specific operation example of the program information generation unit 105, a method of editing the pose of the appearing character will be described. Here, when the final pose of the character is designated, the program generator 104 changes the pose from the current pose.
The posture is gradually changed while automatically calculating the pose on the way to the designated final posture. The pose is determined by specifying the angles of the joints that make up the character (there are three values of an angle around the x-axis, an angle around the y-axis, and an angle around the z-axis; hereinafter referred to as the joint angle). It

First, from the menu bar 202, "pause"
When is clicked, the pause menu 301 of FIG. 3 is displayed. When "Edit" is clicked from the pose menu 301, the pose edit preparation dialog 401 shown in FIG. 4 is activated.

FIG. 5 shows the pose edit preparation dialog 401.
An enlarged view of is shown. A name combo box 501 is a combo box for selecting a character whose pose is to be edited from the list of characters appearing in the program shown in the program information 101. A pose name edit box 502 is used to input the name of the edited pose. Fifty
A file name edit box 3 is used to input the name of a pose file in which data regarding a pose is described. A reference button 504 is a button for activating a dialog for selecting the pose file. A character display unit 505 displays an image of the character selected in the name combo box 501.
Reference numeral 506 denotes a joint structure display unit, which has TVML language specifications 1.
1 is a visual representation of the joints of the character defined in 1.

The joint clicked on the joint structure display portion 506 is highlighted in red, and the ROT_X edit box 508 and ROT_Y edit box 50 are displayed.
9 indicates that the joint reflects the value input to the ROT_Z edit box 510. The name of the joint is displayed in the joint name combo box 507. further,
The joint for which the joint angle is set is highlighted in blue.

A joint name combo box 507 is clicked to display a list of joint names defined in the TVML language specification 1.1. R from the list above
OT_X edit box 508, ROT_Y edit box 509, ROT_Z edit box 5
A joint that reflects the value input in 10 is selected.

Reference numeral 508 denotes a ROT_X edit box, which is a joint structure display section 506 and a joint name combo box 5.
The angle around the x-axis of the joint selected in 07 is input. 5
A ROT_Y edit box 09 is used to input the angle around the y-axis of the joint selected in the joint structure display unit 506 and the joint name combo box 507. 508 is ROT_
The Z edit box is used to input the angle around the z axis of the joint selected in the joint structure display unit 506 and the joint name combo box 507.

Reference numeral 511 denotes a detail button, and when clicked, a GUI (graphical
Display the user interface). A cancel button 512 is a button for canceling the setting of the joint angle for the joint selected in the joint structure display unit 506 or the joint name combo box 507. Reference numeral 513 denotes a release button, which releases the setting of joint angles for all joints. A preview button 514 is provided on the preview screen 204 displayed on the program information generation unit 105.
It is a button for displaying the edited pose. 515
Is a file save button, and is a button for saving the edited pose data in the storage device 102 as a file. An OK button 516 is a button for outputting the content edited in the pause edit preparation dialog 302 to the program information display window 203.

This output is in the form of a TVML script, and the same number of scripts as the number of joints for which joint angles are set are output. The format of the output script is shown below. character: definepose (nam
e, pose, joint, rotx, roty, ro
tz, movex, movey, movez) name: character name pose: specify pose name joint: specify joint rotx, roty, rotz: specify joint angle movex, movey, movez: specify joint position In pose edit preparation dialog, first , In the name combo box 501, a character for which a pose is set is selected. Next, in the joint structure display unit 506 or the joint name combo box 507, the joint for which the joint angle is set is selected, and the ROT_X edit box 508 and RO are selected.
The joint angle is set by inputting a numerical value in the T_Y edit box 509 and the ROT_Z edit box 510. There are the following three methods for inputting the joint angle. (1) Directly input the numerical value in the edit box (2) Click the spin button (3) Drag the mouse When the joint angle is input by the above method, the joint is defined with the center point defined for each joint as the origin. The angle of is changed.

When the joint angles have been set for all joints, the preview button 514 is clicked to check the results. Then, the save to file button 515 is clicked to save the pose data to a file. Finally,
The OK button 516 is clicked to output the TVML script to the program information display window 203.

The above procedure is a method of defining a pose from a blank state, that is, a state in which nothing is described, but the pose can be edited by reading a previously created pose file.

The procedure for reading the pose file is as follows.
Click 4. Then, the dialog shown in FIG. 6 is displayed.

FIG. 6 is a dialog for reading a pose file, and 601 is a dialog for reading a pose file. A directory selection tool box 602 selects a directory in which a pose file exists. A file icon display unit 603 displays a list of pause files in the directory as icons. 6
A file name edit box 04 displays the file name of the pause file selected in the file icon display portion. A file filter combo box 605 is used to select an extension of a file displayed in the file icon display portion. Normally, only the pose file (extension is ".pose") is displayed. An open button 606 determines the pose file to be read. 607
Is a cancel button.

Pose file read dialog 601
When is activated, a list of pause files is displayed in the file icon display portion 603. The displayed file icons are all the pose files in the directory containing the file in which the information of the character selected in the name combo box 501 of the pose edit preparation dialog 401 is described. Select the pose file to be read from the file icon display area 603, and open button 6
Click 06 to edit pose dialog 4
The pose data is reflected in 01. This allows the user to edit the pose created in advance and the pose that was being edited.

FIG. 7 shows an example of a screen when a pose in which the character's left arm is raised is created by the above procedure.

Further, as a function of supporting the editing of the pose, there is a function of user's viewpoint. This is a function to switch the images from five cameras prepared in front, above, to the right, to the left, and diagonally above the right. The operation method will be described with reference to FIG. A user viewpoint selection menu 801 is a menu list of selectable viewpoints. Reference numeral 802 denotes a front item, which switches the preview screen to a screen from the front. An upper item 803 switches the preview screen to a screen from above. A right item 804 switches the preview screen to a screen from the right side. Reference numeral 805 denotes a left item, which switches the preview screen from the left. 8
06 is a diagonally right upper item, and switches the preview screen from the diagonally right upper screen.

First, when a user viewpoint item is clicked from the menu bar 202 of the program information generation unit 201, a user viewpoint selection menu 801 is displayed. Here, if the front item 802 is clicked, it switches to an image showing the character from the front as shown in FIG. Similarly, item 8 above
When 03 is clicked, it is switched to an image showing the character from above as shown in FIG. Clicking the right item 804 switches to the screen shown in FIG. Clicking the left item 805 switches to the screen shown in FIG. Clicking the upper right diagonal item 806 switches to the screen shown in FIG.

The program information generator 105 described above is used.
As a method to define the pose of the character in
ROT_X, RO to determine the angle of each joint
It is necessary to input three numerical values T_Y and ROT_Z. This is because it is not possible to intuitively recognize in which direction the joint will move by inputting a numerical value, so the work of inputting a numerical value and confirming the moving direction of the joint on the preview screen occurs. However, because of this, many operation errors are likely to occur, the operation is complicated, and work efficiency may be deteriorated.

Therefore, an example of another embodiment of the present invention will be described with reference to FIGS. 14 to 17. First, an example of a method of creating a pose for raising a character's left hand using the present invention will be described. FIG. 14 shows a state in which the pose edit preparation dialog 401 is activated with the front menu 802 selected from the user viewpoint menu 801. Here, 1401 is an example of a preview screen when a character is displayed from the front.

Here, 1402 is a mouse cursor. An X item 1403 indicates an x coordinate component of joint position coordinates. A Y item 1404 indicates a y-coordinate component of joint position coordinates. Reference numeral 1405 denotes a Z item, which indicates az coordinate component of joint position coordinates.

First, the left hand is selected from the joint structure display portion 506 or the joint name combo box 507. Next, the front item 802 is selected from the user viewpoint selection menu 801. At this time, the Z-axis item becomes inactive, indicating that the numerical value in the Z-axis direction cannot be changed. This state is shown in FIG.

Next, the pose edit preparation dialog 401
Drag the mouse on. Then, the value of Y item 1404 increases or decreases in proportion to the movement of the mouse in the vertical direction,
The numerical value of the X item 1403 increases or decreases in proportion to the left-right movement of the mouse. At the same time, on the preview screen 204, the character's left arm moves around the shoulder joint.

FIG. 15 is a diagram showing the character's left arm raised as described above. Reference numeral 1501 indicates a preview screen when the character's left arm has been raised. 1502
Indicates the position of the mouse cursor after raising the character's left arm. Reference numeral 1503 indicates the locus of the mouse cursor from the position of the mouse cursor 1402 shown in FIG. 14 to the movement of the mouse cursor 1502. Reference numeral 1504 indicates a locus along which the left arm of the character has moved by the above operation.

According to the above operation, when the position of the left hand is designated by the mouse cursor, the position of the elbow is automatically calculated so that the position of the elbow becomes a biologically natural position with the position of the shoulder as a base point.
That is, when the distance between the left hand and the shoulder becomes larger than the length of the arm, the left hand does not move to a position longer than the length of the arm. Conversely, if the distance between your left hand and your shoulders is less than the length of your arm, bend your elbow in a natural direction. Therefore, the left hand moves in a circle with the shoulder as the center point and the arm length as the radius.

Similarly, the user viewpoint menu 80 is shown in FIG.
The example of a screen when the left item 802 is selected in 1 is shown. Reference numeral 1601 denotes a preview screen when the character is projected from the left viewpoint. In the case of the viewpoint from the left, the X item 1403 of the pose edit preparation dialog 401 becomes inactive, indicating that the coordinate designation of the left hand is valid for the Y coordinate and the Z coordinate.

FIG. 17 shows a screen example when the upper item is selected in the user viewpoint menu 801. 1701 is a preview screen when the character is projected from the above viewpoint. Y item 14 of pose edit preparation dialog 401
When 04 is inactive and the viewpoint is from above, the coordinate designation of the left hand is valid for the X coordinate and the Z coordinate.

As described above, the position, not the angle, is specified for the joint selected in the joint structure display portion 506 of the pose preparation dialog 401 or the joint name combo box 507. In this case, one base point (left shoulder in the above description) is determined, and the position of the selected joint is determined with this as the center. When another joint (left elbow in the above description) exists between the base point and the selected joint, the joint position is automatically calculated so that the natural position is obtained, and Follow and move.

Furthermore, by combining with the user's viewpoint, a two-dimensional joint position determination method using a mouse becomes possible. Specifically, it is possible to specify the X and Y coordinates in the case of a frontal view, the Y and Z coordinates in the case of a right or left view, and the X coordinates in the case of a top view. And Z coordinate can be specified respectively.

As a result, the position of the joint can be determined more intuitively than in the past, and the work efficiency of editing the pose is improved.

[0047]

As described above, according to the present invention, in the program editing by the software operating on the program information generation unit 105, the direction and angle of the joint connecting the shoulder and arm of the character can be arbitrarily designated. It becomes possible to do.

Furthermore, the pose of the character is determined by designating the position, not the joint angle. Further, the selected joint moves so as to draw a circle centered on the joint serving as the base point. Furthermore, the direction in which the joint moves is limited to two dimensions by combining with the user's viewpoint. This makes it possible to intuitively specify the position of the joint using the mouse.

As described above, the pose can be determined by an intuitive mouse operation, and when the positions of the joints are determined, the positions and angles of the other joints are automatically calculated. It is possible to edit the pose efficiently.

[Brief description of drawings]

FIG. 1 is a diagram showing an example of a functional configuration related to program information included in a television program editing device.

FIG. 2 is a diagram showing an example of a main screen of a conventional program information generation unit.

FIG. 3 is a diagram showing an example of means for activating a dialog for editing a pose by using a program information generation unit of a conventional technique.

FIG. 4 is a diagram showing an example of an operation for editing a pose by using a program information generation unit of the related art.

FIG. 5 is a diagram showing an example of a dialog for editing a pose of a conventional program information generation unit.

FIG. 6 is a diagram showing an example of a dialog for reading a pause file of a conventional program information generation unit.

FIG. 7 is a diagram showing an example of a result of editing a pose using a program information generation unit of the related art.

FIG. 8 is a diagram showing an example of a user viewpoint menu of a conventional technique.

FIG. 9 is a diagram showing an example of a preview screen from a frontal point of view.

FIG. 10 is a diagram showing an example of a preview screen from a viewpoint from above.

FIG. 11 is a diagram showing an example of a preview screen from a viewpoint from the right.

FIG. 12 is a diagram showing an example of a preview screen from a viewpoint from the left.

FIG. 13 is a diagram showing an example of a preview screen from a perspective from the upper right.

FIG. 14 is a diagram showing an example of a state before editing a pose according to the present invention.

FIG. 15 is a diagram showing an example of a state and an operating method after editing a pose according to the present invention.

FIG. 16 is a diagram illustrating an example of a method of editing a pose from the left viewpoint according to the present invention.

FIG. 17 is a diagram illustrating an example of a method of editing a pose from the viewpoint from above according to the present invention.

FIG. 18 is a diagram showing a block configuration example of a program editing device according to the present invention.

[Explanation of symbols]

101: program information, 102: storage device, 103: computer (CPU), 104: program generation unit, 10
5: program information generation unit, 106: sequencer unit, 10
7: CG animation generation unit, 108: voice synthesis unit, 109: moving image generation unit, 110: video interface, 111: speaker, 112: monitor,
113: keyboard input device, 114: pose information,
115: Animation information, 116: Mouse,
117: input / output interface, 118: sound board, 119: memory, 120: signal bus, 20
1: Main screen of program information generation unit, 202: Menu bar, 203: Program information display window, 204: Preview screen, 301: Pause menu, 401:
Pose edit preparation dialog, 501: Name combo box, 502: Pose name edit box, 5
03: File name edit box, 504: Browse button, 505: Character display section, 506: Joint structure display section, 507: Joint name combo box, 50
8: ROT_X edit box, 509: ROT
_Y: edit box, 510: ROT_Z edit box, 511: detailed button, 512: release button, 513: release all button, 514: preview button, 515: save to file button,
516: OK button, 517: Cancel button,
601: Toolbox for directory selection, 60
2: File icon display area, 603: File name edit box, 604: Filter combo box, 605: Open button, 606: Cancel button, 801: User viewpoint menu, 802: Front item, 803: Top item, 804: Right item 805:
Left item, 806: diagonally right upper item, 901: preview screen from the viewpoint from the front, 1001: preview screen from the viewpoint from above, 1101: preview screen from the viewpoint from right, 1201: preview screen from the viewpoint from left, 1301: a preview screen from a perspective from the upper right, 1401: a preview screen from a front perspective, 1402: a mouse cursor, 14
03: X item, 1404: Y item, 1405: Z item, 1501: Preview screen after pose editing, 1
502: mouse cursor, 1503: locus of mouse cursor, 1504: movement of character's left hand, 16
01: Preview screen from the viewpoint from the left, 170
1: Preview screen from the viewpoint from above.

   ─────────────────────────────────────────────────── ─── Continued front page    (72) Inventor Kazuhito Yaegashi             Hitachi, Ltd. 32 Miyuki-cho, Kodaira-shi, Tokyo             Kokusai Electric Koganei Factory (72) Inventor Aki Hashizume             Hitachi, Ltd. 32 Miyuki-cho, Kodaira-shi, Tokyo             Kokusai Electric Koganei Factory (72) Inventor Tsuneya Kurihara             1-280, Higashikoigakubo, Kokubunji, Tokyo             Central Research Laboratory, Hitachi, Ltd. F-term (reference) 5B050 BA06 BA07 BA08 BA12 CA07                       EA24 FA02 FA13                 5C053 FA14 JA16 LA11

Claims (3)

[Claims] 1. In a program editing apparatus, in a program editing method for generating and editing program information for generating a program by a dialogue with a computer on a monitor screen, a position of a joint possessed by a character appearing in the program is designated. By doing so, a program editing method for designating a pose of the character is performed. 2. In a program editing apparatus, in a program editing method for creating and editing program information for creating a program by interactive operation with a computer on a monitor screen, a joint angle of a character appearing in the program is designated. By doing so, a program editing method for designating a pose of the character is performed. 3. The program editing method according to claim 1, wherein the joint position is edited so that the joint position does not become unnatural by providing a biological limitation on the movement distance or direction of the joint. How to edit the program.