WO2009122510A1

WO2009122510A1 - Computer graphics image displaying system and method

Info

Publication number: WO2009122510A1
Application number: PCT/JP2008/056366
Authority: WO
Inventors: カールハインツフーゲル
Original assignee: 堺市; アンリミテッドゲーエムベーハー
Priority date: 2008-03-31
Filing date: 2008-03-31
Publication date: 2009-10-08
Also published as: JP2009245060A

Abstract

Intended is to make it possible to set the correspondences between visual coordinates and absolute coordinates precisely with the raw materials of time-varying images of a camera disposed at one place, thereby to create smooth animations with a three-dimensional model of a high quality. Conversion parameters are determined by setting a plurality of coordinate setting points on a segment featuring a court, by creating the image of a wire frame on the basis of the lines of the court, and by dragging the coordinate setting points of the wire frame to the corresponding court positions on a screen, thereby to acquire the visual coordinates of the coordinate setting points. The figure data of the three-dimensional human body model computer graphics, which express the actions of a finite number intrinsic to the player of each sports event, are constructed in advance into a database by a figure recording table (14), so that the most optimum data record is set for a player object displayed in a real image frame object for the CG creations.

Description

Computer graphics image display system and method

The present invention relates to a computer graphics image display system and method for producing a three-dimensional computer graphics (hereinafter abbreviated as “CG”) moving image that can be reproduced from a free viewpoint using a video of a ball sport such as soccer as a material.

3D computer graphics (hereinafter referred to as CG) that can be played from a free viewpoint using footage such as soccer can be used for highlight scene display from different viewpoints, tactic analysis, and the like.

In conventional CG production work, when an object is expressed three-dimensionally using shadow expression, the depth information of the target object viewed from the observer's position is acquired and reflected in the shadow expression on the target object. It corresponds with. In order to obtain this depth information, multiple viewpoint cameras are set at the observer's position, shadow information is obtained from the target object image captured by each camera, and light source position information is obtained from the plurality of shadow information. In addition, the intensity information of the light source is calculated and stored in the storage device, and the surface image information obtained by viewing the target object from all directions is acquired from the target object video captured by each camera, and the installation position information of each camera is obtained. Based on this, the three-dimensional coordinates of the acquired entire surface image information are calculated and stored in the storage device. When the video is played back on a video playback device as a video viewed from a freely set viewpoint, a shadow representation of the object surface is calculated and displayed based on the light source information and the three-dimensional coordinate information of the entire surface. ing.

However, in the above-described conventional image production system, it is necessary to install a plurality of cameras in consideration of the mouth angle of the camera in order to eliminate blind spots caused by shooting and produce all free viewpoint images. In addition, processing for synthesizing images taken by the cameras on a computer and generating a three-dimensional image occurs. In this processing, the number of processing steps increases in proportion to the number of cameras, and a computer system having high processing capability is required.

Also, in actual field ball sports such as soccer, players in the field play most of the competition time on the field surface, and the movement in the height direction is infrequent. For this reason, in order to express the player's motion, it is necessary to preferentially grasp the time change of the two-dimensional coordinates on the player's field. As a method for grasping the coordinates of each player, a method has been proposed in which a device for grasping coordinates is attached to a part of the player's body and the coordinates are recognized from the result of communication processing between the external system and the device.

However, in the above-described method, the device worn on the player's body is required to have a strong durability that does not cause breakage or dropout during the game. In addition, the game organizer and the player must approve the wearing of the device, and it is necessary to satisfy various requirements such as not obstructing the movement of the player and not obstructing the game progress in all aspects. .

Non-Patent Document 1 discloses a technique for recognizing a two-dimensional coordinate on a field using a television camera image fixed in one place in a television broadcast sports broadcast without using a coordinate grasping device and system. Suppose that a trapezoidal sports field (hereinafter referred to as “visual coordinates”) captured on a TV camera image is a representation of an actual rectangular field (hereinafter referred to as “absolute coordinates”) by perspective projection. A method of calculating a linear transformation determinant between the visual coordinates and the absolute coordinates from the coordinate measurement results of the four points displayed in the visual coordinates has been proposed.

In this non-patent document 1, a total of four lines, two vertical lines and two horizontal lines forming a partial trapezoid of visual coordinates, are recognized by image processing, and a primary transformation determinant is obtained from four intersection points on the straight line. Two methods have been proposed: a “forced method” to be obtained, and a “rapid method” to obtain a primary transformation determinant from a total of four points including the start point and end point of parallel line pairs displayed on the visual coordinates.

However, although the “forced method” can obtain an accurate conversion result, it requires a lot of calculation processing time, while the “quick method” can obtain a result quickly, but only a part of the coat is photographed, for example. In the case of using such a video as a material, there is a high possibility that the material information includes an error, and a correct result may not be obtained.

Furthermore, in both the “forced method” and the “rapid method”, the focus of the camera does not move, that is, the shape of the visual coordinates does not change, as in the case of taking out one video frame of the video material on which the entire surface of the coat has been shot This is a process that is only possible when the TV camera is focused on the player or the ball, as seen in actual TV broadcast video. In the case where the line moves to the outside of the screen, that is, in a so-called “out of line” state, a correct conversion processing result cannot be obtained.

Also, in order to create a three-dimensional CG from an actual moving image, a player animation assignment operation is performed.

In conventional CG production work, when a moving human body is represented three-dimensionally, a point for the model to grasp the joint position in advance is mounted, the model operation is video-recorded, and the video image is reproduced. Extract only the motion of the point from the inside, and connect the extracted points to reproduce as the motion of the human body wire frame, set the direction and intensity of the light ray hitting the wire frame, and consider the shadow formed by the light ray The surface image mapping process is performed on the wire frame. Furthermore, in order to express the generated 3D human body CG as an animation, the 3D human body CG generated for each video frame is compared between adjacent video frames so that a smooth operation is performed during animation playback. The work of fine-tuning the position of the joint on each video frame is performed.

However, in the above-described conventional CG production work, it is essential to attach joint points to the model. This method requires that the game organizer and the player himself / herself approve the wearing of the points, and satisfy various requirements such as not obstructing the movement of the player and not obstructing the game progress in all aspects. There is.

Even if point wearing is permitted, in order to realize 3D CG production by the above-described conventional CG production work, a wire frame is generated from the captured point moving image, and shadow calculation processing is performed. Therefore, it is necessary to execute a large number of processing steps of performing mapping processing at high speed, and a computer system having high processing capability is required. Furthermore, in order to realize a smooth animation, it is necessary to correct the motion of the 3D object displayed on the video frame between all the video frames. For images and the like, the amount of work is enormous.

Also, in a large-scale space such as a concert hall or an outdoor stadium, a model is created by extracting only the 2D video information of the model from the video frame without attaching points to the model. The technique is shown in Non-Patent Document 2.

In this Non-Patent Document 2, first, in order to create a video that captures the court from multiple viewpoints, eight cameras are arranged uniformly so as to surround the court. The height of the camera is set at an angle of about 45 degrees with respect to the horizontal surface of the court in order to avoid duplication of players on the captured video. Next, with each camera, a court image with no player on the court is acquired in advance as a background image and stored in a storage device. Next, each camera shoots a moving image in which a game is actually played, extracts difference information from the above-described background image in each video frame, and extracts player or ball video information as texture information. . Next, a rectangular polygon is placed perpendicular to the viewpoint direction at the place where the player ball exists on the background information, and this polygon is subjected to mapping processing using the texture extracted in the above processing. If the viewpoint direction is set as an intermediate point of the above-described eight photographing cameras, the rectangular polygon is approximated by a two-dimensional image rotated so as to be perpendicular to the viewpoint direction.

According to the above-described method, it is possible to generate a pseudo three-dimensional human body model by automatic processing even if the player does not wear a point. However, this method is located at a position sufficiently far from a display object in a large-scale space. CG production when a viewpoint is installed is assumed. For example, when a viewpoint is installed at a position sufficiently close to a display object, such as when a viewpoint is installed between players, the CG quality evaluation is not sufficient.
"Fast Camera Calibration for the Analysis of Sport Sequences": Univ. of Tchnol. Eindhoven Dirk Farin, Jongong Han, Peter H. et al. N. de With "Live distribution and presentation of large-scale space free viewpoint video using virtual reality technology" Takayoshi Furuyama, Satoshi Kitahara, Yuichi Ota

As described above, in conventional 3D CG production from a free viewpoint, when producing 3D CG from sports broadcast images, it is necessary to install multiple cameras and have high processing capability. I need a computer system.

In addition, in the conventional 3D CG production from a free viewpoint, it is difficult to produce a smooth animation of the 3D model, and it is necessary to attach special points. It takes time and effort.

In view of the above-described problems, the present invention can accurately set the correspondence between visual coordinates and absolute coordinates using a camera moving image installed in one place as a material, and can smoothly perform a smooth operation using a high-quality three-dimensional human body model. An object of the present invention is to provide a computer graphics image display system and method capable of producing a simple animation.

In order to solve the above-described problems, a computer graphics image display system according to the present invention generates and displays a three-dimensional computer graphics image based on an actual image of a player shown in a video material. In the system, a plurality of coordinate setting points are set on a line segment that characterizes the location of the video material, and a wire frame generating means for generating a wire frame serving as a reference for absolute coordinates, and a plurality of coordinate setting points set on the wire frame Input means for designating the corresponding image portion on the screen and inputting the visual coordinates of the coordinate setting points, and the input visual coordinates and absolute coordinates of the at least four coordinate setting points. Calculate the conversion parameters between visual coordinates and absolute coordinates based on the correspondence between Describes the correspondence between the visual coordinates and absolute coordinates of multiple coordinate setting points based on the calculation means, the conversion parameter table in which conversion parameters are recorded corresponding to each video frame, and the obtained conversion parameters. And a coordinate setting point recording table.

Preferably, for a video frame in which no coordinate setting point is input in the conversion parameter table, a conversion parameter obtained by linearly interpolating the conversion parameter obtained in the preceding and subsequent video frames is described.

Preferably, the input of the visual coordinates of the coordinate setting point is such that the wire frame from the wire frame generation unit is displayed so as to overlap the real video screen, and the coordinate setting point on the wire frame is overlapped with the corresponding part of the real video screen. It is characterized by performing.

The computer graphics image display system according to the present invention is a computer graphics image display system that generates and displays a three-dimensional computer graphics image based on an actual image of a player shown in a video material. A figure record table that pre-stores computer graphics data of a three-dimensional human body model that expresses a unique finite number of motions, and means for designating a start frame and an end frame for a series of video frames that set a data record; And control means for allocating, from the figure recording table, computer graphics data of a three-dimensional human body model that expresses an optimum action in accordance with the movement of the player object from the start point frame to the end point frame.

Preferably, the control means is a computer of a three-dimensional human body model that expresses an optimal action based on the moving distance of the player object from the start point frame to the end point frame and the number of video frames from the start point frame to the end point frame. It is characterized by determining graphics data.

Preferably, the control means manages a series of figure data indicating a motion change from the start to the end of the individual motion as a data group expressing the individual motion of the player.

Preferably, the control means manages a continuum of a series of data groups indicating a motion change from the start to the end of the continuous motion as a sequence expressing the continuous motion of the player.

A computer graphics image display method according to the present invention characterizes a location of a video material in a computer graphics image display system that generates and displays a three-dimensional computer graphics image based on an actual image of a player displayed in the video material. A step of generating a wire frame serving as a reference for absolute coordinates by setting a plurality of coordinate setting points on a line segment, and at least four points among the plurality of coordinate setting points set on the wire frame on the screen Specifying the image portion to be input, inputting the visual coordinates of the coordinate setting points, and based on the correspondence between the input visual coordinates and the absolute coordinates of at least four coordinate setting points, The conversion parameters for each video frame and the conversion parameters corresponding to each video frame. And recording the meter, based on conversion parameters calculated, characterized in that it comprises a step of recording the correspondence between the visual coordinates and absolute coordinates for a plurality of coordinate set point.

The computer graphics image display method of the present invention is a computer graphics image display method for generating and displaying a three-dimensional computer graphics image based on an actual image of a player shown in a video material. Preliminarily storing computer graphics data of a three-dimensional human body model expressing a unique finite number of motions, specifying a start frame and an end frame for a series of video frames for setting a data record, and a start point A step of allocating computer graphics data of a three-dimensional human body model expressing an optimum motion from a figure recording table in accordance with the movement of the player object from the frame to the end point frame.

According to the present invention, in a computer graphics image display system for generating and displaying a three-dimensional computer graphics image based on an actual image of a player shown in a video material, a plurality of lines are defined on a line segment characterizing the location of the video material. Coordinate setting points are set, wire frame generating means for generating a wire frame serving as a reference for absolute coordinates, and at least four of the plurality of coordinate setting points set on the wire frame, corresponding images on the screen Based on the correspondence between the input means for designating the part and inputting the visual coordinates of the coordinate setting points, and the input visual coordinates and absolute coordinates of at least four coordinate setting points, the visual coordinates and the absolute coordinates Calculation means for calculating conversion parameters and conversion parameters are recorded corresponding to each video frame. Since there is a conversion parameter table and a coordinate setting point recording table describing the correspondence between visual coordinates and absolute coordinates for a plurality of coordinate setting points based on the obtained conversion parameters, it is easy to process. The visual coordinates of any point on the screen can be converted into absolute coordinates and captured, and the graphics image specified on the absolute coordinates can be converted into arbitrary visual coordinates and displayed on the screen. .

According to the present invention, for a video frame in which no coordinate setting point in the conversion parameter table is input, a conversion parameter obtained by linear interpolation of the conversion parameter obtained in the preceding and following video frames is described. Natural motion conversion parameters can be generated.

According to the present invention, when inputting the visual coordinates of the coordinate setting point, the wire frame from the wire frame generation unit is displayed so as to be superimposed on the real video screen, and the coordinate setting point on the wire frame is superimposed on the corresponding part of the real video screen. Thus, conversion parameters with high accuracy can be calculated with a simple operation.

According to the present invention, in a computer graphics image display system for generating and displaying a three-dimensional computer graphics image based on an actual image of a player shown in a video material, a finite number unique to each game item player is provided. Figure recording table that pre-stores computer graphics data of a 3D human body model representing motion, means for specifying the start and end frames for a series of video frames that set the data records, and the start and end frames It is equipped with a control means that assigns the computer graphics data of the 3D human body model that expresses the optimal motion from the figure recording table according to the movement of the player object up to the frame, so it is natural with simple processing. Compilation of 3D human body model of movement It can be displayed over data graphics.

According to the present invention, the optimal motion is expressed based on the moving distance of the player object from the start point frame to the end point frame and the number of video frames from the start point frame to the end point frame. The computer graphics of the 3D human body model can be displayed.

According to the present invention, a series of figure data indicating a change in motion from the start to the end of individual motion is managed as a data group representing individual motions of the player. Can be displayed.

According to the present invention, since a continuum of a series of data groups indicating a motion change from the start to the end of a continuous motion is managed as a sequence representing the continuous motion of the player, computer graphics with smooth motion Can be displayed.

According to the present invention, in a computer graphics image display system for generating and displaying a three-dimensional computer graphics image based on an actual image of a player shown in a video material, a plurality of lines are defined on a line segment characterizing the location of the video material. The step of generating a wire frame that is used as a reference for absolute coordinates and specifying the corresponding image portion on the screen for at least four of the multiple coordinate setting points set on the wire frame Then, conversion parameters between the visual coordinates and the absolute coordinates are calculated based on the step of inputting the visual coordinates of the coordinate setting points and the correspondence between the input visual coordinates and the absolute coordinates of at least four coordinate setting points. And a step of recording conversion parameters corresponding to each video frame, and The step of recording the correspondence between the visual coordinates and the absolute coordinates for a plurality of coordinate setting points based on the converted parameters is provided. Can be converted into absolute coordinates and captured, and a graphics image designated on the absolute coordinates can be converted into arbitrary visual coordinates and displayed on the screen.

According to the present invention, in a computer graphics image display method for generating and displaying a three-dimensional computer graphics image based on an actual image of a player shown in a video material, a finite number unique to each game item player is provided. Preliminarily storing computer graphics data of a 3D human body model representing motion, specifying a start frame and an end frame for a series of video frames for setting a data record, and from a start frame to an end frame The computer graphics data of the 3D human body model that expresses the optimal motion is allocated from the figure recording table according to the movement of the player object until the third order of natural movement with simple processing. Computer graphic of former human body model It can display box.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 shows the configuration of a computer graphics image display system according to an embodiment of the present invention.

As shown in FIG. 1, a computer graphics image display system according to an embodiment of the present invention includes a computing device operation device 1, a computing device 2, a video data playback device 4, a video data input interface 5, and a video data memory. 6, display data processing device 7, VRAM (Video Random Access Memory) 7, work display 9, wire frame generation unit 10, conversion parameter table 11, coordinate setting point recording table 12, figure recording table 14 and an operation determination table 15.

The computing device operating device 1 is an input device for an operator to perform various operations on the computing device 2. The arithmetic device operating device 1 selects and operates various processes of the arithmetic device 2, and at the same time, coordinates setting points displayed as wire frames on the work display 9 and visual coordinates displayed as a video screen. Since it is responsible for operations to manipulate the placement with the coordinate setting points and to input the visual coordinate position of the player's object on the video screen and collect it as a parameter on the absolute coordinate, such as a mouse, tablet, touch panel, etc. It is desirable that the device has excellent operability for inputting the coordinates.

The computing device 2 performs processing such as various video information processing and numerical calculation. As will be described later, the arithmetic device 2 displays a wire frame for a video frame that is a target of visual coordinate setting work, acquires visual coordinates on the screen corresponding to the coordinate setting point of the wire frame, and sets the coordinates. A conversion parameter is calculated from the correspondence between the visual coordinates of the points and the absolute coordinates. Further, the arithmetic unit 2 selects an optimum sequence from the figure recording table 14 based on the selected player position and moving direction and the number of recorded video frames at the time of creating the CG, and the operation of the player based on the sequence is performed. Processing is performed in which the figure data constituting the animation is assigned to the player position on each video frame. The arithmetic device 2 further performs various calculations and processes.

The video data reproduction device 4 is a device that reproduces actual video data as a material. More specifically, the video data playback device 4 is a video movie, a VCR (Video Cassette Recorder), a DVD (Digital Versatile Disc) player, or a HDD (Hard Disk Drive) video player. One feature of the embodiment of the present invention is that it does not depend on the format or quality of the actual video data as the material. Any video can be used as a material, such as a video of a soccer game broadcast on television, a video of a soccer game taken by an amateur from the audience, or a video of a soccer game aired several years ago.

The video data input interface 5 is an interface for capturing video data from the video data playback device 4 into the system.

The video data memory 6 is a memory for storing video data captured from the video data playback device 4. It is desirable that the video data memory 6 has a sufficient capacity to store video images of a large number of video frames.

The display data processing device 7 performs a process of collecting the work results of the operator and the processing results of the arithmetic device and sending them to the VRAM 7. The VRAM 7 stores the processing result of the display data processing device as image information.

The work display 9 is a display for displaying the contents of the VRAM 7. As the work display 9, an LCD (Liquid Crystal Display) display or a CRT (Cathode-Ray Tube) display is used.

The wire frame generation unit 10 generates a wire frame image for setting a plurality of coordinate setting points serving as a reference for absolute coordinates (see FIG. 3). In the embodiment of the present invention, a plurality of coordinate setting points are set on a line segment that characterizes the location of a video material (soccer court), such as a center circle, a side line, a penalty area line, and a penalty arc. Correspondence between visual coordinates and absolute coordinates is obtained for these coordinate setting points.

In the conversion parameter table 11, conversion parameters are described for each video frame as shown in FIG. 5 (see FIG. 5). In the coordinate setting point recording table 12, the relationship between the visual coordinates and absolute coordinates is described for each coordinate setting point (see FIG. 4).

The figure recording table 14 is a database for storing figure data. Here, the figure refers to a three-dimensional human body model used on a computer graphics screen.

In the embodiment of the present invention, data groups expressing operations such as “walking”, “starting running”, “sprinting all the way”, “stopping suddenly”, and the like are prepared in advance. At the time of CG production, video frames in which the moving direction and moving speed of the position recording target player change rapidly are selected while frame-by-frame playback of the actual video data is performed, and the foot portion of the player video is marked.

Based on the marking, the figure recording table 14 includes, for each player, the arrangement of each player, the result of rotation correction of figure data, the flag for identifying the manually set video frame, the type of data group, and the type of data group. The figure data record ID and the like are recorded (see FIG. 10).

The operation determination table 15 is a determination table for determining the operation of the player based on the selected player position and moving direction and the number of recorded video frames at the time of CG creation (see FIG. 12). These tables will be described later.

The computer graphics image display system according to the embodiment of the present invention configured as described above can be used for display with changing the viewpoint of a highlight scene, tactical analysis, or the like from actual video data as material.

制作 Production of 3D CG from such a free viewpoint usually requires enormous operations. In addition, there are various restrictions such as multiple dedicated cameras, and the camera work is stable, such as soccer game images shot by amateurs from the audience seats and soccer game images aired several years ago. There are many cases in which an unusable image or a deteriorated image cannot be used.

In contrast, in the computer graphics image display system according to the embodiment of the present invention, a plurality of coordinate setting points are set on the line segment characterizing the coat, and a wire frame image based on the coat line is generated. The coordinate setting point of the wire frame is dragged to a position on the corresponding coat on the screen and superimposed to obtain the visual coordinates of the coordinate setting point, and the conversion parameter is obtained. As a result, an enormous amount of arithmetic processing is not required, and it is possible to display a 3D CG screen without depending on the format and quality of the actual video data as a material, free from various constraints.

Further, in the embodiment of the present invention, figure data of a three-dimensional human body model computer graphics expressing a finite number of movements specific to each game item player is created in advance as a database by the figure recording table 14 and becomes a target of CG production. An optimum data record is set for the player object displayed in the actual video frame. In addition, smooth motion animation is realized by handling motion as a series of figure data, and by treating continuous motion as a sequence. These will be described in detail below.

First, the coordinate setting in the present invention will be described. In the embodiment of the present invention, a court such as a center circle, a side line, a penalty area line, a penalty arc or the like with absolute coordinates viewed from directly above is displayed on a screen displaying an actual video image to be produced by CG. A wire frame including a line segment is displayed in an overlapping manner, and four coordinate setting points set on the wire frame are selected and dragged to the same point shown on the actual video frame. . Thereby, visual coordinates capable of one-to-one correspondence with absolute coordinates can be set.

That is, FIG. 2 explains the conversion between visual coordinates and absolute coordinates. The coat is rectangular when viewed from directly above, but appears to be trapezoidal from the camera's viewpoint. Here, an image viewed from directly above when the original rectangle is formed is indicated by absolute coordinates, and an image viewed from the viewpoint of the camera is defined as visual coordinates.

As shown in FIG. 2, four points (X1, Y1) to (X4, Y4) on the visual coordinates are four points (X′1, Y′1) to (X′4, Y′4) on absolute coordinates. Has been converted. This conversion is shown by the following equation.

From the above equation, solving for X (x1 to x4) and Y (y1 to y4) gives the following.

From the above results, four points (X1, Y1) to (X4, Y4) on the visual coordinates and four points (X′1, Y′1) to (X′4, Y′4) on the absolute coordinates are obtained. If the correspondence is known, the conversion parameter H can be calculated and any point on the visual coordinate can be converted on the absolute coordinate.

If the image is always from a fixed camera, four points (X1, Y1) to (X4, Y4) on the visual coordinates and four points (X′1, Y′1) to (X ′) on the absolute coordinates are always displayed. 4, Y′4) does not change. Therefore, the conversion parameter H obtained once can be used as it is.

However, as described above, in the embodiment of the present invention, various video data are used such as a video of a soccer game taken by an amateur from a audience seat, a video of a soccer game aired several years ago. With these video data, the camera is moved by panning or tilting, zooming up, zooming down, or scene changes. In addition, since the entire court cannot be captured with a single camera, the four reference points are not always displayed on the screen.

Therefore, in the embodiment of the present invention, a wire frame including a line segment characterizing a coat such as a center circle, a side line, a penalty area line, and a penalty arc as shown in FIG. 3 is used.

In other words, the soccer court has line segments that characterize the court, such as the center circle, side line, penalty area line, and penalty arc. As shown in FIG. 3, a plurality of coordinate setting points P1 to P33 are set on these line segments. For these coordinate setting points P1 to P33, as shown in FIG. 4, a coordinate setting point recording table 12 for recording the correspondence between visual coordinates and absolute coordinates is prepared.

Here, since the length of the center circle, side line, penalty area line, penalty arc, etc. is determined in advance by the competition rules, the absolute coordinates of the coordinate setting points P1 to P33 can be set based on this. If the visual coordinates of at least four of the coordinate setting points P1 to P33 are obtained from the video screen, as described above, the correspondence between the visual coordinates and the absolute coordinates can be determined, and based on this, other coordinates can be obtained. The visual coordinates of the set point can be obtained by calculation.

The visual coordinates at the coordinate setting points P1 to P33 are input by displaying the wire frame shown in FIG. 3 on the real screen and dragging the point on the wire frame to the corresponding point on the screen. For example, when the coordinate setting point P1 on the wire frame is dragged so as to overlap with the corresponding point on the corner of the coat on the real screen, the visual coordinates of the coordinate setting point P1 are input, and the visual setting of the coordinate setting point P1 Correspondence between coordinates and absolute coordinates is acquired. Similarly, the visual coordinates of the corresponding points are acquired for the other coordinate setting points.

As described above, if at least four visual coordinates among the coordinate setting points P1 to P33 are obtained, the correspondence between the visual coordinates and the absolute coordinates is known, and the conversion parameters are calculated using the correspondence. It is possible to convert the visual coordinates of an arbitrary point to absolute coordinates.

Also, the visual coordinates based on the calculated conversion parameters are displayed overlaid on the selected actual video frame, the deviation between the actual video frame and the visual coordinates on the display is confirmed, and the actual video frame with a large deviation is set as the setting target The conversion parameter can be corrected by selecting a coordinate setting point having a large visual coordinate deviation from the actual video frame and dragging to a point on the actual video frame.

Also, when panning, tilting, zooming up and zooming down are performed, the correspondence between visual coordinates and absolute coordinates will shift. In this case, the point on the wire frame may be dragged again to the corresponding point on the screen for correction. At this time, a coordinate setting point different from the previous coordinate setting point can be used.

Also, conversion parameters change continuously when the camera is panned or tilted, zoomed up or down. For this reason, when visual coordinates are corrected, conversion parameters in the meantime can be obtained by interpolation.

The conversion parameters obtained for each video frame are recorded in the conversion parameter table 11 as shown in FIG. In the conversion parameter table 11, conversion parameters (h00 to h21) and video frame IDs to be set are recorded in association with each other. Among these conversion parameters, the manual setting flag “1” is added to the conversion parameters obtained in the actual video frame. The conversion parameter in the meantime is obtained by interpolation, and the manual setting flag is “0”.

FIG. 6 is a flowchart describing a work procedure in the coordinate setting described above. In FIG. 6, first, the operator operates the video data playback device 4 through the operation of the arithmetic device 2, and the video data serving as the material of the three-dimensional computer graphics image is transferred to the video data memory through the video data input interface 5. 6 (step S21). Data to be reproduced by the video data reproduction device 4 is required to be able to perform normal reproduction, frame-by-frame reproduction and rewind for each video frame on the work display 9, and the data type and data format are not limited.

Next, the operator performs frame-by-frame playback or rewinding of the video data stored in the video data memory 6 through the operation of the arithmetic device 2 (step S22), and a video frame that is a visual coordinate setting work target is selected. Select (step S23). For the selected video frame, the wire frame generator 10 displays the wire frame having the absolute coordinates shown in FIG. 3 and causes the arithmetic device 2 to calculate the conversion parameter H.

The operator performs visual coordinate setting work using this result (step S24), displays the set visual coordinates on the screen, superimposes the video frames that are the work targets, finely corrects the position, and calculates The apparatus 2 is caused to calculate the conversion parameter H for every fine correction (step S25).

After the fine correction work is completed, the arithmetic unit 2 records the conversion parameters H obtained as a result of steps S24 and S25 in association with all the video frame IDs on the conversion parameter table 11 shown in FIG. The manual setting flag of the setting target video frame ID on the parameter table 11 is set to “1” (step S26).

When step S25 is completed, the computing device 2 counts the number of video frames for which the visual coordinates are set by the operator, and determines whether the number of video frames is greater than 1 (step S27).

If the number of video frames is 1, the worker performs visual coordinate code setting work for another video frame (step S22).

When the number of video frames is 2 or more, the visual coordinate setting parameter is linearly interpolated with the elapsed time as a variable for each video frame in which the visual coordinates existing between the set video frames are not set. Automatically calculated and set, and recorded in association with the interpolation target video frame ID on the conversion parameter table 11 (step S28). At this time, the manual setting flag is set to “0”.

The operator performs the processing from step S22 to step S28 on a plurality of video frames in the processing target video data, and finally confirms the setting state of the visual coordinate code throughout, and if necessary, Fine adjustment is performed on the video frame that needs to be adjusted (step S29).

Next, details of the contents of the visual coordinate setting process in step S24 will be described with reference to FIG. FIG. 7 is a flowchart showing details of step S24 of the present invention. The operator causes the wire frame generation unit 10 to display a wire frame of absolute coordinates on the selected actual video frame through the operation of the arithmetic device 2 (step S241).

Next, the operator sets coordinates on the processing target video data displayed on the actual video frame from among the coordinate setting points P1 to P33 (see FIG. 3) shown on the wire frame. Four points are selected, and the coordinate setting point on the wire frame is dragged to the same point on the actual video frame (step S242).

When the match between the four points is completed, the arithmetic unit 2 calculates the coordinate conversion parameter H between the absolute coordinates and the visual coordinates, and based on this, the remaining coordinate setting points not selected in step S242 are calculated. Coordinates on the visual coordinates are automatically calculated, described in the coordinate setting point recording table 12 shown in FIG. 4, lines are drawn between the points, and are displayed on the screen as the visual coordinates of the wire frame (step S243).

The operator compares the visual coordinates of the wire frame displayed as a result of the automatic calculation of the conversion parameter H with the line displayed on the actual video frame, and finds the portion where the line and the coordinate setting point are shifted, Finely adjust the coordinates of the coordinate setting point where the deviation occurs. The arithmetic unit 2 recalculates the coordinate conversion parameter H every time fine adjustment occurs, and redisplays the coordinate setting points and lines displayed on the visual coordinate court according to the calculation result. Make corrections while watching.

Next, details of the automatic coordinate calculation processing in step S243 in FIG. 7 will be described with reference to FIG. FIG. 8 is a flowchart showing details of step S243 of the present invention. As shown in FIG. 4, a correspondence table between visual coordinates and absolute coordinates of the wire frame is prepared in advance on the coordinate setting point recording table 12. When the operator drags and matches the coordinate setting points on the four selected wire frames to the points on the actual video frame through step S242, the arithmetic unit 2 causes the points on the work display 9 to be dragged and matched. Visual coordinates (X1, Y1) to (X4, Y4) are acquired (step S2431).

Next, the calculation device 2 calculates the conversion parameter H from the visual coordinates and absolute coordinates of each coordinate setting point by the calculation shown in the equation (1). The conversion parameter H and the coordinate (X′n, Y′n) on the absolute coordinate corresponding to the selected coordinate setting point are substituted into the equation (2) to calculate the visual coordinate coordinate (Xn, Yn). (Step S2432).

Next, the arithmetic unit 2 reads the line segment configuration information in the coordinate setting point recording table 12 shown in FIG. 4 and draws the line segments on the work display 9 for all the calculated coordinate setting points. Thus, the visual coordinates reflecting the conversion parameter H are drawn on the work display 9 (step S2433).

When the operator confirms that the visual coordinates reflecting the conversion parameter H substantially match the actual video frame to be set, the operator displays another actual video frame in order to display the next actual video frame to be set. (Step S2434).

When step S2434 is executed, the arithmetic device 2 records the obtained conversion parameter H in the conversion parameter table 11 shown in FIG. 5 in association with the actual video frame ID to be set (step S2435). .

Next, details of the process of fine correction of visual coordinates in step S25 in FIG. 6 will be described with reference to FIG. FIG. 9 is a flowchart showing details of step S25 of the present invention. In FIG. 9, when the operator displays the actual video frame to be finely corrected, the arithmetic unit 2 converts the conversion associated with the displayed actual video frame ID recorded in the conversion parameter table 11 shown in FIG. The visual coordinates based on the parameter H are displayed superimposed on the selected actual video frame (step S251).

The worker confirms the deviation between the actual video frame on the display and the visual coordinates, and selects the actual video frame having a large deviation as a setting target (step S252). The operator selects a coordinate setting point with a large deviation between visual coordinates and the actual video frame, and drags and corrects the point to a point on the actual video frame (step S253).

The arithmetic unit 2 acquires the coordinates of the coordinate setting point that has been drag-corrected as (X′4, Y′4) (step S254), and three coordinate setting points on the visual coordinates other than the coordinate-setting point ID that has been drag-corrected. Then, the coordinates are acquired as (X1, Y1) to (X3, Y3) (step S255).

Next, the calculation device 2 calculates the conversion parameter H from the visual coordinates and absolute coordinates of each coordinate setting point by the calculation shown in the equation (1). The conversion parameter H and the coordinate (X′n, Y′n) on the absolute coordinate corresponding to the selected coordinate setting point are substituted into the equation (2) to calculate the visual coordinate coordinate (Xn, Yn). (Step S256).

Next, the arithmetic unit 2 reads the line segment configuration information in the coordinate setting point recording table 12 shown in FIG. 4 and draws the line segments on the work display 9 for all the calculated coordinate setting points. Thus, the visual coordinates reflecting the conversion parameter H are drawn on the work display 9 (step S257).

When the operator confirms that the visual coordinates reflecting the conversion parameter H substantially match the actual video frame to be set, the operator displays another actual video frame in order to display the next actual video frame to be set. (Step S258).

When step S2434 is executed, the arithmetic device 2 records the obtained conversion parameter H in the conversion parameter table 11 shown in FIG. 5 in association with the actual video frame ID to be set (step S259). . Note that steps S256 to S259 are the same as steps S2432 to S2435 in FIG.

Next, a method for managing the figure record table will be described. In the embodiment of the present invention, figure data of a three-dimensional human body model computer graphics expressing a finite number of movements specific to each game item player is stored in the figure recording table 14 in advance as a database, and a real image to be produced by CG. An optimal figure data record is assigned to the player object displayed in the frame.

In other words, human motion is complex, but if it is limited to the motion of a soccer player, it can be aggregated into several motions such as stationary, walking, running, dribbling, trapping, and heading. Therefore, a series of figure data expressing these actions is determined in advance, and the player object displayed in the actual video frame that is the target of CG production is determined to be most appropriate among these actions. Those are read from the database to generate a CG image.

For example, when the player's movement with respect to the number of recorded video frames is calculated, the speed of the player's movement is obtained. This makes it possible to determine the operation of the player, such as “still”, “walking”, or “running”. When it is determined that “walking”, figure data indicating a series of “walking” operations is read from the figure recording table 14.

In addition, in the embodiment of the present invention, “motion” is a collection of a plurality of figure data so that motions such as “walking”, “starting running”, “sprinting as much as possible”, and “stopping suddenly” can be expressed smoothly. Treated as a data group. Further, a sequence is composed of a plurality of data group sequences so that a series of actions of the player can be expressed as a smooth moving image. This will be described.

In the CG production, an identification ID is assigned to all displayed players. Then, the position recording target player is selected on the screen, and while the actual video data is played back frame by frame, the video frame in which the moving direction and moving speed of the position recording target player change rapidly is selected, and the foot portion of the player video is selected. Is marked. Thereby, the start point video frame and the end point video frame are designated. At this time, the state of the figure data of each player on the absolute coordinate court is recorded.

FIG. 10 shows the structure of the figure recording table 14 in which the state of the figure data of each player on the absolute coordinate court at this time is recorded. As shown in FIG. 10, the figure recording table 14 includes, for each video frame, the arrangement of each player on the absolute coordinates, rotation of figure data, a flag for identifying a manually set video frame, and the type of data group. The figure data record ID is recorded.

The figure data rotation item is the result of the figure data rotation correction by the 3D direction correction tool. The three-dimensional direction correction tool is a correction tool for displaying a player and a ball in a three-dimensional space. That is, the display screen on the work display 9 is two-dimensional, and it is difficult to operate the ball and the player image in the three-dimensional space shown on the absolute coordinate court with a general pointer display. Therefore, a simple operation is realized by using the three-dimensional direction correction tool shown in FIG.

FIG. 13A shows a three-dimensional traveling direction pointer. This pointer has a shape of a Cartesian coordinate axis in a three-dimensional direction, shows the traveling direction of the ball and the player image in an easy-to-see manner, and realizes simple correction work. Further, the pointer can be tilted in any of the three-dimensional directions. As shown in FIG. 13B, the moving direction of the ball and the player image can be easily seen even in the tilted state, and simple correction is possible. Realize work. FIG. 13C shows a three-dimensional rotation pointer. This pointer indicates a three-dimensional rotation plane and rotation direction, and realizes a simple correction work when rotating the player image.

In FIG. 10, the manual setting flag indicates whether or not recording by marking has actually been performed. As described above, a video frame in which the moving direction and moving speed of the position recording target player changes rapidly is selected, and the player video is marked. In this case, the manual setting flag is “1”. Since the movement is continuous between the start video frame and the end video frame, each item is obtained by interpolation, and the manually set flag is “0” for the interpolated video frame.

The data group is a collection of a plurality of figure data for expressing a series of individual actions such as “walking”, “running”, “trapping”, and “heading”.

For example, the "Walking" data group has human body model figure data that shows "right foot forward", human body figure data that shows "right foot on the ground", and "left foot forward" state It can be expressed as a collection of figure data of a human body model that indicates, and figure data of a human body model that indicates a state of “attaching the left foot to the ground”. Each figure data is managed by a wire frame ID. The direction and intensity of the light ray that hits the wire frame is set, and a surface image mapping process is performed in consideration of the shadow formed by the light ray, so that a three-dimensional human body model computer graphics image of the figure is obtained.

Figure data record ID is a set record ID assigned based on a sequence derived from the order of data groups.

That is, FIG. 11 shows the structure of a sequence for expressing a smooth operation. The sequence is configured as a continuum of data groups indicating individual operations, and the data group is configured as a continuum of figure data record IDs indicating operation changes from the start to the end of individual operations.

[Figure data records are selected in advance so that a smooth transition can be made from one data group to the next. Also, the breakdown of the sequence, that is, the composition of the data group, can be selected from a certain data group so that it can express movements other than simple driving such as “falling down” and “trapping” even at the same moving speed. Is prepared to choose.

At the time of CG creation, an optimal data group is selected based on the position and moving direction of the selected player and the number of recorded video frames, and an optimal sequence is selected based on a continuous data group configuration for the player. The figure data constituting the motion animation of the player is assigned to the player position on each video frame. For example, when the player's movement with respect to the number of recorded video frames is calculated, the speed of the player's movement is obtained. This makes it possible to determine the operation of the player, such as “still”, “walking”, or “running”. FIG. 12 is an example of the operation determination table 15 that determines the operation as described above. In this example, an operation is assigned corresponding to the calculated speed, but a direction or the like may be further used.

If it is determined that the action of the player is a series of actions of “walking”, “running”, and “trap on the right foot inside”, the action of the player is “walking” from “still” as shown in FIG. , “Run”, “run”, and “trap on the right foot inside” are regarded as a series of sequences, and the figure data of the data group indicating this sequence is sequentially read from the figure recording table 14.

As described above, in the embodiment of the present invention, by using the sequence, the figure data of the data group “run” is read between “walking” and “running”. It can be replayed and the single movement is not discontinuous.

Next, the processing of figure data according to the embodiment of the present invention will be described in detail. FIG. 14 is a flowchart showing processing of figure data according to the present invention. In FIG. 14, first, the worker assigns identification IDs to all displayed players through the entire real video data to be produced by CG (step S31).

Next, the worker performs frame-by-frame playback of the actual video data and performs position recording processing for all players. As a specific procedure for position recording processing, a position recording target player is selected on the screen (step S33), and the moving direction and moving speed of the position recording target player change abruptly while moving and reproducing the actual video data. The video frame to be selected is selected, and the foot portion of the player video on the work display 9 is marked (step S34).

When the position recording of all the players is completed (step S32), the worker then displays the first video frame of the real video data, and plays back the real video data frame-by-frame for the trajectory of the ball projected on the ground. On the other hand, a video frame in which the moving direction and moving speed of the ball change rapidly is selected, and the locus of the ball projected on the ground on the work display 9 is marked (step S35).

When the work is completed for all the video frames, the worker again displays the first video frame of the real video data, and the ball is moved in the height direction based on the trajectory of the ball projected on the ground recorded in step S35. A video frame in which the moving direction and the moving speed of the moving image rapidly change is selected, and a ball trajectory including a height element is recorded using a three-dimensional direction correction tool (step S36).

When the work is completed for all the video frames, the operator operates the arithmetic device 2 to reproduce the animation of the three-dimensional computer graphics based on the recorded player and ball position, and assigns the sequence and data group. Correction is performed (step S37).

At the time of reproduction of the computer graphics, the arithmetic unit 2 selects an optimal data group based on the selected position and moving direction of one player and the number of recorded video frames, and optimizes based on a continuous data group configuration for the player. A sequence is selected, and figure data constituting a motion animation of the player based on the sequence is assigned to the player position on each video frame.

After the allocation is completed, the operator displays the actual video data and the data after the allocation in a superimposed manner, confirms the motion, direction, and inclination of the allocated figure data while frame-by-frame playback is performed. With the 3D direction correction tool, or by making fine adjustments such as correcting the start or end position of the sequence animation, or correcting the scale of the sequence animation, The operation of the player is produced as a CG animation.

Next, the player ID assignment process and work procedure in step S31 in FIG. 14 will be described in detail with reference to FIG. FIG. 15 is a flowchart showing details of step S31 of the present invention. In FIG. 15, first, the operator operates the arithmetic device 2 to display the first video frame of the real video data in which the visual coordinates are already set on the work display 9 by operating the arithmetic device 2. (Step S311).

Next, the operator operates the arithmetic device 2 through the arithmetic device operation device 1 to give an identification ID to all players displayed on the first video frame of the video data on the work display 9. (Step S312).

Next, the worker reproduces the video data (step S313) (step S314), and determines whether or not a new player to which no ID is assigned is displayed on the video frame in step S312 (step S315). If it is displayed, an ID is assigned to the player using the same procedure as in step S312 (step S316). The operations from step S313 to step S316 are repeated until the last video frame.

Next, the work procedure of the position record player selection process in step S33 in FIG. 14 will be described in detail with reference to FIG. FIG. 16 is a flowchart showing details of step S33 of the present invention. In FIG. 16, first, the operator operates the arithmetic device 2 to display the first video frame of the position setting start work on the work display 9 (step S331).

Next, the operator operates the arithmetic device 2 through the arithmetic device operating device 1, and selects a position setting target player from the players displayed on the work display 9 (step S332).

Next, the work procedure of the player position recording process in step S34 in FIG. 14 will be described in detail with reference to FIG. FIG. 17 is a flowchart showing details of step S34 of the present invention. In FIG. 17, the operator operates the computing device 2 through the computing device operating device 1, and puts a mark indicating the position of the player on the visual coordinates on the displayed position setting target player (step S341). . A detailed flowchart of step S341 is shown in FIG.

Next, the worker reproduces the video data and determines whether or not it is the final video frame (step S342). If it is not the final video frame, the video frame is postponed (step S343), and the position setting target player A video frame in which the running direction or running speed changes abruptly is searched for by frame advance (step S344). If there is a video frame that is changing rapidly, the process returns to step S341. If there is no corresponding final video frame, the position setting target player's position is marked on the final video frame (step S345).

Next, the ball position recording process and work procedure in step S35 of FIG. 14 will be described in detail with reference to FIG. FIG. 18 is a flowchart showing details of step S35 of the present invention. In FIG. 18, first, the worker operates the arithmetic device 2 to display the first video frame of the ball position setting start work on the work display 9 (step S351).

Next, the operator operates the computing device 2 through the computing device operating device 1, and the visual of the ball is displayed on the portion of the ball image displayed on the work display 9 corresponding to the locus projected on the ground. A mark indicating the position on the coordinates is attached (step S352).

Next, the worker reproduces the video data and determines whether or not it is the final video frame (step S352). If it is not the final video frame, the video frame is postponed (step S354) and the ball moves or A video frame whose moving speed is changing rapidly is searched by frame advance (step S355). If there is a current video frame in which the ball is changing rapidly, the process returns to step S352, and if there is no corresponding video frame, the ball position is marked on the final video frame (step S356).

Next, the ball locus recording process and work procedure in step S36 of FIG. 14 will be described in detail with reference to FIG. FIG. 19 is a flowchart showing details of step S36 of the present invention. In FIG. 19, first, the operator operates the arithmetic device 2 to display a soccer court screen based on absolute coordinates having coordinate axes in the vertical, horizontal, and height directions on the work display 9 (step S361). .

Next, the operator operates the calculation device 2 to display the movement trajectory information of the ball recorded through step S34 on the soccer court screen as a line segment and a point indicating the ball (step S362).

Next, the worker instructs to set the display of the figure animation of the player involved in the ball together with the movement trajectory information of the ball. In response to the setting instruction, the arithmetic unit 2 assigns and displays a sequential prepared as a player figure animation involved in the ball (step S363).

The worker compares the movement of the point indicating the ball with the movement of the figure animation, sets correction points on the soccer court screen so that they are in a natural positional relationship with each other (step S364), and sets the three-dimensional direction correction tool. Using this, the height direction component of the line segment of the ball trajectory is corrected (step S365).

After the correction is completed, the movement of the player and the ball is reproduced as an animation on the soccer court screen (step S366), the mutual movement and positional relationship are reconfirmed (step S367). This process is repeated until there is no difference between the positional relationship between the player and the ball throughout the animation.

Next, the sequence assignment and correction process and work procedure in step S37 in FIG. 14 will be described in detail with reference to FIG. FIG. 20 is a flowchart showing details of step S37 of the present invention. In FIG. 20, first, the operator operates the arithmetic device 2 to display the actual video data video frame to be processed on the work display 9 (step S371).

Next, by the operator's operation, the computing device 2 converts the movement trajectory of the ball set on the absolute coordinates and the player figure animation to the visual coordinates on the displayed real video frame based on the equation (2). It is converted into visual coordinates viewed from the same viewpoint, superimposed on the actual video data video frame displayed on the work display 9 (step S372), and the video data and animation are reproduced in synchronization (step S372). S373).

Next, the operator operates the arithmetic device 2, compares the actual image on the video data video frame with the player figure (step S374), and selects a point where a deviation occurs (step S375).

Next, when there is a deviation in the contact position of the ball and the player involved in the ball, the operator corrects the contact position by operating the arithmetic device 2 and shifting the arrangement position of the player figure. (Step S376) If there is a discrepancy in the player's movement, the figure data constituting the data group is corrected by operating the three-dimensional direction correction tool and the arithmetic unit 2, and another figure is necessary if necessary. The record is changed (step S377).

Next, the worker reproduces only the corrected ball and the player animation (step S378), confirms whether or not the ball trajectory and the player's motion are reproduced smoothly (step S379), and is not smooth. If YES in step S376, the flow returns to step S376. If smooth, the flow returns to step S373 to repeat until the ball trajectory and the player's motion are smoothly reproduced throughout the animation.

Next, details of the player animation assignment procedure shown in step S341 in FIG. 17 will be described with reference to FIG. FIG. 21 is a flowchart showing details of step S341 of the present invention. In FIG. 21, first, the figure recording table 14 shown in FIG. 11 is made writable (step S34101).

In step S341 shown in FIG. 17, when the operator records a mark at the foot of the player's position in the first video frame, the arithmetic unit 2 detects the mark coordinate on the visual coordinate (step S34102), (2 ) Is converted from visual coordinates to absolute coordinates (step S34103), the actual video frame ID and absolute coordinates are written into the figure recording table 14 of FIG. 11, and 1 indicating the position mark setting by the operator is recorded in the manual setting flag. (Step S34104).

The worker and the computing device 2 perform the operations and processes from step S34101 to step S34104 for all real video frames for which position marks should be set (step S34105).

When the worker and the coping processing device 2 finish the work and processing for all setting video frames, the arithmetic device 2 extracts a plurality of data records in which 1 is input to the manual setting flag (step S34106). ), The video frame ID included in the data record and the absolute coordinates recorded in the record are extracted, and the number of video frames existing between the extracted video frames adjacent to each other, that is, the required between video frames. The time is counted (step S34107).

Similarly, the difference in the absolute coordinates of the arrangement of each player between the extracted video frames adjacent to each other, that is, the position movement distance of the player between the video frames is calculated (step S34108).

The result of step S34108 is divided by step S34107, and the moving speed between the video frames of the player is calculated (step S34109). In accordance with the setting criteria of the motion determination table 15 shown in FIG. Data group candidates to be displayed are displayed (step S34110), and the operator sets the corresponding data group (step S34111).

When the setting of the data group is completed over the entire video frame, the arithmetic unit 2 sets the corresponding sequence from the arrangement of the data groups (step S34112), and the figure data record ID recorded in the sequence is set to the figure recording. Recording is performed for each real video frame record in the table 14 (step S34113), and the figure recording table 14 is closed (step S34114).

As described above, in the embodiment of the present invention, a plurality of coordinate setting points are set on a line segment that characterizes a coat, and an image of the wire frame based on the line of the coat is generated. The coordinate setting point is dragged to a position on the corresponding court on the screen and overlapped to obtain the visual coordinates of the coordinate setting point to obtain the conversion parameter. As a result, an enormous amount of arithmetic processing is not required, and it is possible to display a 3D CG screen without depending on the format and quality of the actual video data as a material, free from various constraints.

For example, when panning, tilting, zooming up and zooming down are performed, the correspondence between visual coordinates and absolute coordinates is shifted. In this case, the point on the wire frame may be dragged again to the corresponding point on the screen for correction.

Also, in a soccer game, when switching from defense to offense, the camera pans from the court on the ally side to the opponent's court following the movement of the ball. In this case, in the previous time, assuming that the coordinate setting points P1, P4, P5, and P8 of the teammate side in FIG. , P8 comes off the screen. At this time, for example, the coordinate setting points P3, P6, P7, and P11 of the opponent's court may be used.

Further, in the embodiment of the present invention, figure data of a three-dimensional human body model computer graphics expressing a finite number of movements specific to each game item player is created in advance as a database by the figure recording table 14 and becomes a target of CG production. An optimum data record is set for the player object displayed in the actual video frame. In addition, smooth motion animation is realized by handling motion as a series of figure data, and by treating continuous motion as a sequence.

In the embodiment of the present invention, a user can freely set a viewpoint using a camera moving image installed in one place where the entire subject space cannot be added and the focus of the video always changes. It is possible to easily and accurately set an accurate two-dimensional coordinate axis on the court, which is necessary for producing a quality three-dimensional image.

Further, in the embodiment of the present invention, the display viewpoint is set at a position sufficiently close to the target object through a short work time and a simple content work using the television broadcast video data of the sports game in which the player is not wearing points as a material. Even in this case, it is possible to produce a smooth animation using a high-quality three-dimensional human body model. From this, it is possible to produce high-quality three-dimensional CG animation from any sports game relay video by preparing a figure record table, data group, and sequence specific to each sport.

The present invention is not limited to the above-described embodiment, and various modifications and applications are possible without departing from the spirit of the present invention.

In addition to the production of computer graphics images of sports matches such as soccer, the present invention provides accurate object space necessary for producing 3D images using video data of broadcasting studios, theaters, concert halls, etc. It can be used to set a two-dimensional coordinate axis including a depth direction. Create 3D computer graphics as a new video product from past entertainment video data, create 3D computer graphics at the scene of an accident from the camera image of a drive recorder installed in a sales vehicle, security camera It is possible to produce 3D computer graphics of crime scenes from images.

It is a block diagram of an example of the computer graphics image display system of the embodiment of the present invention. It is explanatory drawing of the conversion parameter in this invention. It is explanatory drawing of the wire frame in this invention. It is explanatory drawing of the coordinate setting point recording table in this invention. It is explanatory drawing of the conversion parameter table in this invention. It is a flowchart used for description of the coordinate transformation process of this invention. It is a flowchart which shows the detail of step S24 of this invention. It is a flowchart which shows the detail of step S243 of this invention. It is a flowchart which shows the detail of step S25 of this invention. It is explanatory drawing of the figure recording table in this invention. It is explanatory drawing of the sequence in this invention. It is explanatory drawing of the operation | movement determination table in this invention. It is explanatory drawing of the three-dimensional direction correction tool in this invention. It is a flowchart which shows the process of the figure data of this invention. It is a flowchart which shows the detail of step S31 of this invention. It is a flowchart which shows the detail of step S33 of this invention. It is a flowchart which shows the detail of step S34 of this invention. It is a flowchart which shows the detail of step S35 of this invention. It is a flowchart which shows the detail of step S36 of this invention. It is a flowchart which shows the detail of step S37 of this invention. It is a flowchart which shows the detail of step S341 of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Computation device operation device 2 Computation device 3 Visual coordinate coordinate setting point 5 Video data input interface 6 Video data memory 7 Display data processing device 9 Work display 10 Wire frame generation unit 11 Conversion parameter table 12 Coordinate setting point recording table 14 Figure Recording table 15 Operation determination table

Claims

In a computer graphics image display system for generating and displaying a three-dimensional computer graphics image based on an actual image of a player shown in a video material,
Wire frame generating means for setting a plurality of coordinate setting points on a line segment characterizing the location of the image material and generating a wire frame serving as a reference for absolute coordinates;
Input means for designating corresponding image portions on the screen and inputting visual coordinates of the coordinate setting points for at least four of the plurality of coordinate setting points set on the wire frame;
Calculating means for calculating conversion parameters between the visual coordinates and the absolute coordinates based on the correspondence between the input visual coordinates and the absolute coordinates of the at least four coordinate setting points;
A conversion parameter table in which the conversion parameters are recorded corresponding to each video frame;
A computer graphics image display system comprising: a coordinate setting point recording table in which a correspondence relationship between visual coordinates and absolute coordinates for the plurality of coordinate setting points is described based on the obtained conversion parameters.
The conversion parameter obtained by linearly interpolating the conversion parameter obtained in the preceding and subsequent video frames is described for a video frame in which the coordinate setting point is not input in the conversion parameter table. Computer graphics image display system.
The input of the visual coordinates of the coordinate setting point is such that the wire frame from the wire frame generation unit is displayed so as to overlap the real video screen, and the coordinate setting point on the wire frame is superimposed on the corresponding part of the real video screen. The computer graphics image display system according to claim 1, wherein the computer graphics image display system is performed.
In a computer graphics image display system for generating and displaying a three-dimensional computer graphics image based on an actual image of a player shown in a video material,
A figure record table in which computer graphics data of a three-dimensional human body model representing a finite number of movements specific to each sporting event player is pre-stored;
Means for designating a start frame and an end frame for a series of video frames for setting the data record;
Control means for allocating, from the figure recording table, computer graphics data of a three-dimensional human body model that expresses an optimum motion according to the movement of the player object from the start point frame to the end point frame. Computer graphics image display system.
The control means is a three-dimensional human body model that expresses an optimal motion based on the moving distance of the player object from the start point frame to the end point frame and the number of video frames from the start point frame to the end point frame. 5. The computer graphics image display system according to claim 4, wherein the computer graphics data is determined.
The said control means manages the continuous body of a series of figure data which shows the operation | movement change from the start to the end of individual operation | movement as a data group expressing the individual operation | movement of a player. Computer graphics image display system.
The said control means manages the continuous body of a series of data groups which show the operation | movement change from the start to the end of a continuous operation | movement as a sequence showing the continuous operation | movement of a player. Computer graphics image display system.
In a computer graphics image display system for generating and displaying a three-dimensional computer graphics image based on an actual image of a player shown in a video material,
Setting a plurality of coordinate setting points on a line segment that characterizes the location of the video material, generating a wire frame that is a reference for absolute coordinates;
Specifying the corresponding image portion on the screen for at least four of the plurality of coordinate setting points set on the wire frame, and inputting the visual coordinates of the coordinate setting points;
Calculating conversion parameters between visual coordinates and absolute coordinates based on the correspondence between the input visual coordinates and absolute coordinates of the at least four coordinate setting points;
Recording the conversion parameter corresponding to each video frame;
Recording a correspondence relationship between visual coordinates and absolute coordinates for the plurality of coordinate setting points based on the obtained conversion parameters. A computer graphics image display method, comprising:
In a computer graphics image display method for generating and displaying a three-dimensional computer graphics image based on an actual image of a player shown in a video material,
Pre-accumulating computer graphics data of a three-dimensional human body model representing a finite number of movements specific to each sporting event;
Designating a start frame and an end frame for a series of video frames for setting the data record;
Allocating, from the figure recording table, computer graphics data of a three-dimensional human body model that expresses an optimal motion in accordance with the movement of the player object from the start point frame to the end point frame. Computer graphics image display method.