JP4231216B2 - Virtual scene shooting method and apparatus - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a virtual scene shooting method and an apparatus therefor, and in particular, a virtual scene shooting method for generating a virtual scene by synthesizing both images of the same movement by linking camera work in virtual space and camera work in real space. It relates to the device.
[0002]
[Prior art]
2. Description of the Related Art Conventionally, a camera captures the real world and a subject that are spread in front of a lens by a cameraman with a viewfinder, and shoots it as a real photograph. When shooting a moving subject such as a person or animal, in addition to tracking the subject with the finder, the cameraman sometimes keeps an eye on the finder to observe and predict the subject's movement while checking the subject directly, The camera is set to an appropriate angle of view for the subject.
[0003]
Also called a virtual studio where the space conditions of the real space and the virtual space are matched, and the live-action video and computer graphics (hereinafter referred to as “CG”) video are combined and displayed on the same screen for viewing. In this system, changes in the direction and position of the camera in the real space and the lens value correspond to the camera in the virtual space (hereinafter referred to as “virtual camera”), and the spatial position between the real space and the virtual space. Consistency of relationship. In other words, the subject of the camera operation was on the side of the real space moved by the cameraman.
[0004]
[Problems to be solved by the invention]
In recent years, as program production and video production by video composition of live-action video and CG video using virtual studios increase, subjects in virtual space that cannot be seen in real space (hereinafter referred to as “virtual subjects”) More and more photographers have to follow the movement.
[0005]
However, the cameraman can only see the CG video that is the virtual subject in the camera finder or the video that is combined with the live-action video on the floor monitor. The subject's movement cannot be predicted and shot.
[0006]
The combined video of CG video and live-action video that can be viewed with a camera finder, etc., causes a delay of one frame (about 33 ms in the case of NTSC system) in order to perform video synthesis processing. Camera operations performed while looking at the camera result in a delay in camera work. Because of this, virtual body There was a problem that it was very difficult to move the camera in real space following the instantaneous movement and high-speed movement of the camera.
[0007]
The present invention has been made in view of the above points. body The real-time camera can be moved following the instantaneous movement and high-speed movement of the virtual object. body It is an object of the present invention to provide a virtual scene photographing method and apparatus capable of obtaining a composite image that accurately tracks the movement of the camera.
[0008]
[Means for Solving the Problems]
The invention according to claim 1 is a motion of a virtual camera that captures the virtual subject in a virtual space based on the motion of the virtual subject by computer graphics or control information thereof. data Set
Above Virtual camera movement data , A real space camera that captures the real space corresponding to the movement of the virtual camera Converted to control data ,
Driven based on the control data By synthesizing the video photographed by the real space camera and the video photographed by the virtual camera based on the movement of the virtual object by the computer graphics or the control information thereof, to obtain the video of the virtual scene ,
Virtual subject body The real-time camera can be moved following the instantaneous movement and high-speed movement of the virtual object. body It is possible to obtain a composite image that accurately tracks the movement of the video.
[0009]
According to the second aspect of the present invention, the movement of the virtual camera that captures the virtual subject in the virtual space based on the movement of the virtual subject by computer graphics or the control information thereof. data Virtual camera operation setting means for setting,
Above Synchronize the movement data of the virtual camera with the movement of the virtual camera Real space camera to shoot real space Convert to control data Real space camera operation setting means,
Set by the real space camera operation setting means The drive based on the control data Video synthesizing means for synthesizing a video photographed by a camera in a real space and a video photographed by the virtual camera based on the movement of the virtual subject by the computer graphics or the control information thereof to obtain a video of the virtual scene And having
Virtual subject body The real-time camera can be moved following the instantaneous movement and high-speed movement of the virtual object. body It is possible to obtain a composite image that accurately tracks the movement of the video.
[0010]
The invention according to claim 3 is the virtual scene photographing device according to claim 2,
Set by the real space camera operation setting means The drive based on the control data Delay means for delaying at least one of a video photographed by a camera in a real space and a video photographed by the virtual camera based on the movement of the virtual object by the computer graphics or control information thereof; By
It is possible to synchronize the motion timing of the video captured by the real space camera and the video of the virtual subject by computer graphics.
[0011]
DETAILED DESCRIPTION OF THE INVENTION
FIG. 1 is a block diagram of a first embodiment of the virtual scene photographing apparatus of the present invention. In this embodiment, when the movement of the virtual subject is determined in advance, the movement of the virtual camera is set based on the movement, and the movement data of the virtual camera is adapted to the automatic photographing camera device in the real space, A virtual subject moving in a virtual space is photographed by an automatic photographing camera device.
[0012]
In FIG. 1, as an initial setting, the scales of the real space in the automatic photographing camera device 10 and the virtual space in the CG video generation device 20 are matched. Here, in order to match the scales of the real space shown in FIG. 2A and the virtual space shown in FIG. _r = (0,0,0) and the origin O of the virtual space _v = (0,0,0) and the width x of the real space _r , Height y _r , Depth z _r X in the virtual space _v Axis, y _v Axis, z _v Correspond to each axis.
[0013]
Then, the scale of the virtual space is set so that, for example, the length 1 m in the real space becomes the unit length of the virtual space, and the point A of the real space _r Coordinates (x _r , Y _r , Z _r ) Is (x ₁ , Y ₁ , Z _l ), The corresponding point A in the virtual space _v Coordinates (x _v , Y _v , Z _v ) Is (x ₁ , Y ₁ , Z ₁ ).
[0014]
Next, the camera C in the real space shown in FIG. _r And the virtual camera C shown in FIG. _v Match the position and orientation of the. Real space camera position (x _r , Y _r , Z _r ) = (X ₂ , Y ₂ , Z ₂ ) Is the width x from the origin of the real space set earlier _r , Height y _r , Depth z _r Obtained by surveying based on direction. Then, the position of the virtual space (x _v , Y _v , Z _v ) = (X ₂ , Y ₂ , Z ₂ ), Virtual camera C with the same lens scale and aperture scale as real space _v Set. And real space camera C _r And virtual camera C _v Adjust the direction of.
[0015]
For example, real space camera C _r Position (x _r , Y _r , Z _r ) = (X ₂ , Y ₂ , Z ₂ ) To origin O _r Camera direction when looking at (-x ₂ , -Y ₂ , -Z ₂ This is virtual camera C _v Is the same. And this virtual camera C _v Direction (-x ₂ , -Y ₂ , -Z ₂ ) With reference to the rotation amount of the camera in two spaces (θx as shown in FIG. ₃ , Θy ₃ , Θz ₃ ). As a result, two cameras C _r , C _v Tune as if they were shooting in the same direction with the same camera in the same place in the same space.
[0016]
On the other hand, in the virtual subject motion setting unit 21 in the CG video generation device 20 shown in FIG. 1, the virtual subject H is a point (x _v , Y _v , Z _v ) = (X ₄ , Y ₄ , Z ₄ ) Is set to perform an operation of making one round in t seconds on a circumference of radius = r on a plane parallel to the x and z planes. When the movement of the virtual subject H is replaced with real space, a point (x _r , Y _r , Z _r ) = (X ₄ , Y ₄ , Z ₄ ) Around the circumference of the radius rm in parallel with the floor surface.
[0017]
In the virtual camera operation setting unit 22, the virtual subject H is assigned to the virtual camera C. _v The position (x _v , Y _v , Z _v ) = (X ₅ , Y ₅ , Z ₅ ). A virtual camera C that follows the movement of the virtual subject H and moves in t seconds. _v The camera work of the virtual camera C _v Set while looking at the screen corresponding to the viewfinder, and save the motion data.
[0018]
Subsequently, the virtual camera operation setting unit 22 determines the position of the automatic shooting camera device in the real space and the virtual camera C in the virtual space. _v Point corresponding to the position of (x _r , Y _r , Z _r ) = (X ₅ , Y ₅ , Z ₅ ). And virtual camera C _v Is converted into control data for the photographing unit 12 and supplied to the drive control unit 11 of the automatic photographing camera device 10.
[0019]
The drive control unit 11 drives the imaging unit 12 based on the supplied control data, executes camera work, and images the real space. The real space video obtained by the imaging unit 12 is output as, for example, an NTSC video signal and supplied to the image composition device 30 through the video delay device 15.
[0020]
In the rendering unit 23 in the CG video generation device 20, a virtual camera C that performs the set camera work. _v The image of the virtual subject H photographed in step (2) is converted (rendered) into an NTSC video signal and output. This video signal is supplied to the image composition device 30 through the video delay device 25.
[0021]
The image composition device 30 performs image composition of the image signal of the virtual subject H from the CG image generation device 20 and the image of the real space photographed by the automatic photographing camera device 10, thereby causing a virtual that does not exist in the real space. It can be displayed as an image of the subject H moving in real space.
[0022]
For example, when the movement data of the virtual subject is converted into the control data of the automatic shooting camera device, the delay of the real space video is increased due to a delay, and the timing of the movement of both videos is shifted at the stage of image synthesis. The video delay device 25 adds the same delay to the video of the virtual subject, thereby matching the timings of both. Conversely, when the delay of the video of the virtual subject is large, the video delay device 15 adds the same delay to the video in the real space, thereby matching the timings of both.
[0023]
In the present invention, the orientation and movement of the virtual camera, lens information, and the like are set based on the position and movement data of the virtual subject of the CG, and the automatic shooting camera device is driven in accordance with this to perform shooting in real space. This makes it possible to track a virtual subject faster and more accurately, and a real-world camera accurately tracks the movement of a virtual subject that could not be completely followed by humans. This makes it possible to achieve highly reliable automatic shooting and efficient production.
[0024]
FIG. 6 shows a block diagram of a second embodiment of the virtual scene photographing apparatus of the present invention. In this embodiment, a virtual subject whose movement is not determined in advance, such as a case where a virtual subject is arbitrarily moved by a human operation or the like, or a case where the motion of a virtual subject is automatically calculated by a random number or the like, is automatically taken. It is something to shoot with.
[0025]
In FIG. 6, as an initial setting, the scales of the real space in the automatic photographing camera device 10 and the virtual space in the CG video generation device 40 are matched. Here, in order to match the scales of the real space shown in FIG. 2A and the virtual space shown in FIG. _r = (0,0,0) and the origin O of the virtual space _v = (0,0,0) and the width x of the real space _r , Height y _r , Depth z _r X in the virtual space _v Axis, y _v Axis, z _v Correspond to each axis.
[0026]
Then, the scale of the virtual space is set so that, for example, the length 1 m in the real space becomes the unit length of the virtual space, and the point A of the real space _r Coordinates (x _r , Y _r , Z _r ) Is (x ₁ , Y ₁ , Z _l ), The corresponding point A in the virtual space _v Coordinates (x _v , Y _v , Z _v ) Is (x ₁ , Y ₁ , Z ₁ ).
[0027]
Next, the camera C in the real space shown in FIG. _r And the virtual camera C shown in FIG. _v Match the position and orientation of the. Real space camera position (x _r , Y _r , Z _r ) = (X ₂ , Y ₂ , Z ₂ ) Is the width x from the origin of the real space set earlier _r , Height y _r , Depth z _r Obtained by surveying based on direction. Then, the position of the virtual space (x _v , Y _v , Z _v ) = (X ₂ , Y ₂ , Z ₂ ), Virtual camera C with the same lens scale and aperture scale as real space _v Set. And real space camera C _r And virtual camera C _v Adjust the direction of.
[0028]
For example, real space camera C _r Position (x _r , Y _r , Z _r ) = (X ₂ , Y ₂ , Z ₂ ) To origin O _r Camera direction when looking at (-x ₂ , -Y ₂ , -Z ₂ This is virtual camera C _v Is the same. And this virtual camera C _v Direction (-x ₂ , -Y ₂ , -Z ₂ ) With reference to the rotation amount of the camera in two spaces (θx as shown in FIG. ₃ , Θy ₃ , Θz ₃ ). As a result, two cameras C _r , C _v Tune as if they were shooting in the same direction with the same camera in the same place in the same space. Up to this point, the process is the same as in the first embodiment.
[0029]
As shown in FIG. 7, in a studio in real space, three wooden pillars 101, 102, and 103 stand in a chroma key back set 100, and a scene in which a virtual subject jumps around is generated. The operator 105 that moves the virtual subject moves the control input device 110 such as a joystick while viewing the monitor screen 109 that displays the entire virtual space where the three virtual pillars 106, 107, and 108 corresponding to the pillars of the real space exist. The virtual subject 112 is moved so as to avoid the pillars 106, 107 and 108.
[0030]
Control input information output from the control input device 110 is supplied from the terminal 41 shown in FIG. 6 to the virtual subject motion setting unit 42 and the virtual camera motion setting unit 43 of the CG video generation device 40. Based on the control input information, the virtual subject motion setting unit 42 generates a video in which the virtual subject 112 flies around in the virtual space and supplies the video to the rendering unit 45.
[0031]
The virtual camera operation setting unit 43 determines the optimal angle of view of the virtual camera for photographing the virtual subject 112 based on the control input information, and generates motion data corresponding to the camera work of the virtual camera to generate the data conversion unit 44. And supplied to the rendering unit 45. The data conversion unit 44 converts the movement data of the virtual camera into control data of the automatic shooting camera device and supplies the control data to the drive control unit 11 of the automatic shooting camera device 10.
[0032]
The drive control unit 11 drives the photographing unit 12 based on the supplied control data to execute camera work, and the three wooden pillars 101, 102, and 103 stand in the chroma key back set 100. Shoot the state of the space. As shown in FIG. 8B, the real space video obtained by photographing the wooden pillar 101 is output as, for example, an NTSC video signal and supplied to the image composition device 30 through the video delay device 15.
[0033]
In the rendering unit 45 in the CG video generation device 40, the image of the virtual subject 112 flying around the virtual space as shown in FIG. 8A photographed by the virtual camera that performs the set camera work is converted into an NTSC video signal. Convert and output. This video signal is supplied to the image composition device 30 through the video delay device 25.
[0034]
The image synthesizing device 30 includes the video signal of the virtual subject 112 from the CG video generating device 40 shown in FIG. 8A and the video of the pillar 101 taken by the automatic photographing camera device 10 shown in FIG. 8B. By performing image composition, the virtual subject 112 can be displayed as an image shown in FIG.
[0035]
It is to be noted that, when the timing of movement of both videos is shifted at the stage of image synthesis, the timing of both is matched using the video delay devices 15 and 25 as in the first embodiment.
[0036]
In addition to this, the optimum angle of view of the virtual camera can be automatically and quickly determined from the rendered image of the virtual subject using the automatic photographing camera system described in Japanese Patent Application No. 8-153133 proposed by the present applicant. There is also a method in which the automatic photographing camera device 10 is controlled based on the optimum angle of view and the real space is photographed in conjunction with the virtual camera. The automatic photographing camera system described in the above Japanese Patent Application No. 8-153133 includes a sensor camera for photographing a wide-angle image corresponding to the field of view of the camera operator, and a camera operation including a photographing direction by a control signal from the outside. A subject to be photographed is recognized from a controllable photographing camera and a stationary body or moving body in a wide-angle image photographed by the sensor camera, and the current position of the subject in a three-dimensional space is sequentially measured. And a three-dimensional position measurement unit that generates reliability information indicating the reliability of the measurement information and transmits the reliability information together with the measurement information, and the measurement information from the three-dimensional position measurement unit in the three-dimensional space of the subject A motion analysis unit that analyzes the motion of the camera, a reliability calculation unit that calculates the reliability of the measurement information based on the reliability information from the three-dimensional position measurement unit, and the camera work of the imaging camera There are a plurality of control modes for each form, the control mode is automatically switched according to the reliability calculated by the reliability calculation unit, and the camera work of the photographing camera is controlled according to the control parameter of the control mode after the switching. A camera work control unit,
[0037]
Further, when the form includes an automatic tracking control form and the reliability does not exceed a threshold, the automatic tracking control form for controlling automatic tracking according to the motion of the subject analyzed by the motion analysis unit Control in the first control mode of
When the reliability exceeds a threshold value, a second control mode of the automatic tracking control mode that controls automatic tracking based on prediction information of the movement of the subject obtained from a plurality of measurement information up to the present time And control by the second control mode until the reliability is equal to or less than a threshold value,
The form includes an automatic tracking control form, and when the period in which the reliability exceeds the threshold exceeds an allowable period, the current control mode is controlled based on a model expression of the movement of the subject. Switch to the third control mode of the automatic tracking control mode, and control according to the third control mode until the reliability is equal to or less than a threshold,
A first control mode of the field-of-view control mode in which the form includes a field-of-view control mode of the shooting camera, and when the reliability exceeds a threshold value, the field of view of the shooting camera is expanded to an allowable value. And control by the first control mode of the visual field control mode until the reliability is equal to or lower than a threshold value,
The reliability information from the three-dimensional position measurement unit includes at least one information of first information indicating a failure of the sensor camera and second information indicating the size of the recognized subject,
In the case where the current position in the three-dimensional space to be imaged is measured using two sensor cameras, the three-dimensional position measurement unit recovers the failure from the point in time when a failure of one of the sensor cameras is detected. Until the time of detecting the measurement based on the wide-angle image from the other sensor camera of the current position,
When the reliability exceeds a threshold or when the measurement information is not input and the subject in the field of view of the photographing camera is included, a stationary body or a moving body in the photographed image of the photographing camera is included. A measurement information creation unit that recognizes the subject to be photographed from the inside and uses the measurement information obtained by measuring the current position in the screen of the subject as measurement information from the three-dimensional position measurement unit, thereby automatically This enables tracking and unattended shooting.
[0038]
Next, a third embodiment of the virtual scene shooting method of the present invention will be described. In this embodiment, a scene in which the subject of motion to be photographed changes from a real space subject to a virtual subject on the way is photographed by an automatic photographing camera device.
[0039]
Also in this embodiment, the process of synchronizing the camera in the real space and the virtual camera at the same location in the same place in the same place so as to photograph the same direction is the same as in the first and second embodiments. It is the same.
[0040]
FIG. 9 shows a block diagram of a third embodiment of the virtual scene photographing apparatus of the present invention. In FIG. 9, as an initial setting, the scales of the real space in the automatic photographing camera device 50 and the virtual space in the CG video generation device 60 are matched.
[0041]
Next, the movement of the performer in the real space is photographed using the automatic photographing camera device 50 of the automatic photographing camera system described in Japanese Patent Application No. 8-153133. The performer 117 shown in FIG. 10A has a bow 118. As shown in FIG. 10B, small position sensors 119, 120, and 121 that can be mounted on a bow such as a magnetic sensor are provided at the upper end position of the bow, the center position of the bow, the lower end position of the bow, and the center position of the string, respectively. , 122 are attached, and the small position sensors 119 to 122 are used to change the shape of the bow 118 as shown in FIGS. 10B and 10C or in the real space as shown in FIG. The position of the bow 118 can be measured in real time. In addition to the magnetic sensor, an optical sensor or an ultrasonic sensor may be used.
[0042]
Detection signals from the small position sensors 119 to 122 are supplied from a terminal 66 shown in FIG. 9 to the position detection unit 61 of the CG image generation device 60, and position information and shape information are obtained. The position information and shape information are supplied to the virtual subject operation setting unit 62 and the virtual camera operation setting unit 63. The virtual subject motion setting unit 62 generates an image of a CG arrow based on the position information and the shape information and supplies it to the rendering unit 65. The virtual space shown in FIG. 11B includes a CG arrow 124 as a virtual subject. Based on the position information and shape information of the bow 118 in the real space shown in FIG. The position, orientation, movement, etc. of 124 are generated.
[0043]
The virtual camera operation setting unit 63 determines the optimum angle of view of the virtual camera for photographing the CG arrow 124 based on the position information and the shape information, and generates motion data corresponding to the camera work of the virtual camera to generate data. The data is supplied to the conversion unit 64 and the rendering unit 65. The data conversion unit 64 converts the movement data of the virtual camera into control data of the automatic shooting camera device and supplies the control data to the drive control unit 51 of the automatic shooting camera device 50.
[0044]
In this case, the position D of the arrow in the virtual space _v = (X ₆ , Y ₆ , Z ₆ ) Is the center position D of the string of the bow 118 in the real space. _r = (X ₆ , Y ₆ , Z ₆ ), And the arrow 124 is the center position D of the bow string. _r = (X ₆ , Y ₆ , Z ₆ ) And the center position E of the bow 118 _r = (X ₇ , Y ₇ , Z ₇ ) Connecting the extension line direction (x ₆ -X ₇ , Y ₆ -Y ₇ , Z ₆ -Z ₇ )
[0045]
Performer 117 draws this bow 118 and pulls it away. _r = (X ₆ , Y ₆ , Z ₆ ) Release the puller. The fingertip of the performer 117 who is pulling a string is loaded with a small electrode switch 126 through which a weak current flows, for example, when in contact, and the current does not flow through the electrode switch 126 at the moment when the finger is released. The signal of the electrode switch 126 is supplied from a terminal 67 shown in FIG. 9 to the virtual subject operation setting unit 62 and virtual camera operation setting unit 63 of the CG video generation device 60 and the drive control unit 51 of the automatic photographing camera device 50.
[0046]
The CG arrow 124 indicates the center position E of the bow 118. _r = (X ₇ , Y ₇ , Z ₇ ) And the center position D of the string _r = (X ₆ , Y ₆ , Z ₆ ) Connecting the extension line direction (x ₆ -X ₇ , Y ₆ -Y ₇ , Z ₆ -Z ₇ ) The speed at which the arrow 124 of the CG pops out is set in advance, and the movement of the arrow 124 is calculated from the direction of the arrow 124. The data converter 64 also calculates motion data representing the motion of the virtual camera from the direction and speed of the arrow 124, and converts the motion data into control data for the automatic camera device 64 to drive the automatic camera device 50. It supplies to the control part 51 and drives the imaging | photography part 52. FIG. All of these operations start from the moment the finger is released.
[0047]
That is, the subject of the movement to be photographed moves from the performer 117 in the real space to the arrow 124 of the virtual subject. Then, the rendered image of the arrow 124 of the virtual subject photographed by the virtual camera and the real space image photographed by the automatic photographing camera device 50 are combined, resulting in a virtual moving at high speed that does not exist in the real space. The arrow 124 of the subject can be displayed as an image moving at high speed in the real space.
[0048]
If the timing of movement is shifted at this compositing stage, for example, if a delay occurs when rendering a virtual subject shot by the virtual camera, a delay equivalent to that is generated in the video shot by the automatic shooting camera device. Thus, matching the timings of both is the same as in the first and second embodiments.
[0049]
In this way, the real space can be accurately photographed by the real space camera in response to the dynamic movement and the strange movement unique to the virtual subject. In addition, because the same shooting can be repeated many times, special shooting such as multiple composition, simulation and rehearsal when performing with a performer and a virtual subject can also be performed, and composite video production of live action and virtual subject This can contribute to the efficiency of so-called virtual scene generation work.
[0050]
The virtual camera operation setting units 22, 43, 63 correspond to the virtual camera operation setting means described in the claims, the virtual camera operation setting unit 22, the data conversion units 44, 64 correspond to the real space camera operation setting means, The video composition device 30 corresponds to the video composition means, and the video delay devices 15 and 25 correspond to the delay means.
[0051]
【The invention's effect】
As described above, the invention according to claim 1 is a motion of a virtual camera that captures a virtual subject in a virtual space based on the motion of the virtual subject by computer graphics or control information thereof. data Set Virtual camera movement data The movement of the virtual camera In synchrony Real space camera to shoot real space Converted to control data , Driven based on control data A virtual scene image is obtained by synthesizing an image captured by a real space camera and an image captured by the virtual camera based on the movement of a virtual object by computer graphics or control information thereof to obtain a virtual scene image. Subject body The real-time camera can be moved following the instantaneous movement and high-speed movement of the virtual object. body It is possible to obtain a composite image that accurately tracks the movement of the video.
[0052]
According to the second aspect of the present invention, the movement of the virtual camera that captures the virtual subject in the virtual space based on the movement of the virtual subject by computer graphics or the control information thereof. data Virtual camera operation setting means for setting, Synchronize virtual camera movement data with virtual camera movement Real space camera to shoot real space Convert to control data Set by the real space camera operation setting means and the real space camera operation setting means. Driven based on control data Video synthesizing means for synthesizing a video photographed by a camera in a real space and a video photographed by the virtual camera based on the movement of a virtual object by computer graphics or control information thereof to obtain a video of a virtual scene; By having a virtual subject body The real-time camera can be moved following the instantaneous movement and high-speed movement of the virtual object. body It is possible to obtain a composite image that accurately tracks the movement of the video.
[0053]
In the invention according to claim 3, it is set by the real space camera operation setting means. Driven based on control data By having delay means for delaying at least one of a video photographed by a real space camera and a video photographed by the virtual camera based on the movement of a virtual object by computer graphics or control information thereof. It is possible to synchronize the motion timing of the video captured by the camera in the real space and the video of the virtual subject by computer graphics.
[Brief description of the drawings]
FIG. 1 is a block diagram of a first embodiment of a virtual scene photographing apparatus of the present invention.
FIG. 2 is a diagram for explaining processing for synchronizing a real space camera and a virtual camera;
FIG. 3 is a diagram for explaining processing for synchronizing a real space camera and a virtual camera;
FIG. 4 is a diagram for explaining a process of synchronizing a real-space camera and a virtual camera.
FIG. 5 is a diagram for explaining a process of synchronizing a real-space camera and a virtual camera.
FIG. 6 is a block diagram of a second embodiment of the virtual scene photographing apparatus of the present invention.
FIG. 7 is a diagram for explaining a second embodiment of the present invention.
FIG. 8 is a diagram for explaining a second embodiment of the present invention.
FIG. 9 is a block diagram of a third embodiment of the virtual scene photographing apparatus of the present invention.
FIG. 10 is a diagram for explaining a third embodiment of the present invention.
FIG. 11 is a diagram for explaining a third embodiment of the present invention.
[Explanation of symbols]
10,50 Automatic camera device
11, 51 Drive control unit
12, 52 Shooting unit
15, 25 Video delay device
20 CG image generator
21, 42, 62 Virtual subject operation setting section
22, 43, 63 Virtual camera operation setting unit
23, 45, 65 Rendering part
30 Video composition device
44, 64 data converter
61 Position detector

Claims (3)

Based on the movement of the virtual subject by computer graphics or its control information, set the motion data of the virtual camera that captures the virtual subject in the virtual space,
The virtual camera motion data is converted into real space camera control data for photographing the real space in synchronization with the virtual camera motion,
A video shot by the real space camera driven based on the control data is combined with a video shot by the virtual camera based on the movement of the virtual object by the computer graphics or control information thereof. A virtual scene photographing method characterized by obtaining a video of a virtual scene. A virtual camera operation setting means for setting motion data of a virtual camera for photographing the virtual subject in a virtual space based on the movement of the virtual subject by computer graphics or control information thereof;
Real space camera operation setting means for converting the movement data of the virtual camera into control data of a real space camera for photographing the real space in synchronization with the movement of the virtual camera ;
The virtual camera based on the video captured by the real space camera driven based on the control data set by the real space camera operation setting means and the movement of the virtual subject by the computer graphics or the control information thereof A virtual scene photographing apparatus comprising: a video composition unit that synthesizes a video of a virtual scene by synthesizing with the video photographed by the computer. The virtual scene photographing device according to claim 2, wherein
The virtual camera based on the video captured by the real space camera driven based on the control data set by the real space camera operation setting means and the movement of the virtual subject by the computer graphics or the control information thereof A virtual scene photographing device comprising: delay means for delaying at least one of the images photographed by the camera.

Images