JP2020137012A - Synthetic dynamic image generation apparatus - Google Patents

Abstract

To provide synthetic dynamic image data in which moving image data of a subject as a user is synthesized on the basis of information in regard to a moving image of a synthetic destination, and provide a synthetic dynamic image generation apparatus applying high amusement to the subject.SOLUTION: A synthetic dynamic image generation apparatus comprises: photographing means 11 for photographing a subject A; storage means 15 for storing subject photographing information containing a position and a field angle of the photographing means 11 for photographing template moving image data D1 and synthetic subject data D2 so as to be corresponded to the template moving image data D1; adjusting means 12 for sequentially adjusting the position and the field angle of the photographing means 11 on the basis of the subject photographing information; and generation means 16 for generating synthetic moving image data D3 on the basis of the photographing template moving image data D1 and the synthetic subject data D2. The subject photographing information is sequentially changed in accordance with elapse of a reproduction time of the template moving image data D1.SELECTED DRAWING: Figure 2

Description

The present invention relates to a composite moving image generator that synthesizes a predetermined moving image and a subject based on the behavior of a camera.

Conventionally, techniques related to compositing moving images and still images have been used in various scenes such as studio photography with chroma key, creation of ID cards with facial photographs, and three-dimensional modeling.

For example, Patent Document 1 describes an image compositing device that can synthesize a background image in response to camera work by acquiring the position and movement of the camera when shooting a subject. ..

Further, Patent Document 2 describes a face image printing system that recognizes the position of a face and automatically adjusts the position of the camera so as to be at a predetermined position of the captured image.

Further, in Patent Document 3, a composite moving image is generated by synthesizing a subject with a moving image of a background in consideration of a distance and an angle by using camera parameters such as a camera position, a line-of-sight direction, and a focal length. An image generator and a composite moving image generation method are described.

Japanese Unexamined Patent Publication No. 2008-054155 Japanese Unexamined Patent Publication No. 2004-201331 Japanese Patent No. 3309841

Here, the techniques described in Patent Document 1 and Patent Document 2 are techniques for controlling the behavior of a camera that captures a subject according to information such as the position and movement of the subject. Further, the technique described in Patent Document 3 is a technique for controlling the size of the composite moving image or the like according to information such as the positional relationship between the subject and the composite moving image.

Therefore, the techniques described in Patent Documents 1 to 3 cannot synthesize the subject and the moving image based on the information about the moving image of the synthesis destination.

The present invention has been made in view of the above circumstances, and provides composite moving image data in which moving image data of a subject as a user is synthesized based on information on a moving image of a synthesis destination, and provides the subject with composite moving image data. An object of the present invention is to provide a composite moving image generator that provides high entertainment value.

In order to solve the above problems, the present invention
Shooting means to shoot the subject and
A storage means for storing the template moving image data and the subject shooting information including the position and angle of view of the shooting means for shooting the subject data for compositing, which is associated with the template moving image data.
An adjusting means for sequentially adjusting the position and angle of view of the shooting means based on the subject shooting information, and
A generation means for generating composite moving image data based on the template moving image data and the composite subject data is provided.
The subject shooting information is characterized in that it changes sequentially with the lapse of the reproduction time of the template moving image data.

According to the present invention, the subject and the template moving image data can be combined according to the change in the composition of the template moving image data with the passage of the reproduction time.
That is, the subject (user) can be incorporated into the world view of the template moving image data, and the subject can be given high entertainment value.

In a preferred embodiment of the present invention, the template moving image data is provided with a compositing region, and the compositing subject data is displayed in the compositing region of the composite moving image data.

With such a configuration, only a predetermined part of the subject can be incorporated into the world view of the template moving image data, so that the variation of the template moving image data is widened and the subject is given higher entertainment. Is possible.

A preferred embodiment of the present invention is provided with an audio information acquisition means for acquiring audio information based on the audio of the subject, and the generation means generates the synthetic moving image data including the audio information.

With such a configuration, the sound of the subject itself can be combined with the world view of the template moving image data, so that the subject can be given a higher degree of entertainment.

In a preferred embodiment of the present invention, a person's head is displayed in the compositing area of the template moving image data, and the subject shooting information is a moving image based on the angle, position and size of the person's head. It is characterized in that it contains internal head information.

With such a configuration, it is possible to give the subject a high level of entertainment like a print club (registered trademark).

In a preferred embodiment of the present invention, the photographing means is configured to be movable around the head of the subject by the adjusting means.

With such a configuration, when the composition of the template moving image data changes, especially when the position or angle of view of the head of a person changes, it is possible to accurately acquire the subject shooting information according to this change. It becomes.

A preferred embodiment of the present invention is characterized in that a plurality of the photographing means are provided.

With such a configuration, even if the composition of the template moving image data changes in a complicated manner, by relaying between each shooting means, the subject shooting information corresponding to this change can be accurately obtained. It becomes possible to acquire.

A preferred embodiment of the present invention is characterized in that a contact prevention means is provided between the photographing means and the subject.

With such a configuration, it is possible to prevent the subject from unexpectedly contacting the photographing means or the adjusting means.

A preferred embodiment of the present invention is provided with a display processing means for displaying at least one of the template moving image data, the subject data for compositing, and the composite moving image data, and transmitting the display processing result.

With such a configuration, the contents of the template moving image data, the subject data for compositing, and the composite moving image data can be grasped before the composite moving image data is output, so that the subject is more satisfied. It is possible to acquire high-quality composite moving image data.

According to the present invention, there is a composite moving image generator that provides synthetic moving image data in which moving image data of a subject as a user is synthesized based on information about the moving image of the synthesis destination, and gives the subject a high degree of entertainment. Can be provided.

It is the schematic of the synthetic moving image generation system using the synthetic moving image generation apparatus which concerns on embodiment of this invention. It is a schematic perspective view of the synthetic moving image generator which concerns on embodiment of this invention. It is a figure for demonstrating the usage method and operation mode of the synthetic moving image generator which concerns on embodiment of this invention. It is a figure for demonstrating the usage method and operation mode of the synthetic moving image generator which concerns on embodiment of this invention. It is a figure for demonstrating the usage method and operation mode of the synthetic moving image generator which concerns on embodiment of this invention. It is a figure for demonstrating the usage method and operation mode of the synthetic moving image generator which concerns on embodiment of this invention. It is a sequence diagram which shows the processing procedure of the synthetic moving image generator which concerns on embodiment of this invention.

Hereinafter, the synthetic moving image generator according to the embodiment of the present invention will be described with reference to FIGS. 1 to 7.
The embodiments shown below are examples of the present invention, and the present invention is not limited to the following embodiments. Further, in these figures, reference numeral 1 indicates a composite moving image generator according to the present embodiment.

FIG. 1 is a schematic view showing a composite moving image generation system S using the synthetic moving image generation device 1.

The composite moving image generation system S is a management server that manages the synthetic moving image generation device 1, the user terminal U that stores the synthetic moving image generated by the synthetic moving image generation device 1, and the synthetic moving image generating device 1. It has Y and.
Further, the composite moving image generation device 1, the user terminal U, and the management server Y are connected to each other via the network N.
The management server Y monitors, for example, the presence or absence of a defect in the composite moving image generation device 1, adds / deletes template moving image data, stores composite moving image data, and the like.

The composite moving image generation device 1 includes a photographing means 11, an adjusting means 12, an audio information acquiring means 13, a contact preventing means 14, a storage means 15, and a processing unit T.
The processing unit T has a generation means 16 and a display processing means 17.

The photographing means 11 is a camera that photographs the subject A (user, see FIG. 3).

The adjusting means 12 sequentially adjusts the position and the angle of view of the shooting means 11 for shooting the subject A based on the subject shooting information.

The voice information acquisition means 13 is a microphone that acquires voice information based on the voice of the subject A.

The contact preventing means 14 is provided between the photographing means 11 and the adjusting means 12 and the subject A, and prevents the subject A from unexpectedly contacting the photographing means 11 and the adjusting means 12.

The storage means 15 is associated with the template moving image data and the template moving image data, and stores the subject shooting information including the position and the angle of view of the shooting means 11 for shooting the subject data for composition, such as an HDD or SSD. It is a non-volatile auxiliary storage device.

The generation means 16 generates composite moving image data based on the template moving image data and the composite subject data.

The display processing means 17 performs display processing of template moving image data, composite subject data, and composite moving image data, and transmission of display processing results.

Hereinafter, specific aspects of each means will be described with reference to FIG.

As shown in FIG. 2, the composite moving image generation device 1 is roughly configured in a substantially rectangular parallelepiped box shape having an opening P1 as an entrance / exit on one side surface thereof.
Further, a curtain portion P2 hangs down from the upper portion of the opening portion P1.
In addition, in FIG. 2, the curtain portion P2, the side surface on the front side, and the ceiling are shown by dotted lines on their outer shapes and shown as a state in which the inside is transparent.

Further, the composite moving image generation device 1 includes a substantially rectangular parallelepiped base portion P3.
A display unit P4 for displaying the display processing result by the display processing means 17 is placed on the upper portion of the base portion P3, and a coin insertion unit P5 is provided on one side surface facing the opening P1.

The display unit P4 is a moving image data display unit P4a for displaying the template moving image data D1, the subject data D2 for compositing, the composite moving image data D3, etc., and the selection / reproduction of the template moving image data D1 by the subject A, and the coins. It is composed of an operation screen display unit P4b for displaying a touch panel type operation screen for performing operations such as return.
The display unit P4 does not necessarily have to be composed of two display units, and may be any number.

The photographing means 11 (camera) has a zoom-in / zoom-out function, and two units are provided in the adjusting means 12 provided on the ceiling of the composite moving image generation device 1 to move around the head A1 of the subject A. It is configured to be possible.
More specifically, each photographing means 11 is attached to each sliding pedestal 12b (see FIG. 5A) slidably provided along the rail portion 12a (see FIG. 5A) of the adjusting means 12. The lens is provided so as to always face the head A1 of the subject A.
The number of photographing means 11 is not limited to two and may be any number.

The adjusting means 12 is composed of a rail portion 12a that is curved in an inverted U shape and has lightening at a plurality of locations, a sliding pedestal 12b, and a rotating portion 12c provided at the apex of the rail portion 12a. Has been done.
The rail portion 12a hangs down from the ceiling via the rotating portion 12c, and is configured to be rotatable around an axis extending vertically downward from the apex of the rail portion 12c.
The means for fixing the rotating portion 12c to the ceiling may be any means as long as the entire adjusting means 12 can be stably held.

The voice information acquisition means 13 (microphone) is mounted on the upper part of the base portion P3 and on both sides of the operation screen display portion P4b.
The number of voice information acquisition means 13 is not limited to two, and may be any number, and may be provided by means such as hanging from the ceiling or being incorporated inside the display unit P4.

The contact prevention means 14 is semi-transparent as a whole, and is composed of a hemispherical contact prevention means main body 14a and a thin plate-shaped main body holding portion 14b extending from both sides of the contact prevention means main body 14a.
The contact prevention means main body 14a is configured such that the lower part is opened and the inside is hollow.
Each end of the thin plate-shaped main body holding portion 14b is fixed to both side surfaces of the composite moving image generator 1.
The contact prevention means 14 does not necessarily have to be semi-transparent as a whole, and only the contact prevention means main body 14a may be semi-transparent. Further, as the material of the contact preventing means 14, for example, acrylic resin is preferably used. Further, the fixing means between the main body holding portion 14b and both side surfaces may be any means as long as the entire contact preventing means 14 can be stably held.

Next, the usage method and operation mode of the composite moving image generation device 1 will be described with reference to FIGS. 3 to 6.

FIG. 3 is an enlarged perspective view of the composite moving image generation device 1 in a state where the subject A is inside.

First, as shown in FIG. 3, the subject A faces the display unit P4 in a state in which its head A1 is housed inside the contact prevention means main body 14a.
It is preferable to provide, for example, a chair whose height can be adjusted inside the composite moving image generator 1 so that the position of the head A1 can be adjusted according to the height of the subject A.

Next, the subject A inserts a coin into the coin insertion unit P5 and operates the composite moving image generation device 1.
By doing so, the display processing means 17 displays the template moving image data selection screen on the operation screen display unit P4b.
The operation of the composite moving image generator 1 does not necessarily have to be triggered by the insertion of coins. For example, an IC card reader is provided on the base portion P3, and the subject A passes through a user terminal U such as a smartphone. The composite moving image generator 1 may be configured to operate by paying.

Next, the subject A selects the desired template moving image data D1 from the plurality of template moving image data stored in the storage means 15 by using the operation screen display unit P4b.
By doing so, the display processing means 17 displays the template moving image data D1 on the moving image data display unit P4a.

In the present embodiment, for example, as shown in FIG. 4, it is assumed that the template moving image data D1 in which the person X is wakeboarding is selected.

Here, FIG. 4 shows the state of the screen transition of the template moving image data D1 with the passage of the reproduction time.
That is, the reproduction time of the template moving image data D1 elapses in the order of FIGS. 4 (a), 4 (b), and 4 (c), and the person X gradually becomes the subject A while riding the wave. The content is like approaching towards.
Further, the template moving image data D1 is provided with a composite region Z indicated by a alternate long and short dash line so as to surround the outline of the head X1 of the person X.

Next, as the reproduction time of the template moving image data D1 described above elapses, the subject A is photographed by each photographing means 11.
At this time, the adjusting means 12 adjusts the position and the angle of view of each shooting means 11 based on the subject shooting information associated with the template moving image data D1.

The subject shooting information includes moving image inner head information based on the angle, position, and size of the head X1 of the person X.
More specifically, the moving image inner head information includes image information based on a change in the angle position of the head X1 seen from the subject A in the template moving image data D1, and the size of the compositing area Z with respect to the screen size. It is composed of information such as size information based on changes in.
That is, the image information and the size information are information that changes sequentially with the passage of the reproduction time of the template moving image data D1.

For example, as shown in FIG. 4, when the size of the head X1 of the person X changes with the passage of the reproduction time, the angle of view of each photographing means 11 is adjusted by the adjusting means 12 based on the size information. Zooms in or out.
In the present embodiment, as the reproduction time elapses, the size of the compositing area Z with respect to the screen size increases (approaches), so that each photographing means 11 is zoomed in accordingly.

Further, although not shown in FIG. 4, when the angle position of the head X1 of the person X changes with the passage of the reproduction time, the head X1 of the person X seen from the subject A is based on the image information. The adjusting means 12 adjusts the position and the angle of view of each shooting means 11 so that the angle position and the angle position of the head A1 of the subject A photographed by the shooting means 11 are the same.

That is, as shown in FIG. 5A, each photographing means 11 moves along the rail portion 12a (arrow a) based on the image information, so that the angle position of the subject A in the sagittal plane direction is changed. It will be adjusted.
Further, as shown in FIG. 5B, the rotation of the rotating portion 12c (arrow b) adjusts the angle position of the subject A in the coronal plane direction based on the image information.
5 (a) and 5 (b) are side views and top views of the subject A, the adjusting means 12, and the contact preventing means 14 in the state shown in FIG. 3, respectively. Further, in FIG. 5A, the contact prevention means main body 14a is shown in a cross-sectional view.

Here, as shown in FIG. 5A, the rail portion 12a has upper and lower side surfaces curved in a substantially U shape, and a plurality of (four) rollers provided on each sliding pedestal 12b. R is configured to abut on the inner peripheral surface of the curved portion.
As a result, the sliding pedestal 12b slides smoothly along the rail portion 12a.
When the angle position of the head X1 of the person X does not change significantly as in the template moving image data D1, only one photographing means 11 may be operated. On the contrary, for example, in the case of template moving image data in which a person performing figure skating rotates on a link, the image information changes in a complicated manner, so each shooting means 11 is operated to slide or relay. As a result, the subject data D2 for compositing is acquired.

In this way, while the template moving image data D1 is being reproduced, the subject A is photographed by each photographing means 11 while the position and angle of view of each photographing means 11 are constantly adjusted by the adjusting means 12.
Then, the sequential change in the angle, position, and size of the head A1 of the subject A, which is synchronized with the sequential change of the moving image inner head information with the passage of the reproduction time of the template moving image data D1, is used as the subject data D2 for composition. It is acquired and stored in the storage means 15.
It should be noted that the subject data D2 for composition naturally includes a sequential change in the facial expression of the subject A.

Next, the generation means 16 generates the composite moving image data D3 based on the template moving image data D1 and the acquired composite subject data D2.
That is, as shown in FIG. 6, the composite moving image data D3 is generated by replacing the inside of the compositing region Z of the template moving image data D1 with the compositing subject data D2.

When the subject A emits a voice during the reproduction of the template moving image data D1, the voice information based on the voice is acquired by the voice information acquisition means 13 and stored in the storage means 15.
Then, the acquired audio information is used to generate the composite moving image data D3 as a part of the composite subject data D2, and the generation means 16 generates the composite moving image data D3 including the audio information.
That is, in this case, the composite moving image data D3 generated by the generation means 16 emits audio information at the same timing as the reproduction time when the subject A emits the sound in the template moving image data D1.

FIG. 7 is a sequence diagram showing a processing procedure of the composite moving image generation device 1. Using this, the flow until the subject A acquires the composite moving image data D3 will be described.

In step S1, the subject A starts the operation of the composite moving image generation device 1.
In the present embodiment, the processing unit T starts the operation of the composite moving image generation device 1 when the subject A inserts a coin into the coin insertion unit P5.

In step S2, the processing unit T performs display processing on the display unit P4.
In the present embodiment, the template moving image data selection screen is displayed on the operation screen display unit P4b.

In step S3, the photographing means 11 and the adjusting means 12 transmit the user recognition data to the processing unit T.
The user recognition data is information for specifying the position inside the contact prevention means main body 14a of the head A1, the orientation of the face, and the size of the head A1 based on the feature amount of the face of the subject A. ..

In step S4, the processing unit T performs the user recognition process based on the user recognition data.

In step S5, the processing unit T performs initial position standby processing on the photographing means 11 and the adjusting means 12 based on the user recognition data.
As a result, the photographing means 11 and the adjusting means 12 move to the initial position before reproducing the template moving image data according to the position and size of the head A1 of the subject A.

In step S6, the subject A selects desired template moving image data using the operation screen display unit P4b.

In step S7, the processing unit T performs a reproduction process on the display unit P4.
In the present embodiment, the selected template moving image data D1 is reproduced on the moving image data display unit P4a.

In step S8, the processing unit T performs subject shooting information transmission processing to the shooting means 11 and the adjusting means 12 at the same timing as in step S6.

In step S9, at the same timing as in step S6, the processing unit T performs the voice information acquisition start processing on the voice information acquisition means 13.

In step S10, the photographing means 11 and the adjusting means 12 transmit the subject data for synthesis to the processing unit T. It should be noted that the transmission of the composite subject data D2 is sequentially performed during the reproduction of the template moving image data D1.

In step S11, the voice information acquisition means 13 transmits voice information to the processing unit T. The audio information is transmitted sequentially during the reproduction of the template moving image data D1.

In step S12, the processing unit T generates the composite moving image data D3 based on the template moving image data D1, the composite subject data D2, and the audio information, and performs output processing (upload) to the management server Y.

In step S13, the management server Y generates a download URL unique to the composite moving image data D3.

In step S14, the management server Y generates a QR code (registered trademark) unique to the composite video data D3 based on the download URL unique to the composite video data D3.

In step S15, the management server Y transmits the QR code (registered trademark) unique to the composite moving image data D3 to the processing unit T.

In step S16, the processing unit T performs display processing of the QR code (registered trademark) peculiar to the composite moving image data D3 on the display unit P4.
As a result, the QR code (registered trademark) unique to the composite moving image data D3 is displayed on the display unit P4.

In step S17, the subject A downloads the composite moving image data D3 by his / her own user terminal U via the QR code (registered trademark) and saves it in the user terminal U.

According to the present embodiment, the subject A and the template moving image data D1 can be combined according to a change in the composition or the like of the template moving image data D1 with the lapse of the reproduction time.
That is, the subject A can be incorporated into the world view of the template moving image data D1, and the subject A can be given high entertainment value.

Further, since the subject data D2 for compositing is displayed in the compositing area Z of the composite moving image data D3, only a predetermined part of the subject A can be incorporated into the world view of the template moving image data D1. The variation of the template moving image data D1 is widened, and it is possible to give the subject A a higher level of entertainment.

Further, the audio information acquisition means 13 for acquiring the audio information based on the audio of the subject A is provided, and the generation means 16 generates the composite moving image data D3 including the audio information to generate the world view of the template moving image data D1. Since the voice of the subject A itself can be combined with the subject A, it is possible to give the subject A a higher level of entertainment.

Further, the head X1 of the person X is displayed in the compositing area Z of the template moving image data D1, and the subject shooting information includes the moving image inner head information based on the angle, position and size of the head X1 of the person X. By including the above, it is possible to give the subject A a high degree of entertainment like a print club (registered trademark).

Further, since the photographing means 11 is configured to be movable around the head A1 of the subject A by the adjusting means 12, the composition of the template moving image data D1 is changed, particularly the position of the head X1 of the person X and the like. When the angle of view changes, it is possible to accurately acquire the subject shooting information according to the change.

Further, even if the composition of the template moving image data D1 changes in a complicated manner by providing a plurality of photographing means 11, the change can be accommodated by relaying between the photographing means 11. It is possible to accurately acquire subject shooting information.

Further, since the contact preventing means 14 is provided between the photographing means 11 and the subject A, it is possible to prevent the subject A from unexpectedly contacting the photographing means 11.

Further, by providing the display processing means 17 for displaying the template moving image data D1, the subject data D2 for compositing, and the composite moving image data D3 and transmitting the display processing result, before the composite moving image data D3 is output. Since the contents of the moving image data D1 to D3 can be confirmed, the subject A can acquire the composite moving image data D3 having a higher degree of satisfaction.

The various shapes, dimensions, and the like of each component shown in the above-described embodiment are examples, and can be variously changed based on design requirements and the like.

For example, in the present embodiment, the template moving image data D1 is reproduced on the moving image data display unit P4a, but the preview of the composite moving image data D3 is displayed in real time during the reproduction of the template moving image data D1. It may be configured to be used.
Further, for example, in the operation screen display unit P4b, the subject A may be configured to be able to select whether or not to synthesize his / her own voice with the composite moving image data D3.
Further, the means for downloading the composite video data D3 is not limited to the QR code (registered trademark), and for example, an application linked with the composite video generator 1 is used, and the composite video data D3 is transmitted to the user terminal via this application. It may be configured to be stored in U.

1 Synthetic moving image generator P1 Opening P2 Curtain P3 Base part P4 Display part P4a Moving image data display part P4b Operation screen display part P5 Coin insertion part 11 Photographing means 12 Adjusting means 12a Rail part 12b Sliding pedestal R Roller 12c Rotation Unit 13 Voice information acquisition means 14 Contact prevention means 14a Contact prevention means Main body 14b Main body holding unit T Processing unit 15 Storage means 16 Generation means 17 Display processing means D1 Template moving image data D2 Composite subject data D3 Composite moving image data A Subject A1 Head X Person X1 Head Z Synthesis area U User terminal Y Management server N Network

Claims (8)

Shooting means to shoot the subject and
A storage means for storing the template moving image data and the subject shooting information including the position and angle of view of the shooting means for shooting the subject data for compositing, which is associated with the template moving image data.
An adjusting means for sequentially adjusting the position and angle of view of the shooting means based on the subject shooting information, and
A generation means for generating composite moving image data based on the template moving image data and the composite subject data is provided.
The composite moving image generation device, characterized in that the subject shooting information changes sequentially with the lapse of the reproduction time of the template moving image data. The template moving image data is provided with a compositing area.
The composite video generation device according to claim 1, wherein the composite subject data is displayed in the composite region of the composite video data. The head of a person is displayed in the compositing area of the template moving image data.
The composite moving image generation according to any one of claims 2, wherein the subject shooting information includes moving image inner head information based on the angle, position, and size of the head of the person. apparatus. A voice information acquisition means for acquiring voice information based on the voice of the subject is provided.
The synthetic moving image generation device according to any one of claims 1 to 3, wherein the generating means generates the synthetic moving image data including the audio information. The composite moving image generation device according to any one of claims 1 to 4, wherein the photographing means is configured to be movable around the head of the subject by the adjusting means. The composite moving image generation device according to any one of claims 1 to 5, further comprising a plurality of the photographing means. The composite moving image generation device according to any one of claims 1 to 6, wherein a contact prevention means is provided between the photographing means and the subject. Any of claims 1 to 7, further comprising a display processing means for displaying at least one of the template moving image data, the composite moving subject data, and the composite moving image data, and transmitting the display processing result. The composite moving image generator described in Crab.