JP2011091725A - Server apparatus, cellular phone and composite motion picture creation system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a composite motion picture creation system wherein a synthetic motion picture can be easily created by a user interface which is easy to understand, using a cellular phone. <P>SOLUTION: A server apparatus includes: an MOS storage unit for storing a plurality of MOSs, each of which is a motion picture decorating an original motion picture; a communication unit for receiving from a cellular phone the original motion picture and MOS selection information; a motion picture compositing unit for combining the selected MOS with the original motion picture; and a WEB server unit for disclosing the composite motion picture over the Internet with a predetermined URL. The cellular phone includes: a motion picture capturing unit for capturing an original motion picture; an MOS selection unit for selecting an MOS from among a plurality of candidates; and a communication unit for transmitting to the server apparatus the original motion picture and the MOS selection information and receiving the URL disclosing the composited motion picture obtained by compositing the selected MOS with the original motion picture. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a synthesized moving image creating system that creates a synthesized moving image by synthesizing another moving image with a moving image shot by a mobile phone.

  Consumer video cameras have become very popular and anyone can easily shoot video. Mobile phones with video cameras are also popular. Furthermore, with the advancement of moving image processing technology and processing devices, it is also common to synthesize moving images with moving images (for example, Patent Document 1).

JP 2007-280069 A

  As mentioned above, video cameras have become very popular, but many videos taken by so-called amateurs are “unintelligible” because the intention is not communicated to the listener, even if the video was shot with a certain intention. . Therefore, by synthesizing an additional image of the telop using a moving image processing technique, it is possible to obtain an image that is meaningful to the viewer.

  However, in the case of a moving image shot with a mobile phone as described above, it is functionally difficult to synthesize an additional image such as a telop with the mobile phone.

  This invention makes it possible to easily create a composite video with a user interface that is easy to understand by using a mobile phone by transmitting and receiving data such as a video taken by the mobile phone between the mobile phone and the server device. The purpose is to provide a system.

  The server device according to the first aspect of the present invention includes a MOS storage unit that stores a plurality of MOSs (Monogatari Stamps) that are videos that decorate an original video that is a video shot by a mobile phone, and the original video from the mobile phone. A communication unit that receives the MOS selection information for selecting one of the plurality of MOSs, a video synthesis unit that synthesizes the selected MOS with the original video, and a synthesized video with a predetermined URL And the WEB server unit that is published on the Internet, and the communication unit notifies the mobile phone of the predetermined URL by e-mail.

  According to a second aspect of the present invention, in the first aspect of the invention, the communication unit adjusts the position on the screen and / or the position on the time axis for synthesizing the selected MOS with the original video from the mobile phone. The video composition unit further receives the MOS arrangement information to be synthesized, and the video composition unit synthesizes the selected MOS with a position on the screen and / or a position on the time axis specified by the MOS arrangement information of the original video. Features.

  The invention of claim 3 further comprises a text analysis unit for analyzing the contents of text data in the invention of claim 2, wherein the communication unit is on the time axis for synthesizing a MOS with an original video as the MOS arrangement information. It is characterized in that the text data of the content indicating the position and the text data of the content indicating the position on the screen where the MOS is combined with the original moving image are received and input to the text analysis unit.

  According to a fourth aspect of the present invention, in the second or third aspect of the invention, the communication unit receives the original moving image by e-mail and receives the MOS selection information and the MOS arrangement information via the WEB. To do.

  According to a fifth aspect of the present invention, in the first to fourth aspects of the present invention, the MOS storage unit stores, as the MOS, a moving image stamp including a meaning and a moving image such as a character or a telop with a moving image effect. It is characterized by being.

  The mobile phone according to the invention of claim 6 is a MOS selection unit that accepts a selection operation for selecting, from among a plurality of candidates, a moving image shooting unit that shoots an original moving image and a MOS (Monogatari Stamp) that is a moving image that decorates the original moving image , And the original video and the MOS selection information by the MOS selection unit are transmitted to the server device, and at the same time, the URL for receiving the composite video in which the selected MOS is synthesized with the original video is received. And a section.

  The invention according to claim 7 is the invention according to claim 6, wherein the communication unit includes correction information for adjusting a position on the screen and / or a position on the time axis for synthesizing the selected MOS with the original moving image. It further transmits to the said server apparatus, It is characterized by the above-mentioned.

  According to an eighth aspect of the present invention, in the seventh aspect of the invention, the MOS arrangement information input unit includes, as the MOS arrangement information, text data having a content indicating a position on the time axis for synthesizing the MOS with the original moving picture and the original moving picture. It is characterized in that it accepts the input of text data with contents indicating the position on the screen where the MOS is synthesized.

  A ninth aspect of the present invention is a composite moving image creating system including devices including the server device according to the first aspect and the mobile phone according to the sixth aspect.

  According to the present invention, since the server device synthesizes a decorative video MOS with the original video received from the mobile phone, the mobile phone can create a composite video simply by shooting the original video and selecting the MOS. It becomes possible.

The block diagram of the synthetic | combination moving image production system which is embodiment of this invention The figure which shows an example of the contents of the synthetic animation created with this system The figure which shows the procedure of animation composition in this system The figure which shows the procedure of animation composition in this system The figure which shows the example of the MOS selection page displayed on a mobile telephone The figure which shows the example of the correction page of the appearance timing and position of MOS displayed on a mobile phone The figure which shows the procedure of the synthetic animated picture inspection in this system

  A composite moving image creating system according to an embodiment of the present invention will be described with reference to the drawings. This composite video creation system adds or emphasizes the message nature intended by the photographer by synthesizing a MOS (Monogatari Stamp) with a video (original video) of a predetermined time (for example, 15 seconds) shot with a mobile phone. This is a system that creates a moving image (synthetic moving image), saves the combined moving image on a server, and makes it visible to the public.

  Here, the MOS is a moving image stamp having a meaning, that is, a moving image in which a graphic (stamp) having a specific meaning is moved in the screen, a character or a telop with a moving image effect, or the like. The figure having a specific meaning is, for example, a heart mark having a love meaning or a bowl-like figure having an explosion meaning. The moving image effect is, for example, an effect that gradually enlarges characters or moves them within the screen. Details of the MOS will be described later.

  FIG. 1 is a system configuration diagram of a synthetic moving image creating system. The composite video creation system includes at least a server device 1 and a mobile phone 2. The server device 1 is connected to a mobile phone network 3 and the Internet 4. The mobile phone 2 is connected to the server device 1 via the mobile phone network 3. The cellular phone network 3 is also connected to the Internet 4 via a gateway 5, and the cellular phone 2 is also connected to the Internet through a route of the cellular phone network 3 -gateway 5 -Internet 4. A general personal computer (personal computer) 6 is also connected to the Internet 4.

  The server device 1 includes a control unit 10. The control unit 10 includes a CPU, and functionally includes a moving image editing / synthesizing unit 15, a WEB server unit 16, a mail server / client 17, and a text command analysis unit 18 according to a program. In addition, a moving image storage unit 11, a MOS storage unit 12, a telephone network communication unit 13, and a network control unit 14 are connected to the server device 1.

  The MOS storage unit 12 is a storage unit that stores a MOS to be combined with an original moving image. The moving image editing / synthesizing unit 15 is a functional unit that combines the MOS with the original moving image received from the mobile phone 2 to create a combined moving image. In addition, the moving image editing / synthesizing unit 15 corrects the MOS combining timing and position with respect to the original moving image based on the command sent from the mobile phone 2.

  In this embodiment, a text command that is a text-based command is used as the command. The text command uses a plain text such as “slightly later” as a command for instructing the timing for synthesizing the MOS with the original moving image. The same applies to the command for designating the synthesis position.

  The text command analysis unit 18 of the server device 1 analyzes the meaning of the text command received from the mobile phone 2 and transfers the instruction content to the moving image editing / synthesis unit 15. The text command analysis unit 18 directly analyzes the text command in natural language, and selects an optimal instruction content (deviation value) by referring to a database that stores and stores past text commands and analysis results thereof.

  The mail server client 17 creates a mail account for sending / receiving mails to / from the mobile phone 2, receives the original video from the mobile phone 2 using the mail account, and transmits the URL of the composite video. The moving image editing / synthesizing unit 15 corrects the MOS combining timing and position with respect to the original moving image in accordance with the contents of the text command analyzed by the text command analyzing unit 18. The moving image storage unit 11 is a storage unit that stores the original moving image received from the mobile phone 2, the combined moving image created by the moving image editing / composing unit 15, and the like. The WEB server unit 16 is a functional unit that creates a WEB page and publishes it on the Internet 4. Examples of the WEB page created by the WEB server unit 16 include a MOS selection page that displays a MOS selection screen, and a composite video selection page that displays a composite video selection screen stored in the composite video storage unit 11. The telephone network communication unit 13 is a functional unit that communicates with a mobile phone via the mobile phone network 3. The network control unit 14 is a functional unit that communicates with the mobile phone 2 and the personal computer 6 via the Internet 4.

  The mobile phone 2 includes a control unit 20, an operation unit 21, an audio signal processing unit 22, a display unit 23, a moving image shooting unit 24, a storage unit 25, a telephone network communication unit 26, and a wireless communication unit 27. The control unit 20 is composed of a microcomputer and functionally includes a mailer unit 28 and a browser 29 by a program. The operation unit 21 includes a group of keys such as numeric keys, function keys, and cursor keys, and an interface. The audio signal processing unit 22 includes a microphone, and includes an A / D converter that converts an audio signal input from the microphone into a digital signal. The display unit 23 includes a liquid crystal display and a display control unit. The moving image shooting unit 24 includes a camera and a compression processing unit that compresses a moving image shot by the camera into MPEG4, for example. The storage unit 25 stores the moving image and various data captured by the moving image capturing unit 24. The telephone network communication unit 26 is a functional unit that controls communication via the mobile phone network 3, and the wireless communication unit 27 is a communication unit that performs communication with a wireless base station.

  When the user selects the moving image shooting mode, the mobile phone 2 activates the moving image shooting unit 24 to shoot a moving image, and stores the shot moving image file in the storage unit 25. When the user selects the Internet browsing mode, the mobile phone 2 activates the browser 29 and connects to the Internet.

  With reference to FIG. 2, the synthesized moving image created by this synthesized moving image creating system will be described. A user of the mobile phone 2 captures a moving image (original moving image) using the function of the moving image capturing unit 24. The length of the original moving image is arbitrary, but about 15 seconds is appropriate. The upper part of FIG. 2 is a frame advance image obtained by capturing the original moving image at an appropriate interval. This figure illustrates an original moving image that is panned (tilted) from a peach image sold at a fruit store front to a price tag image.

  The user views a MOS selection page sent from the server device 1 and selects a desired MOS. The middle part of FIG. 2 is a MOS frame advance image selected by the user. In this figure, the catchphrase of “1 quality, 2 services, 3-4, 5 prices, 5 prices” is folded along the time axis, “1 quality,” “2 services,” , MOSs that are displayed while switching between “no 34” and “fifth price” are illustrated. This MOS is intended to make the message that the store wants to convey to consumers more easily understood by synthesizing it with the original video of the restaurant's food and the inside of the store. The user can further edit the position on the screen (on the original moving image) for synthesizing the MOS and the position (timing) on the time axis. Information for editing the timing and position is sent to the server device 1 as MOS arrangement information. The correction information mobile phone 2 transmits the MOS selection information, the original moving image, the MOS arrangement information, and the like to the server device 1 using the electronic mail function.

  The server device 1 reads the MOS selected by the user, and creates a synthesized moving image by synthesizing this MOS at a position on the time axis on the screen where the original moving image is designated. The lower part of FIG. 2 is a frame-by-frame image of the synthesized moving image created. In this figure, only the peach is displayed first, then the words “quality first,” “service second” are displayed in order, and the image moves from peach to price tag, Finally, “No 34” and “Large Price” are displayed on the price tag image. This fruit store product (peach) is of good quality and reasonable price. Is expressed accurately. The server device 1 stores the composite video, the original video, the MOS selection information, and the like in the video storage unit 1 and publishes the composite video on the WEB page.

  Thus, the MOS is a moving image having a message property, a story property, or a logical property in order to clarify the intention of the original moving image. For example, a meaningful figure such as a character or a heart mark is mainly used as a component of a moving image. In other words, the MOS is an additional moving image made up of a moving image stamp containing meaning, a character or a telop with a moving image effect, or the like.

  That is, when a user shoots a moving image, he / she shoots with intention, but since he is not a shooting professional, the intention cannot be expressed well in the video. Therefore, by synthesizing a MOS that reinforces the intention of the video with the original moving picture, the video is intended to be processed into a moving picture that expresses the user's intention well.

  As shown in the above example, for example, the user wants to display “delicious” by photographing the food served at the restaurant or the food sold at the store. When a moving image is photographed with an intention such as "The intention can be clarified by synthesizing a MOS that represents deliciousness in the original moving image. By using this, it becomes possible to create a commercial spot of 15 seconds using a moving image taken by an amateur user himself.

  Next, a procedure for creating a composite moving image by the mobile phone 2 and the server device 1 will be described with reference to FIGS. First, in FIGS. 3 and 4, the user captures a moving image using the mobile phone 2 and stores it in the storage unit 25 (S100). The user activates the browser unit 29 of the mobile phone 2 and accesses a site for creating a synthetic moving image provided in the server device 1 (S101). The site for creating the composite video is a member-based site, and the server device 1 performs member authentication for access from the mobile phone 2 (S102). This member authentication may be performed by inputting a mobile phone terminal ID or a personal identification number. After member authentication, the server apparatus 1 transmits the above-described MOS selection page to the mobile phone 2 that has accessed (S103). The mobile phone 2 receives this MOS selection page and displays it on the display unit 23 (S104).

  FIGS. 5A to 5C are diagrams showing examples of MOS selection pages displayed on the display unit 23 of the mobile phone 2. First, a page shown in FIG. 5A is displayed as a MOS selection page, and a MOS genre selection is accepted. The genre is a type of MOS video form. In this figure, choices of genres such as “catch copy”, “animation”, and “character / logo / other” are displayed. FIG. 6B is a diagram showing a page displayed when a catch phrase is selected on the page of FIG. On this page, the captured images and titles at the beginning of a plurality of MOSs are displayed. When the user selects one of the MOSs, an image with a large selected MOS is displayed on the display unit 23 of the mobile phone 2 together with a play button and a selection confirmation button, as shown in FIG.

  The user selects one of a plurality of MOSs on the pages of FIGS. 6A and 6B, and performs an operation of auditioning the MOS when the screen of FIG. 6C is displayed. The mobile phone 2 accepts this audition operation (S105), and transmits this selected audition information to the server device 1 (S106).

  The server device 1 receives the selected trial listening information, reads the selected MOS from the MOS storage unit 12, and performs streaming transmission to the mobile phone 2 (S107). The mobile phone 2 receives the streaming data and reproduces and displays the MOS (S108). The user listens to the streaming-reproduced MOS to determine whether it is a desired one. If it is desired, the user operates the selection confirmation button “Use this MOS” in FIG. Confirm (Yes in S109). If not desired, the page is returned to the page of FIG. 6B or FIG. 6A, and the MOS is selected again (No in S109).

  When the page is returned by the user's operation, the mobile phone 2 returns the operation to S104, displays the page of FIG. 6 (B) or (A) again, and accepts the input of the selection operation. If a selection confirmation operation (OK operation) is performed in S109, an OK message and a message requesting moving image transmission are transmitted to the server apparatus 1 (S110).

  Receiving the video transmission request, the server device 1 transmits a mail software (mailer) activation command to the mobile phone 2 (S111). The email software launch command comes with a dedicated email address for this video composition.

  The mobile phone 2 receives the mail software activation command and activates the mail software (S112). The mobile phone 2 activates the mail software, launches a mail creation screen, and writes the sent mail address in the destination column. When the user attaches and transmits an original moving image, the mobile phone 2 attaches the moving image saved in S100 to the mail on the mail creation screen (S113) and transmits the mail (S114). This mail is transmitted to a mail address created in advance.

  The server device 1 receives mail sent from the mobile phone 2. Since the e-mail address (mailbox) dedicated to video composition this time has been created, it is easy to determine which mobile phone 2 the received e-mail is from, even if there are multiple mobile phones 2 accessing in parallel Can be judged. When the server 1 receives the mail, the server 1 reads out the MOS selected in S105 from the MOS storage unit 12, and performs a composition process for synthesizing the MOS into the original moving image received by the mail (S115). The composition position and timing of the MOS with respect to the original moving image at this time are the same default position and default timing as the streaming reproduction in S108. Then, this synthesized moving image is released with a predetermined URL (S116). The URL is returned to the mobile phone 2 (S117). Accordingly, the user can view the synthesized moving image by accessing the URL with the mobile phone 2.

  When the user operates the mobile phone 2 to view the composite video, the mobile phone 2 accesses the URL sent from the server device 1 (S118). The server device 1 transmits the composite moving image by streaming (S119). The mobile phone 2 receives and displays the streaming video (S120).

  After that, the server device 1 transmits a timing / position correction page for accepting correction of the MOS combining position and timing (S121). The mobile phone 2 receives this timing / position correction page and displays it on the display unit 23 (S122). This timing / position correction page is, for example, as shown in FIG. The user sees the streaming reproduction of the composite moving image in S120, and inputs that it is OK if this is acceptable (YES in S123). When correcting the MOS combining position and timing (NO in S123), a word (text command) for instructing correction is input. The cellular phone 2 receives an input of a text command instructing the correction (S124), and transmits it to the server device 1 (S125).

  The timing / position correction page shown in FIG. 6 displays a command input window for correcting the appearance timing (synthesis timing) of the MOS and the position of the MOS. Two command input windows are provided for each timing and position. One is a pull-down menu type input window, and the other is a text input window in which text can be freely input. FIG. 6 shows a state in which the pull-down menu of the appearance timing is selected and the pull-down menu options are expanded.

  The text commands are, for example, “a little before”, “a little later”, “more before”, “more after”, etc. Such a typical command is registered in advance in a pull-down menu and selected by the user, so that it is possible to save the user's input. Further, if an option of the pull-down menu is selected, the content of the command can be determined depending on which option is selected, so that text analysis in the server device 1 is not necessary. When text is input to the text input window, the text is transmitted as it is to the server device 1, and the content is analyzed by the text command analysis unit 18 of the server device 1.

  Based on the command input to the mobile phone 2, the server device 1 corrects and recombines the MOS combining timing and position with respect to the original moving image (S127). Then, the moving image is disclosed with a predetermined URL (S116), and this URL is transmitted to the mobile phone 2 by mail (S117). This URL may be the same as the URL at which the synthesized video is first disclosed.

  When the user who has seen the synthesized video inputs OK, the mobile phone 2 transmits a message indicating OK to the server device 1 (S128). The server device 1 that has received the OK message stores the original moving image, the combined moving image, the MOS selection information, and the MOS timing / position correction information in the moving image storage unit 11 (S129). Note that a plurality of types of synthesized moving images are created for each type of terminal device. That is, since the mobile phone 2 and the personal computer 6 have different communication speeds, image display capabilities, and the like, those that match them are created with the same content. Moreover, since the display method of a moving image for mobile phones differs depending on the mobile phone carrier (mobile phone company), a composite moving image is created for each of a plurality of carriers.

  Through the above processing, the user can view the synthesized moving image completed by accessing the URL with the mobile phone 2. Also, by sending this URL to a friend / acquaintance by e-mail or pasting it on a blog, the synthesized video can be widely disclosed to friends / acquaintances.

  Here, with reference to FIG. 7, the browsing procedure of a synthetic | combination moving image is demonstrated. Note that the composite video can be viewed not only by the mobile phone 2 but also by other mobile phones and personal computers. When the terminal device (mobile phone, personal computer) accesses the URL sent from the server device 1 (S140), the server device 1 determines whether the accessed terminal device is a mobile phone or a PC (S141). ). In the case of a mobile phone, the carrier (telephone company) is also determined. Then, the composite moving image matching the type of the terminal device is transmitted by streaming (S142). The terminal device receives the streaming video and reproduces and displays it (S143).

  In the embodiment described above, both the position on the screen and the appearance timing can be corrected for the synthesis of the MOS with respect to the original moving image, but only one of them may be used.

  In addition, by adding sound such as music and sound effects to the MOS, it is possible to further enhance the saliency and the message property than the case of only the video.

  Furthermore, any text by the user may be added to the MOS. By displaying the text input by the user with MOS, the user's expression can be enriched. In this case, acceptance of text input by the user may be performed at any timing before and after the adjustment of the timing and position of the MOS after the selection of the MOS is confirmed in the procedures of FIGS.

  The personal computer 6 may be provided with a “handwritten MOS registration function”. The “handwritten MOS registration function” is a server device as a MOS that accepts handwritten input such as illustrations, symbols, and texts by a user, animates the contents of the handwritten input by moving and / or transforming along the time axis 1 is a function to be registered. The handwritten MOS registered in this way can be used in the same manner as an existing MOS.

  Also, a MOS composed of lyrics and music of the song “Cabi” is distributed, and the user is invited to participate in the contest of “Karaoke movie” (a movie that shoots a scene suitable for lyrics and music). “Sabi” can be memorized and used to promote the popular song.

  In this way, in CM, amateurs are mobile phone cameras so that brand value is imprinted on the viewer's memory by synthesizing a catch phrase or company logo with a video that expresses the value of the product. It supplements the limited expressive power of the captured video (or still image), and enables a clear and impactful message to be transmitted to the viewer.

1 Server device 2 Mobile phone 3 Mobile phone network 4 Internet 6 Personal computer

Claims (9)

A MOS storage unit that stores a plurality of MOSs (Monogatari Stamps) that are moving pictures that decorate original moving pictures that are shot with a mobile phone;
A communication unit that receives the original video from the mobile phone and receives MOS selection information for selecting any one of the plurality of MOSs;
A video synthesis unit that synthesizes the selected MOS with the original video;
A WEB server unit that publishes the synthesized video on the Internet at a predetermined URL;
With
The communication unit is a server device that notifies the mobile phone of the predetermined URL by electronic mail. The communication unit further receives, from the mobile phone, MOS arrangement information for adjusting the position on the screen and / or the position on the time axis for combining the selected MOS with the original video,
The server apparatus according to claim 1, wherein the moving image combining unit combines the selected MOS with a position on a screen and / or a position on a time axis specified by the MOS arrangement information of the original moving image. A text analysis unit for analyzing the content of the text data is further provided.
The communication unit includes, as the MOS arrangement information, text data that indicates the position on the time axis for synthesizing the MOS with the original moving image and text data that indicates the position on the screen where the MOS is combined with the original moving image. The server device according to claim 2, wherein the server device receives and inputs these into the text analysis unit.   The server device according to claim 2, wherein the communication unit receives the original moving image by electronic mail, and receives the MOS selection information and the MOS arrangement information via WEB.   5. The server device according to claim 1, wherein the MOS storage unit stores, as the MOS, a moving image stamp including a meaning and a moving image such as a character or a telop with a moving image effect. 6. . A video shooter to shoot the original video,
A MOS selection unit that accepts a selection operation of selecting a MOS (Monogatari Stamp) that is a moving image that decorates the original moving image from a plurality of candidates;
A communication unit that transmits the original moving image and MOS selection information by the MOS selection unit to the server device, and receives a URL where a synthesized moving image in which the selected MOS is combined with the original moving image is disclosed;
Mobile phone equipped with. A MOS arrangement information input unit for receiving input of MOS arrangement information for adjusting the position on the screen and / or the position on the time axis for synthesizing the selected MOS with the original video,
The mobile phone according to claim 6, wherein the communication unit further transmits the input MOS arrangement information to the server device.   The MOS arrangement information input unit includes, as the MOS arrangement information, text data that indicates the position on the time axis for synthesizing the MOS with the original moving image and contents that indicate the position on the screen where the MOS is combined with the original moving image. The mobile phone according to claim 7, which receives input of text data.   A synthesized moving image creating system including devices including the server device according to claim 1 and the mobile phone according to claim 6.

Images