JP2012137560A - Karaoke device and control method and control program for karaoke device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To easily use image data, such as uploaded photograph data, as a background dynamic image for karaoke.SOLUTION: Based on an analysis result of two or more uploaded still pictures, a karaoke playing terminal 13 selects an effect setting used for generating a dynamic image which contains at least a part of still picture from two or more still pictures. Then the terminal 13 generates the dynamic image in which an effect composing the effect setting is applied and reproduces the generated dynamic image in synchronization with the reproduction of a karaoke musical piece.

Description

  Embodiments described herein relate generally to a karaoke apparatus, a karaoke apparatus control method, and a control program.

  Conventionally, a karaoke apparatus processes karaoke music data and outputs karaoke music as accompaniment music via an acoustic system such as a speaker, and outputs a lyrics image in synchronization with the karaoke music.

In parallel with these, a karaoke apparatus is known that processes video data stored in a video CD and the like, reproduces a background video, and superimposes a lyrics image on the background video.
It has also been proposed to display image data input from the outside during the performance of karaoke music as a background image of the lyrics image.
Also known are those having a function of displaying an effect on a still image.

Japanese Patent Laid-Open No. 11-259079

  By the way, when the user brings in the image data to be displayed as the background image of the lyrics image, it is conceivable that the effect to be applied to the still image is determined by the user, for example.

  However, the task of determining the effect for each still image is complicated for the user. In particular, the karaoke apparatus is intended for the user to sing karaoke music and sing, and it is difficult to think about editing the background moving image with much effort.

  Accordingly, an object of the present invention is to provide a karaoke apparatus, a karaoke apparatus control method, and a control program for generating a moving image that can easily use captured image data such as photograph data as a background moving image for karaoke. There is to do.

  The karaoke apparatus according to the embodiment includes an analysis unit that analyzes a plurality of still images, and the effect setting selection unit is configured to analyze a moving image including at least some of the still images among the plurality of still images based on the analysis result. Select the effect settings used for generation.

  Thereby, the moving image generating means generates a moving image to which the effect constituting the effect setting is applied using the selected effect setting, and the karaoke reproducing means reproduces the karaoke music and plays the karaoke music. The generated moving image is reproduced in synchronization with the reproduction.

FIG. 1 is an explanatory diagram of a schematic configuration of a communication karaoke system including a karaoke apparatus according to an embodiment. FIG. 2 is a block diagram of the karaoke performance terminal. FIG. 3 is an explanatory diagram of a functional configuration of a main part of the karaoke apparatus. FIG. 4 is a diagram for explaining the structure of the material information. FIG. 5 is an explanatory diagram of a configuration example of analysis information. FIG. 6 is a diagram illustrating an example of an effect determined based on the smile level and the number of people. FIG. 7 is an explanatory diagram of a specific effect example corresponding to each effect collection described above. FIG. 8 is a process flowchart of the karaoke playback process. FIG. 9 is a process flowchart of the material analysis process.

Next, embodiments will be described with reference to the drawings.
FIG. 1 is an explanatory diagram of a schematic configuration of a communication karaoke system including a karaoke apparatus according to an embodiment.
The communication karaoke system 10 has a karaoke host 11 having a karaoke database (not shown) storing karaoke music data and the like, and a plurality of karaoke performance terminals 13 connected to the karaoke host 11 via a communication network 12 such as the Internet or VPN. And a plurality of user operation terminals 14 connected to each karaoke performance terminal 13 via a wireless communication network.

FIG. 2 is a block diagram of the karaoke performance terminal.
The karaoke performance terminal 13 receives a controller 101 for controlling the karaoke performance terminal 13 as a whole and an operation input of the karaoke performance terminal 13 by a user directly or indirectly via the user operation terminal 14, or a user-owned USB device or memory card. A user interface 102 that accepts data input from the computer, and a hard disk drive (HDD) 103 that stores various data and databases.

  The karaoke performance terminal 13 includes a communication interface (I / F) 104 that performs communication with the karaoke host 11 via the communication network 12, and an optical disk drive 105 that performs recording / reproduction of an optical disk such as a CD or a DVD. And a display controller 108 for performing various displays on the display 107 based on the display image data stored in the VRAM 106.

  Furthermore, the karaoke performance terminal 13 superimposes the input sound from the microphones 109A and 109B on the karaoke sound signal corresponding to the karaoke sound data input from the controller side, and outputs various images to the speaker 110. And a camera 112.

  In the above configuration, the controller 101 includes a CPU 121 that controls the entire controller 101, a ROM 122 that stores various control programs in a nonvolatile manner, and a RAM 123 that temporarily stores various data and functions as a working area. Yes.

  The user I / F 102 includes an operation panel 125 on which an operator (not shown) for performing various operations by the user, a USB controller 127 for controlling an external USB device connected via the USB connector 126, and a card connector 128. A card controller 129 that controls an external memory card connected via the remote controller, and a remote control interface (I / F) 130 that is remotely operated by wireless communication from the user operation terminal 14.

FIG. 3 is an explanatory diagram of a functional configuration of a main part of the karaoke apparatus.
Here, a functional configuration for realizing a moving image generation function among the functions of the moving image reproduction application program 202 will be described.

  This moving image generation function is used not only for the material data 51 stored from an external device (USB memory, memory card, etc.) via the user I / F 102 (the above-mentioned USB controller 127, card controller 129, etc.) but also in the HDD 103. The present invention can also be applied to the material data 51 stored in a predetermined directory and the material data 51 stored via the communication interface 104 and the communication network 12.

  Here, the material data 51 is, for example, still image data 301 </ b> A, audio data 301 </ b> B, moving image data 301 </ b> C, and the like when the material data 51 stored in a predetermined directory in the HDD 103 is taken as an example.

  The moving image reproduction application program 202 is developed on the RAM 123 of the controller 101, and includes a material input unit 21, a material analysis unit 22, and a moving image reproduction unit 23 when viewed functionally.

  When the material data 51 is input via the user I / F 102 such as the USB controller 127 or the card controller 129, the material input unit 21 stores the material data 51 in the material database 301 constituting the database 131 in the HDD 103. . Here, the material database 301 is a database for storing material data 51 used for a generated moving image.

  Specifically, the material database 301 stores still image data 301A, sound data 301B, moving image data 301C, and the like as material data 51. The material data 51 stored in the material database 301 is used as a material candidate for a moving image to be generated.

  In addition, the material input unit 21 notifies the material analysis unit 22 that the material data 51 is stored in the HDD 103.

In response to the notification from the material input unit 21, the material analysis unit 22 starts analyzing the material data 51.
In the following description, a case will be described in which photographic data is input as material data 51 that is an analysis target of analysis processing. For the purpose of analysis, it is assumed that facial expressions (especially smiles) and the number of persons included in the photographic data as the material data 51 are output as analysis results.

  The material analysis unit 22 includes a face image detection unit 221, a facial expression detection unit 222, and a number of people detection unit 223, when roughly classified. In the following description, it is assumed that the material data 51 to be analyzed is still image data 301A.

  The face image detection unit 221 executes face detection processing for detecting a face image from the still image data 301A. The face image can be detected by, for example, analyzing the feature of the still image data 301A and searching for a region having a feature similar to a face image feature sample prepared in advance. Here, the face image feature sample is feature data extracted by statistically processing the face image features of a large number of persons.

  When the face detection process is executed, the position (coordinates), size, frontality, etc. of each face image included in the still image data 301A are detected.

  Furthermore, the face image detection unit 221 classifies the plurality of face images detected from the still image data 301A into groups for each face image estimated as the same person.

  Further, the face image detection unit 221 may identify (specify) a person corresponding to the detected face image. In that case, the face image detection unit 221 determines whether or not the detected face image is the person using, for example, a face image feature sample of the person to be identified. The face image detection unit 221 assigns a face ID for each person to the detected face image based on the above result. The face image detection unit 221 outputs the detected face image information (the face image itself and the classification result) to the expression detection unit 222 and the number of people detection unit 223.

  Thereby, the facial expression detection unit 222 to which the facial image information is input detects a facial expression corresponding to the facial image detected by the facial image detection unit 221. Then, the facial expression detection unit 222 calculates a degree (likelihood) indicating the likelihood that the facial image is the detected facial expression.

  In the present embodiment, the facial expression detection unit 222 determines whether the facial expression corresponding to the detected face image is “smile”. Specifically, the facial expression detection unit 222 determines that a face image having a feature similar to the “smile” face image feature sample is “smile”, for example.

  When the facial expression detection unit 222 determines that the facial expression corresponding to the face image is “smile”, the facial expression detection unit 222 calculates the likelihood of estimating that the facial image is a smile as the smile level. In this case, when a plurality of images are detected from one still image data 301A, the facial expression detection unit 222 calculates, for example, the average smile level of the plurality of face images and the smile level of the still image data 301A. And

  The smile level is not limited to a numerical value, and may be expressed by a relative index such as “high” or “low”. When expressing a smile level as a relative index, if a plurality of face images are detected from one still image data 301A, the facial expression detection unit 222 is set to a larger number of face images, for example. Is determined based on the smile level of the still image data 301A.

  In the following description of the present embodiment, only the smile level will be described as an example for the sake of simplification. However, the facial expression detection unit 222 is not limited to a smile, but an angry face, a crying face, a surprised face, no expression, etc. The likelihood of any facial expression may be calculated.

  On the other hand, the number-of-people detection unit 223 detects the number of people included in the still image data 301A. For example, the number-of-people detection unit 223 sets the number of face images detected by the face image detection unit 221 as the number of persons included in the still image data 301A. In addition, the number of persons detection unit 223 may calculate the number of persons including a person caught in the back, for example, by detecting the whole body or part of the body of the person including the face image.

  The number of people is not limited to a numerical value, and may be expressed by a relative index such as “large” or “low”. For example, when the number of face images equal to or greater than the threshold value is detected from the still image data 301A, the number of people detection unit 223 determines the number of still image data 301A to be “large”.

  The material analysis unit 22 stores later-described material information 302 A added to the material data 51 and analysis information 302 B generated by the analysis of the material analysis unit 22 in the material information database 302 in the HDD 103.

FIG. 4 is a diagram for explaining the structure of the material information.
The material information 302A includes information indicating a material ID, file path, file size, file format, generation date / time, generation location, type, image size, playback time, and input path.

  Here, the “material ID” is identification information uniquely assigned to identify the material data 51. “File path” indicates a location where the material data 51 is stored on the HDD 103. “File size” indicates the data size of the material data 51. “File format” is the data format of the material data 51 (for example, mpeg format, wma format, etc. for moving images, jpeg format, bmp format, etc. for still images, mp3 format, wav format, etc. for audio). Indicates. “Generation date / time” indicates information (for example, November 10, 2010) indicating the date / time when the material data 51 was generated. The “generation location” indicates position information (for example, longitude / movement information by GPS positioning) that represents a location where the material data 51 is generated. “Type” indicates the type of data content of the material data 51 (for example, still image, sound, moving image, etc.). “Image size” indicates an image size (for example, 1024 × 768 pixels) when the material data 51 corresponds to the still image data 301A or the moving image data 301C. “Reproduction time” indicates the reproduction time during reproduction at normal speed when the material data 51 corresponds to the audio data 301B or the moving image data 301C. “Input path” indicates a path (for example, an external storage medium, an external storage device, a server on a network, etc.) through which the material data 51 is input to the karaoke performance terminal 13.

FIG. 5 is an explanatory diagram of a configuration example of analysis information.
As shown in FIG. 5, the analysis information 302B includes, for example, the above-described material ID, smile level, number of persons, and face image information.

The face image information indicates information based on the analysis result by the face detection process described above. Accordingly, the face image information includes, for example, information indicating the face image, size, position, and face ID. The face image information may include the smile level of each face image.
Note that as many pieces of face image information as the number of face images detected from one still image data 301A are stored in the analysis information 302B.

  In addition, the material analysis unit 22 obtains objects such as a person (whole body including a facial image, a part of a body, etc.), a landscape (sea, mountains, flowers, etc.), an animal (dog, cat, fish, etc.) May be detected (recognized), and analysis information 302B including information indicating the analysis results (detection results) may be generated.

  Further, the material analysis unit 22 may estimate the shooting time, the shooting position, and the like from the material information 302A and the still image data 301A, and generate analysis information 302B including information indicating the analysis results (estimation results). In this case, as shown in FIG. 5, analysis information 302B includes person image information including person images, sizes, positions, and person IDs, landscape information including landscape images, sizes, positions, and attributes, animal images, sizes. , Animal information including position and attribute, shooting time, and shooting position.

  Note that the material analysis unit 22 may analyze the audio data 301B and generate analysis information 302B including information about the person and the number of persons corresponding to the detected voice, the atmosphere and genre of the detected music, and the like. .

  Further, the material analysis unit 22 analyzes each image frame included in the moving image data 301C in the same manner as the still image data 301A, and generates analysis information 302B including the above-described smile level, number of persons, face image information, and the like. Also good.

  The material analysis unit 22 notifies the moving image reproduction unit 23 that the material information 302A and the analysis information 302B corresponding to the input material data 51 are stored in the material information database 302.

In response to the notification from the material analysis unit 22, the video playback unit 23 generates a composite video (moving image) using the material data 51, and starts a process of playing back (displaying) the generated composite video. At that time, the moving image reproducing unit 23 refers to the material information database 302 and extracts the material data 51 satisfying a predetermined condition from the material database 301 to generate a synthesized moving image.
In this case, the video playback unit 23 includes an effect extraction unit 231, a composite video generation unit 232, and a composite video output unit 233.

  The effect extraction unit 231 extracts effect data 303A suitable for the captured material data 51 from the effect database 303. Here, the effect data 303A includes transitions that connect scenes in addition to normal video effects such as zoom, rotation, noise addition, mosaication, contour extraction, and embossing.

  Specifically, the effect extraction unit 231 first extracts the smile level and the number of people included in the analysis information 302B corresponding to the extracted material data 51 from the material information database 302.

  Then, the effect extraction unit 231 selects effect data 303A suitable for the extracted material data 51 based on the extracted smile level and the number of people. For example, the effect extraction unit 231 determines the smile level index and the number of people in the entire still image data 301A from the smile level and the number of people corresponding to each of the extracted still image data 301A (material data 51). Is calculated.

  That is, for example, the effect extraction unit 231 determines the number of still image data 301A including a face image having a smile degree equal to or greater than the first threshold among the plurality of extracted still image data 301A. It is determined as an index of smile level in the entire 301A.

  In addition, for example, the effect extraction unit 231 may determine the average smile level corresponding to each of the extracted still image data 301A as an index of the smile level of the entire still image data 301A.

  Further, for example, the effect extraction unit 231 calculates the number of the still image data 301A whose number of persons is equal to or greater than the second threshold value among the plurality of extracted still image data 301A in the plurality of still image data 301A as a whole. To be an indicator of the number of people. Further, for example, the effect extraction unit 231 may determine an average of the number of people corresponding to each of the plurality of extracted still image data 301A as an index of the number of people of the plurality of still image data 301A as a whole.

  As described above, the smile level and the number of people may be represented by relative indices. Therefore, for example, when a smile level of “high” or “low” is set for each of a plurality of extracted still image data 301A, the effect extraction unit 231 is set to more still image data 301A. The index (for example, “high”) that is present is determined as the smile level of the entire still image data 301A. Further, for example, when a smile level of “high” is set in still image data 301A having a predetermined ratio (first threshold value) or more among a plurality of extracted still image data 301A, the effect extraction unit 231 Determines the smile level of the entire plurality of still image data 301A to be “high”.

  Similarly, for example, when the number of “large” or “small” is set in each of the extracted still image data 301A, the effect extraction unit 231 is set to more still image data 301A. The index of the person (for example, “less”) is determined as the total number of the still image data 301A. In addition, for example, when the number of “large” is set in the still image data 301A that is equal to or higher than a predetermined ratio (second threshold) among the plurality of extracted still image data 301A, the effect extraction unit 231 Then, the number of the entire still image data 301A is determined to be “large”.

  The effect extraction unit 231 determines the effect data 303A suitable for the plurality of still image data 301A based on the smile level and the number of people corresponding to the entire extracted plurality of still image data 301A determined as described above. decide.

FIG. 6 is a diagram illustrating an example of an effect determined based on the smile level and the number of people.
The effect extraction unit 231 has a large number of people and a high smile level in the case of a material with a large number of people and a high smile level according to the smile level and the number of people corresponding to the entire extracted plurality of still image data 301A. An effect collection 51A that is considered suitable for a high material is selected.

  The effect extraction unit 231 selects an effect collection 51B that is considered suitable for a material with a large number of people and a low smile level when the number of people is a material with a low smile level.

  The effect extraction unit 231 selects an effect collection 51C that is suitable for a material with a small number of people and a low smile level when the number of people is low and the smile level is low.

  In addition, when the material is a material with a small number of people and a high smile level, the effect extraction unit 231 selects an effect collection 51D that is considered suitable for a material with a small number of people and a high smile level.

FIG. 7 is an explanatory diagram of a specific effect example corresponding to each effect collection described above.
As the effect collection 52A suitable for a material having a large number of people and a high smile level, a group of effects having an effect (decoration) reminiscent of a happy impression or a cheerful impression is used. Therefore, an effect that can excite the place is given.

  In addition, as the effect collection 52B suitable for a material having a large number of people and a low smile level, a group of effects having an effect reminiscent of a ceremony is used. Therefore, for example, an effect that creates a solemn atmosphere is given.

In addition, as the effect collection 52C suitable for a material with a small number of people and a low smile level, a group of effects having an effect of recalling a cool impression or a near-future impression is used.
In addition, a group of effects having an effect of recalling an impression such as fantasy or magic is used for the effect collection 52D suitable for a material having a small number of people and a high smile level.

  These effect collections 52A to 52D are designed to change the impression recognized by the user by changing the color, shape, movement (motion), object, and the like used for the effect.

  Therefore, for example, the effect collection 52 </ b> A that has an effect of recalling a happy impression or an energetic impression includes effects using bright colors or vivid colors. Further, for example, the effect collection 52 </ b> C having an effect of recalling a cool impression or a near-future impression includes an effect using a geometric shape.

  Note that the effect extraction unit 231 does not limit the effects suitable for the plurality of extracted still image data 301A to the effect collections classified into the four types as shown in FIGS. You can also choose from. In that case, a predetermined type of effect collection corresponding to the number of people and the smile level (value range) is specified in advance, and the effect extraction unit 231 extracts a plurality of still images extracted from the specified predetermined type of effect collection. An effect collection suitable for the image data 301A is selected.

  Further, the effect extraction unit 231 may select an effect collection suitable for the plurality of extracted still image data 301A using an index other than the number of people and the smile level.

  Next, the effect extraction unit 231 extracts the effect data 303A corresponding to the selected effect collection from the effect database 303, and outputs the extracted effect data 303A to the synthesized moving image generation unit 232.

Thereby, the synthetic moving image generating unit 232 generates a synthetic moving image that becomes a karaoke background moving image including the captured material data 51.
At this time, the effect data 303A extracted by the effect extraction unit 231 is applied to the material data 51 included in the synthesized moving image.

Further, for example, the effect data 303A extracted by the effect extraction unit 231 is applied to the face image (object) of a person appearing in the still image data 301A (material data 51) included in the synthesized moving image.
For example, the combined moving image generation unit 232 generates a combined moving image including still image data 301A that is displayed at a timing defined by the effect data 303A.

Further, the synthesized moving image may include audio data 301B output at a predetermined timing.
Then, the composite video generation unit 232 outputs the generated composite video to the composite video output unit 233.

  Note that the effect extraction unit 231 may apply effect data 303A based on a facial expression (for example, smile level) and the number of people to the captured material data 51. In that case, the synthesized moving image generating unit 232 generates a moving image (synthetic moving image) including a plurality of still images to which the effect is applied by the effect extracting unit 231 as a karaoke background moving image.

The composite video output unit 233 outputs the composite video generated by the composite video generation unit 232.
The synthesized moving image output unit 233 reproduces the synthesized movie and displays it on the screen (display 107).

  The synthetic video output unit 233 may encode the synthetic video and store the encoded synthetic video file in a predetermined storage device (for example, the HDD 103).

  With the above configuration, the moving image playback application program 202 determines an effect (effect group) 303A suitable for the material data 51 used for the synthesized moving image as the karaoke background moving image.

  Specifically, the effect extraction unit 231, for example, based on the smile level and the number of people of each of the plurality of still image data 301 </ b> A used for the composite video, the smile level and the number of people of the plurality of still image data 301 </ b> A as a whole. Determine the indicators. The effect extraction unit 231 selects the effect data 303A suitable for the plurality of still image data 301A used for the synthesized moving image as the karaoke background moving image based on the determined index of smile level and number of people.

  Therefore, without the user performing an operation of selecting the effect data 303A used for the composite video, the composite video generation unit 232 includes a karaoke background moving image (including a plurality of still image data 301A to which the appropriate effect data 303A is applied ( Composite moving image) can be generated.

Next, generation and playback processing of a karaoke background moving image according to the embodiment will be described.
FIG. 8 is a process flowchart of the karaoke playback process.
First, the controller 101 of the karaoke performance terminal 13 takes in the material data 51 used for the synthesized moving image as the karaoke background moving image (step S11).

  The material data 51 can be captured from an external USB device via the USB connector 126, from an external memory card via the card connector 128, photographed and captured by the camera 112, and stored in the HDD 103. A method of capturing shared material data, a method of downloading shared material data from the karaoke host 11 via the communication network 12, and the like are conceivable.

  As a method of capturing the material data 51 by a general user, a method of capturing from the USB connector 126, the card connector 128, or the camera 112 is employed.

  Specifically, when an external storage device such as a USB memory, a USB-connected hard disk, or a USB-connected SSD (Solid State Drive) is connected to the USB connector 126, a still image such as photo data is connected via the USB controller 127. Data is taken in as material data 51.

  When an external memory card is connected to the card connector 128, still image data such as photo data is taken in as material data 51 via the card controller 129 and the card controller 129.

  In addition, when the camera 112 has photographed by the user's operation, the photographed photograph data is captured as the material data 51.

  Next, the controller 101 executes the moving image reproduction application program 202 to perform material analysis processing (step S12).

FIG. 9 is a process flowchart of the material analysis process.
In the following, it is assumed that the material analysis target material data 51 is still image data 301A such as photographic data.

First, the material input unit 21 determines whether still image data 301A has been input via the interface unit or the like (step S21).
If the still image data 301A is not input in the determination in step S21 (step S21; No), the standby state is entered.

  If the still image data 301A is input in the determination of step S21 (step S21; Yes), the material input unit 21 stores the input still image data 301A in the material database 301 (step S22). Then, the material input unit 21 notifies the material analysis unit 22 (face image detection unit 221) that the still image data 301A has been input.

Next, the face image detection unit 221 detects a face image from the input still image data 301A (step S23).
That is, the face image detection unit 221 detects the position (coordinates), size, frontality, and the like of each face image included in the still image data 301A. In this case, the face image detection unit 221 may recognize (identify) a person corresponding to the detected face image.
Then, the face image detection unit 221 outputs information indicating the detected face image to the expression detection unit 222 and the number of people detection unit 223.

As a result, the facial expression detection unit 222 determines the smile level of the face image detected by the face image detection unit 221 (step S24).
Here, the smile level is an index indicating the degree of likelihood (likelihood) that the detected face image is a smile. When a plurality of face images are detected from one still image data 301A, the smile level of the still image data 301A is determined based on the smile levels of the face images.

In addition, the number-of-people detection unit 223 determines the number of persons included in the still image data 301A based on the number of face images detected by the face image detection unit 221 (step S25).
As a result, the material analysis unit 22 stores the analysis information 302B including the smile level, the number of people, face image information, and the like corresponding to the still image data 301A in the material information database 302 (step S26).

  Through the above process, the smile level and the number of faces included in the input still image data 301 </ b> A are determined, and analysis information 302 </ b> B including the smile level and the number of persons is stored in the material information database 302.

  Next, the controller 101 functions as the effect extraction unit 231 based on the moving image reproduction application program 202, and selects a series of effects used for moving image generation based on the analysis result (step S13). That is, the controller 101 functions as the effect extraction unit 231 and selects an effect collection suitable for the captured material data 51 based on the analysis information 302B corresponding to the captured material data 51. Then, the controller 101 extracts the effect data 303A corresponding to the selected effect collection from the effect database 303.

Subsequently, the controller 101 functions as the synthetic moving image generating unit 232, and generates a synthetic moving image using the extracted material data 51 and the effect data 303A (step S14). Here, the generated composite video includes the material data 51 to which the effect data 303A has been applied. When the controller 101 functions as the effect extraction unit 231, the controller 101 may apply the selected effect data 303 </ b> A to the captured material data 51.
As a result, when the controller 101 functions as the composite video generation unit 232, the controller 101 generates a composite video including the material data 51 to which the effect data 303A has been applied.

  Subsequently, the controller 101 functions as the synthetic moving image output unit 233, and displays the synthetic moving image on the display 107 together with the lyrics corresponding to the karaoke music through the display controller 108 as a background moving image of the karaoke music.

  In parallel with this, the controller 101 controls the sound controller 111 to mix karaoke music with the user's input voice input from the microphones 109A and 109B and to output the sound from the speaker 110 (step S15).

  In this case, if the setting for recording is made by the user, or if the setting for recording is made as the basic setting, the synthesized moving image and the input voice of the user are added to the karaoke music as the background moving image. The sound obtained by mixing the music is recorded on the HDD 103 or recorded on a writable CD or a writable DVD set in the optical disc drive 105 in advance.

  Further, the controller 101 determines whether or not trick playback is performed such as changing the tempo of the song or changing the playback speed of the composite video, such as fast-forwarding (step S16). If it has been interrupted (step S16; Yes), the playback end timing of the karaoke song is calculated, and the playback of the composite video as the background moving image of the karaoke song is also ended in accordance with the playback end timing. The reproduction moving image is regenerated (step S17), and the process returns to step S15.

  In addition, in the determination in step S16, if trick playback is not performed (step S16; No), the lyrics corresponding to the karaoke music are used as the background moving image of the karaoke music in step S15 until the playback of the karaoke music is finished. At the same time, it is displayed on the display 107, and the process of mixing the karaoke music with the user's input voice and outputting the sound from the speaker 110 is continued.

  Through the above processing, a synthesized moving image using the captured material data 51 and effect data 303A suitable for the material data 51 can be generated as a background moving image of karaoke music, and karaoke playback can be performed.

  As described above, according to this embodiment, the user does not perform any work for selecting the effect data 303A suitable for the captured material data 51, and the material data to which the appropriate effect data 303A is applied. A synthesized moving image including 51 is generated as a background moving image of karaoke music. The karaoke performance terminal 13 can be used in a state in which the karaoke music is reproduced while the generated background moving image of the karaoke music is displayed on the display 107.

  That is, although the user has no knowledge about the effect data 303A, the moving image playback application program 202 can easily use the synthesized moving image including the material data 51 on which the appropriate effect data 303A is applied as the background image of karaoke. Can be generated.

  Note that all the procedures of the synthetic moving image generation process of the present embodiment can be executed by software. For this reason, the same effect as that of the present embodiment can be easily realized simply by installing and executing this program on a normal computer through a computer-readable storage medium storing a program for executing the procedure of the synthetic moving image generation process. be able to.

  Further, the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment.

  In the above description, the effect, transition, and scene switching are mainly selected based on the smile level and the number of people.

  However, it is also possible to detect the tone, beat, etc. of the karaoke music and select an effect, transition, and scene switching suitable for the tone, beat, etc. It is also possible to detect the interval between karaoke songs (No. 1 and No. 2), interlude period, etc., and select a still image to be displayed, an effect to be added, a transition, a scene change, etc. is there.

  Further, it is possible to extract the display switching timing of lyrics based on the control data of the display (switching) timing in the lyrics display, and to select a still image, an effect, a transition, a scene switching, or the like.

  Furthermore, it is possible to analyze the lyrics included in the lyrics data and select a still image to be displayed along with the contents of the lyrics, an effect to be added, a transition, a scene change, and the like. Specifically, when the lyrics contain many sad words, a still image that matches a dark and majestic image is displayed, an effect is added, and transitions, scene switching, and the like are performed. Also, if the lyrics contain many words that express fun and brightness, a still image that matches the bright, pop-like image is displayed, effects are added, transitions, scenes are displayed. Switch between them.

  In addition, the camera 112 takes a picture of the participating members at the start of karaoke, for example, and it is considered that those people are more preferred according to the age group, gender, etc. of the person included in the photo image corresponding to the photo data. It may be configured to display a still image, add an effect, and perform transitions, scene switching, and the like.

  In the above description, when the material data 51 is input via the user I / F 102, it has been described that the moving image is generated using all material data (all still image data, etc.). It is also possible to configure to generate a moving image using a part of the karaoke music according to the tone, performance time, and the like. For example, when the number of material data 51 is large with respect to the performance time of karaoke music, or when the number of set scenes is less than the number of material data 51, the material data is appropriately thinned out. In addition, if the performance time is shortened during trick play, the number of material data 51 used for regenerating a moving image may be thinned out as appropriate.

  Moreover, you may delete some components from all the components shown by embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

10 Communication Karaoke System 11 Karaoke Host 12 Communication Network 13 Karaoke Performance Terminal (Karaoke Device)
21 Material Input Unit 22 Material Analysis Unit (Analysis Means)
23 video playback section 51 material data 101 controller (analysis means, effect setting selection means, moving image generation means, karaoke playback means)
107 Display (karaoke playback means)
108 Display controller (karaoke playback means)
110 Speaker (karaoke playback means)
111 Sound controller (karaoke playback means)
131 Database (analysis means)
202 moving image reproduction application program 221 face image detection unit (analysis means)
222 Facial expression detection unit (analysis means)
223 Number detection unit (analysis means)
231 Effect extraction unit (effect setting selection means)
232 Synthetic video generation unit (moving image generation means)
233 Synthetic video output unit (karaoke playback means)
302 Material information database (analysis means)
302A Material information (analysis means)
302B Analysis information (analysis means)
303 Effect database (effect setting selection means)

Claims (8)

An analysis means for analyzing a plurality of still images;
An effect setting selection means for selecting an effect setting used for generating a moving image including at least a part of the still images among the plurality of still images based on the result of the analysis;
Using the selected effect setting, moving image generating means for generating the moving image to which an effect constituting the effect setting is applied;
Karaoke playback means for playing back the generated moving image in synchronization with the playback of the karaoke music while playing back the karaoke music;
A karaoke apparatus comprising:   2. The karaoke apparatus according to claim 1, wherein the karaoke playback means superimposes a lyrics image corresponding to the karaoke song on the moving image being played in synchronization with the playback of the karaoke song.   3. The karaoke apparatus according to claim 1, further comprising a still image capturing unit that captures the plurality of still images via a camera, a recording medium, or a communication network and outputs the captured still images to the analysis unit.   When the playback tempo or playback speed of the karaoke music is changed, the moving image generation means associates the playback tempo or the playback speed with the number of the still images used for generating the moving image or each still image. The karaoke apparatus according to any one of claims 1 to 3, wherein the time for displaying is dynamically changed.   5. The recording apparatus according to claim 1, further comprising a recording unit that superimposes a user's input voice on the karaoke piece reproduced by the karaoke reproducing unit and the moving image and records the same on a recording medium. Karaoke equipment. The analysis means analyzes karaoke music,
The effect setting selection means selects an effect setting used for generating a moving image based on the result of analysis such as the tone of the karaoke music obtained, beats, etc.
A karaoke apparatus according to any one of claims 1 to 5, wherein A method for controlling a karaoke device executed in a karaoke device,
An analysis process for analyzing multiple still images;
Based on the result of the analysis, an effect setting selection process for selecting an effect setting used for generating a moving image including at least some of the still images among the plurality of still images;
Using the selected effect setting, a moving image generation process for generating the moving image with the effect constituting the effect setting;
A karaoke playback process of playing back the generated moving image in synchronization with the playback of the karaoke music while playing back the karaoke music;
A method for controlling a karaoke apparatus, comprising: A control program for controlling a karaoke device by a computer,
The computer,
An analysis means for analyzing a plurality of still images;
An effect setting selection means for selecting an effect setting used for generating a moving image including at least a part of the still images among the plurality of still images based on the result of the analysis;
Using the selected effect setting, moving image generation means for generating the moving image to which an effect constituting the effect setting is applied;
Karaoke playback means for playing back the generated moving image in synchronization with the playback of the karaoke music while playing back the karaoke music,
A control program characterized by functioning as

Images