JP2005318295A - Image generation system and method, image generation program, and information recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image generation system for enabling a viewer himself or herself to prepare an image which is rich in variety without anxiety in preparing continuous images to be reproduced with music. <P>SOLUTION: This image generation system S for generating an image to be output in parallel with the music includes a database 1 for storing a plurality of templates preliminarily set in accordance with various types of pieces of music being templates as the models of the contents of images to be output in parallel with the plurality of types of pieces of music having mutually different attributes; and a template control part 12 for extracting one template corresponding to an image to be actually output in parallel with the music to be actually output from the plurality of stored templates, and for generating image information corresponding to the image to be actually output by using the extracted extraction template. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present application belongs to the technical field of an image generation system, an image generation method, an image generation program, and an information recording medium, and more specifically, an image generation system and an image for generating an image that is reproduced and output simultaneously with music The present invention belongs to a technical field of a generation method, an image generation program used for generating the image, and an information recording medium on which the image generation program is recorded.

With the spread of digital still cameras in recent years, it is also possible to create one piece of content independently by combining an image photographed by the digital still camera and music. As a prior art regarding this point, there is one disclosed in Patent Document 1 below.
JP-A-2001-339682 (FIGS. 2, 7 and 14)

  In the conventional technique disclosed in Patent Document 1, the playback time of music is acquired, the playback time of the corresponding image is determined, and the switching timing for switching the image played back (displayed) with the music is set to a certain level. Or a time arbitrarily set by the viewer, or automatic setting based on detection of a musical measure, and further automatic setting based on detection of a characteristic sound of music. Further, a plurality of switching patterns of the images are set in advance from several types of patterns depending on the music genre arbitrarily set by the viewer.

  However, according to the above-described conventional technology, variations in the variation of continuous images created by the viewer are limited, and in particular, when creating a plurality of images corresponding to the same music, unless the viewer edits arbitrarily. As a result, there is a problem that the listener is often bored even when the created image is reproduced together with music.

  Therefore, the present application was made in view of the above problems, and an example of the purpose is to create images that are rich in change without getting bored when viewers themselves create continuous images that are played along with music. An image generation system and an image generation method that can be used, an image generation program used for generating the image, and an information recording medium on which the image generation program is recorded.

  In order to solve the above-described problem, the invention according to claim 1 is an image generation system for generating an image output in parallel with music, and in parallel with a plurality of types of music having different attributes. Storage templates such as a database for storing a plurality of preset image templates corresponding to each type of music, each of which is a template for the contents of the image to be output, and the storage Extraction of a template control unit or the like that extracts one image template corresponding to the image that is actually output in parallel with the music that is actually output from among a plurality of image templates that are actually output And a template for generating image information corresponding to the image to be actually output using the extracted extraction template And a generation means such as control unit.

  In order to solve the above problem, an invention according to claim 14 is an image template that is a template of the contents of the image to be output in parallel with the plurality of types of music having different attributes. An image generation method executed in an image generation system that includes a storage unit that stores a plurality of preset image templates corresponding to each type of music, and generates the image using the image template The one image template corresponding to the image to be actually output in parallel with the actually output music is extracted as an extraction template from the plurality of stored image templates. Using the extraction step and the extracted extraction template, a generator that generates image information corresponding to the image to be actually output And, equipped with a.

  In order to solve the above problems, the invention according to claim 15 is directed to a computer included in an image generation system that generates an image output in parallel with music, and a plurality of types of the music having different attributes. Storage means for storing a plurality of preset image templates corresponding to each type of music, each of which is a template of the content of the image to be output in parallel Extraction means for extracting, as an extraction template, one image template corresponding to the image that is to be actually output from the plurality of image templates that are actually output in parallel with the music; and The extracted extraction template is used to function as generation means for generating image information corresponding to the image to be actually output.

  In order to solve the above-mentioned problem, in the invention described in claim 16, the image generation program described in claim 15 is recorded so as to be readable by the computer.

  Next, the best mode for carrying out the present application will be described with reference to FIGS. Each embodiment described below accumulates a plurality of pieces of song information relating to a song as music, reproduces the song in response to a request from a viewer, and displays an image suitable for the content of the song. This is an embodiment in the case where the present application is applied to an image generation system that automatically creates an image based on an image that has been shot and recorded or stored.

(I) First Embodiment First, a first embodiment according to the present application will be described with reference to FIGS. 1 is a block diagram illustrating a schematic configuration of the image generation system according to the embodiment, FIG. 2 is a diagram illustrating a source of a template to be described later, and FIG. 3 is a song information and material image according to the embodiment. FIG. 4 is a diagram illustrating a search coordinate space used when searching for music to be reproduced in the image generation system, and FIG. 5 is a diagram illustrating the additional information added to the image generation system. FIG. 6 is a diagram illustrating keywords used when searching for songs to be played back, FIG. 6 is a diagram illustrating changes in the environment that can be detected by a sensor described later, and FIG. 7 is a diagram illustrating songs according to the first embodiment. It is a flowchart which shows the reproduction | regeneration processing of an image.

  As shown in FIG. 1, the image generation system S according to the first embodiment is a storage unit composed of a recording medium such as a non-volatile memory or a hard disk, an image storage unit, a character information storage unit, and a sound effect information storage unit. A database 1, a search processing unit 7, an operation unit 8 as input means, an image input unit 9, a receiving unit 10 that acquires information from the Internet 16 (or outside the broadcast etc.), a music analysis unit 11, Template control as extraction means, generation means, analysis means, material image information extraction means, keyword information extraction means, additional information extraction means, keyword content information generation means, music information extraction means, image information extraction means, and sound effect information extraction means Connected to a display unit 14, a template input / output unit 13, a display unit 14 for displaying an operation state, such as a display, and a speaker 17. The reproduction control unit 15, a sensor 18 as an environmental change detection unit including various detection devices such as a temperature sensor and a clock, and a bus 19 that connects the above constituent members to each other so that information can be exchanged between them. Yes. In addition, by providing the receiving unit 10, the display unit 14, the speaker 17, and the sensor 18 in an external device (not shown), the receiving unit 10 and the display unit 14 are provided from the image generation system S according to the first embodiment. The speaker 17 and the sensor 18 may be separated.

  The database 1 also stores a template recording unit 2 that stores and records a plurality of templates, and still image information corresponding to a still image that is captured by a digital still camera and displayed on the display unit 14. Still image recording unit 3 for recording, moving image recording unit 4 for recording moving image information corresponding to the moving image displayed on display unit 14, and characters for recording character information for displaying characters on display unit 14 An information recording unit 5 and a sound effect information recording unit 6 that records sound effect information corresponding to the sound effect reproduced together with the tune and the image are configured.

  Next, the overall operation will be described.

  First, each recording unit constituting the database 1 has a hierarchical structure in which the template, still image information, moving image information, character information, and sound effect information are crowned with folder names or directory names indicating respective attributes. Form and record each.

  At this time, the template is a template for generating an image (moving image) to be displayed on the display unit 14 in parallel with the music to be reproduced / output. A plurality of images whose switching timing, display order, etc. are preset in the image so as to correspond to the attributes of each song are stored in the template recording unit 2. Then, images to be reproduced are continuously reproduced in accordance with information (such as switching timing) described as the template.

  In addition to the default template originally recorded in the template recording unit 2, the template can be acquired from the outside via the template input / output unit 13 shown in FIG. 1, as illustrated in FIG. Specific examples of the template acquired from the outside include a single addition template, a distribution single addition template, a template with music, and the like as exemplified in FIG.

  On the other hand, the still image recorded in the still image information recording unit 3 is not only the one captured by the digital still camera, but also the still image input from the Internet 16 or the like via the image input unit 9. include. At this time, additional information as exemplified in FIG. 3 is added to each still image when the still image is recorded in the still image recording unit 3.

  Furthermore, the moving image recorded in the moving image recording unit 4 is a moving image input mainly via the image input unit 9 from the Internet 16 or the like. At this time, additional information as illustrated in FIG. 3 is added to the moving image itself, similar to the still image recorded in the still image recording unit 3.

  Furthermore, the character information recorded in the character information recording unit 5 includes, as character information separately, phrases that the viewer of the image generation system S originally input in the past, subtitles acquired together with other moving images, and the like. It is what is recorded.

  The sound effect information recorded in the sound effect information recording unit 6 is recorded separately as sound effects such as environmental sounds that the viewer originally recorded in the past, sounds acquired together with other moving images, and the like. It is what.

  Next, the search processing unit 7 performs keyword search and song information search in an image generation process, which will be described later, on all information stored in the database 1. Here, the search coordinate space used for the search will be specifically described with reference to FIG.

  As shown in FIG. 4, all the information stored in the database 1 of the embodiment is in one-to-one correspondence with coordinates in a coordinate space in which the tempo of the music is the horizontal axis and the chord configuration of the music is the vertical axis. It has been. That is, in FIG. 2, the average value of the tempo (speed (more specifically, bit per minute value)) of each song recorded in the database 1 is set to the X axis, and further, A coordinate space with the major code and minor code occupying the song as a Y axis is set in advance, and templates, still images, moving images, songs, sound effects, and character information (hereinafter referred to as appropriate) recorded in the database 1 are set. Each position of each of the templates and the like is uniquely determined as to which position in the coordinate space as an attribute. Then, the coordinate values (X, Y) in the coordinate space are recorded in the database 1 in a one-to-one correspondence for each template or the like. Then, the search processing unit 7 searches the database 1 for a template or the like desired at that time using the coordinate value as a clue.

  Next, keywords used for search processing in the search processing unit 7 will be described specifically with reference to FIG.

  In the search processing for the database 1 in the search processing unit 7, first, keywords as shown in FIG. 5A are set in advance for each rough classification item as so-called free keywords. Then, in the actual search process, the search processing unit 7 may display these keywords as candidates to be selected by the viewer, or extract a keyword that is the same as or similar to the keyword freely input by the viewer. And used for the search process. Then, these keywords are corresponded to the keywords while comparing the names of folders or directories constituting the database 1.

  On the other hand, in addition to the keyword illustrated in FIG. 5A, in the image generation system S of the embodiment, the keyword (see FIG. 5B) previously associated with the coordinate space shown in FIG. (Also referred to as a coordinate-corresponding keyword) is also used for the search process. At this time, the coordinate-corresponding keywords are associated one-to-one with respect to each of a certain range in the coordinate space shown in FIG. In the example shown in FIG. 4, the range of the coordinate correspondence keyword “NORNORI” is set at the upper right in the coordinate space, and one song and one template are associated with each other.

  Next, the sensor 18 includes a thermometer, a speedometer in the case where the image generation system S is for in-vehicle use, and the like as described above, and has a function as a trigger described later. At this time, examples of information acquired from the outside via the sensor 18 include information shown in FIG. The external environment information shown in FIG. 6 is external environment information assuming that the image generation system according to the embodiment is mainly used for in-vehicle use. The so-called GPS (Global Positioning System) system, speedometer (not shown) , Room temperature meter, tachometer, illuminance sensor and the like.

  In the following description, an in-vehicle sensor 18 as shown in FIG. 6 will be described. In addition to this, as a sensor according to the present application, for example, when the image generation system according to the present application is used for a store, the temperature There are humidity sensors, customer number sensors, lighting or timers, and when the image generation system according to the present application is used for home use, there are temperature / humidity sensors, biological sensors, lighting or timers, and the like.

  Further, in the operation unit 8, an operation necessary for image generation processing described later is performed by the viewer, and an operation signal corresponding to the operation is output to each constituent member to be operated through the bus 19.

  Further, the image input unit 9 acquires a still image, a moving image, or a song from the outside as necessary, and outputs it to the database 1 or the like via the bus 19.

  Next, the song analysis unit 11 analyzes the speed and chord progression of each song and outputs the result to the search processing unit 7 and the like via the bus 19.

  Further, the reproduction control unit 15 reproduces an image and a song based on a template generated in an image generation process to be described later, and outputs song information and sound effect information corresponding to the reproduced song and sound effect to the speaker 17. The sound is emitted and the image is output and displayed on the display unit 14. The reproduction control unit 15 also has a function of reading out a song to be reproduced from a recording medium such as a CD (not shown) through the speaker 17 when a song to be reproduced is designated.

  On the other hand, the display unit 14 acquires information indicating the operation state of the image generation system S itself via the bus 19 and outputs an image corresponding to the operation state to the display unit 14.

  Further, as described above, the template input / output unit 13 functions as an input / output interface for acquiring a template from the outside or outputting the template to the outside.

  Finally, the template control unit 12 is the center of execution of an image generation process described later, and specifically includes a CPU and a volatile memory (not shown).

  Next, the image generation processing according to the first embodiment will be specifically described with reference to FIG. Note that the image generation processing shown in FIG. 7 includes not only the generation of an image that is played back in parallel with a song, but also the display of the generated image and the playback of the corresponding song.

  As shown in FIG. 7, in the image generation processing according to the first embodiment, first, song information corresponding to a song desired to be reproduced is read from a CD or the like (not shown) in the reproduction control unit 15 ( In step S1), the contents (speed and chord configuration) are analyzed in the music analysis unit 11 (step S2).

  Next, the analysis result is compared with the coordinate space shown in FIG. 4, and the coordinate value in the coordinate space of the music to be reproduced is defined (step S3).

  Next, the still image or moving image having the defined coordinate value is retrieved and read in the still image recording unit 3 or the moving image information recording unit 4 (step S4), and further defined. A template having a coordinate value (basic template) is searched and read in the template recording unit 2 (step S5).

  Then, a new template is created in the template control unit 12 based on the basic template using the information acquired in the steps S2 to S4 (step S6).

  Here, specifically, the processing of step S6 specifically includes the display switching order of the image acquired in step S4, the display switching timing of the image, the display timing of the corresponding character information, the playback timing of the corresponding sound effect, the acquisition The display switching pattern of the acquired image and the display effect mode of the acquired image are respectively determined. At this time, the image display switching order, the image display switching pattern, and the image display effect mode are selected so that, for example, the image displayed once and the display switching pattern and display effect mode adopted once are displayed or not adopted. The display switching order, display switching pattern, and display mode effect are determined at random. Further, the display switching timing of the image, the display timing of the character information, and the playback timing of the sound effect are determined so that the respective display switching or playback switching is performed at the switching timing of the chord obtained by the analysis in step S2, for example.

  When a new template is created by the processing in step S6, image display is executed in accordance with the switching timing and switching order included in the template created in parallel with the music to be reproduced along with the character information. Alternatively, a sound effect is output (step S7).

  Then, when the reproduction and display of the song and the image are completed, it is confirmed on the display unit 14 and an indicator (not shown) whether or not the reproduction of the song is finished (step S8). ), The image generation process and the music and image playback / display process according to the first embodiment are finished as they are, and when the same music is played back using another image (step S8; NO), the above is repeated. Returning to step S4, the processing is resumed.

  As described above, according to the image generation processing according to the first embodiment, a plurality of templates should be accumulated in advance, and a template corresponding to a song that is actually output should be extracted and output together with the song. Since the image is generated, when the viewer himself / herself creates a continuous image to be reproduced together with the music, an image rich in change can be created without getting tired.

  In addition, since the contents of a song that is actually output are analyzed and a template corresponding to the contents is extracted to generate an image, an image suitable for the song can be generated.

  In addition, since a template is newly created by selecting an image according to the content of the song that is actually output and an image is generated using it, it is more suitable for that song than when creating an image from the beginning. An image can be generated.

(II) Second Embodiment Next, a second embodiment which is another embodiment according to the present application will be described with reference to FIG. FIG. 8 is a flowchart showing a music and image reproduction process according to the second embodiment.

  The configuration of the image generation system according to the second embodiment is the same as the configuration of the image generation system S according to the first embodiment, and thus detailed description thereof is omitted. Further, in the flowchart shown in FIG. 8, the same processing as in the flowchart shown in FIG.

  In the first embodiment described above, the content of the music to be played back is analyzed, and a template suitable for the content is newly generated to generate an image. However, in the second embodiment described below, , Select a song and an image by a keyword specified by the viewer.

  That is, as shown in FIG. 8, in the image generation processing according to the first embodiment, first, when a search keyword for searching for a song to be played is input in the operation unit 8 (step S10). A song name, an artist name, or a genre name added to each song in a song database (not shown) (the song database may be constructed together with the template recording unit 2 or the like in the database 1 according to the embodiment). Is read (step S11), and the additional information and the input keyword are compared to select a song to be reproduced in the search processing unit 7 (step S12).

  On the other hand, in parallel with the music selection process, the folder name of each image included in the data database 1 is read (step S13), and the keyword input in step S10 and the read folder name and the like are read. A folder storing images to be used for image generation in comparison is selected (step S14).

  Then, the image stored in the selected folder is read (step S4), and character information is extracted from the additional information read in step S11 (step S15). From the image and the character information, these are extracted. A template that is previously set in advance corresponding to a basic template or keyword corresponding to is searched and read in the template recording unit 2 (step S16).

  In parallel with the reproduction of the music to be reproduced, image display in accordance with the switching timing and switching order included in the read template is executed and character information or sound effects are output (step S7). ).

  Then, when the reproduction and display of the song and the image are completed, it is confirmed on the display unit 14 whether or not the reproduction of the song is finished (step S8), and when finished (step S8; YES), the first is left as it is. When the image generation processing and the music / image reproduction / display processing according to the embodiment are finished, and when reproduction using another music or image is performed for the same keyword (step S8; NO), the above-described step S11 or again Returning to 13, the process is resumed.

  As described above, according to the image generation processing according to the second embodiment, as in the case of the first embodiment, a plurality of templates are stored in advance, and a template corresponding to a song to be actually output is stored. Since an image to be extracted and output together with the song is generated, when the viewer himself creates a continuous image reproduced along with the song, an image rich in change can be created without getting tired.

  In addition, according to the image generation processing according to the second embodiment, the template is extracted based on the keyword indicating the content of the song that is actually output, and the image is generated using the template. It is possible to more easily generate an image suitable for.

(III) Third Embodiment Next, a third embodiment, which is another embodiment according to the present application, will be described with reference to FIG. FIG. 9 is a flowchart showing the music and image reproduction processing according to the third embodiment.

  The configuration of the image generation system according to the third embodiment is the same as the configuration of the image generation system S according to the first embodiment, and thus detailed description thereof is omitted. Further, in the flowchart shown in FIG. 9, the same processing as in the flowchart shown in FIG. 7 is denoted by the same step number, and detailed description thereof is omitted.

  In the first embodiment described above, the contents of a song to be played back are analyzed, and a template suitable for the contents is newly generated to generate an image. However, in the third embodiment described below, Then, a template is selected based on additional information added in advance to the music designated by the viewer to generate an image.

  That is, as shown in FIG. 9, in the image generation processing according to the third embodiment, first, additional information added to a song to be reproduced (as described above, song name, artist name, genre name, and record). The company name is included) (step S20), and an image suitable for the additional information is stored in the still image recording unit 3 and the moving image recording unit 4 based on the character information included in the additional information. A search is made (step S21), and a basic template corresponding to the searched image and the additional information is searched and read in the template recording unit 2 (step S22). Instead of the process of step S21, an image suitable for the additional information may be acquired from the outside via the receiving unit 10 and used for the subsequent process of step S22.

  In parallel with the reproduction of the music to be reproduced, image display in accordance with the switching timing and switching order included in the read template is executed and character information or sound effects are output (step S7). ).

  Then, when the reproduction and display of the song and the image are completed, it is confirmed on the display unit 14 whether or not the reproduction of the song is finished (step S8), and when finished (step S8; YES), the first is left as it is. When the image generation processing and the music / image reproduction / display processing according to the embodiment are finished, and when reproduction using another image is performed for the same music (step S8; NO), the process returns to step S20 again. Resume processing.

  As described above, according to the image generation processing according to the third embodiment, as in the case of the first embodiment, a plurality of templates are stored in advance, and a template corresponding to a song to be actually output is stored. Since an image to be extracted and output together with the song is generated, when the viewer himself creates a continuous image reproduced along with the song, an image rich in change can be created without getting tired.

  In addition to this, according to the image generation processing according to the third embodiment, additional information indicating the attributes of the music to be output and an image stored in advance or acquired from the outside should be used to reproduce with the music. Since the image is generated, an image suitable for the outputted music can be generated more easily.

(IV) Fourth Embodiment Next, a fourth embodiment which is another embodiment according to the present application will be described with reference to FIG. FIG. 10 is a flowchart showing the reproduction processing of music and images according to the fourth embodiment.

  The configuration of the image generation system according to the fourth embodiment is the same as the configuration of the image generation system S according to the first embodiment, and thus detailed description thereof is omitted. Further, in the flowchart shown in FIG. 10, the same processing as in the flowchart shown in FIG. 7 is denoted by the same step number, and detailed description thereof is omitted.

  In the first embodiment described above, the contents of the music to be played back are analyzed, and a template suitable for the contents is newly generated to generate an image. However, in the fourth embodiment described below, Although basically the same processing as in the first embodiment, the method of utilizing the coordinate space shown in FIG. 4 when generating an image for reproduction is different.

  That is, as shown in FIG. 10, in the image generation processing according to the fourth embodiment, first, music information corresponding to a music that is desired to be reproduced is obtained from a CD or the like not shown in the reproduction controller 15. Reading (step S1), the contents (speed and chord configuration) are analyzed in the music analysis unit 11 (step S2).

  Next, the analysis result is compared with the coordinate space shown in FIG. 4, and the coordinate value in the coordinate space of the music to be reproduced is defined (step S3).

  Next, a still image or a moving image having the coordinate value in the vicinity of the defined coordinate value is searched and read in the still image recording unit 3 or the moving image information recording unit 4 (step S25), Further, a basic template having the coordinate value in the vicinity of the defined coordinate value is searched and read in the template recording unit 2 (step S26).

  Then, a new template is created in the template control unit 12 based on the basic template using each information acquired in steps S1 to S3 and steps S25 to S27 (step S27). Here, the details of the template creation process executed in step S27 are the same as the template creation process (step S6) in the first embodiment described above, and thus detailed description thereof is omitted.

  When a new template is created by the process of step S27, image display is executed in accordance with the switching timing and switching order included in the template created in parallel with the music to be reproduced along with the character information. Alternatively, a sound effect is output (step S28).

  Then, when the reproduction and display of the music and image are completed, the coordinate value in the coordinate space shown in FIG. 4 of the new template used for displaying the image is newly defined (step S29). The new template is stored in the template recording unit 2 together with the defined coordinate values (step S30), and information indicating the number of times of use of the used material image (currently one time) is stored in the still image recording unit 3 and the moving image recording unit. 4 is recorded (step S31).

  When the recording of the number of times of use is completed, next, it is confirmed on the display unit 14 whether or not the reproduction of the music is to be ended (step S32). When the image generation process and the music / image reproduction / display process are finished, and when the same music is reproduced using another image (step S32; NO), the process returns to step S25 again to resume the process. .

  As described above, according to the image generation processing according to the fourth embodiment, as in the case of the first embodiment, a plurality of templates are stored in advance, and a template corresponding to a song to be actually output is stored. Since an image to be extracted and output together with the song is generated, when the viewer himself creates a continuous image reproduced along with the song, an image rich in change can be created without getting tired.

  In addition, since the contents of a song that is actually output are analyzed and a template corresponding to the contents is extracted to generate an image, an image suitable for the song can be generated.

  In addition, since a template is newly created by selecting an image according to the content of the song that is actually output and an image is generated using it, it is more suitable for that song than when creating an image from the beginning. An image can be generated.

  Furthermore, since the templates used for generating the images are accumulated, when outputting music in a similar state, it is possible to quickly generate images by omitting the process of generating similar templates again.

(V) Fifth Embodiment Next, a fifth embodiment, which is another embodiment according to the present application, will be described with reference to FIG. FIG. 11 is a flowchart showing the reproduction processing of music and images according to the fifth embodiment.

  The configuration of the image generation system according to the fifth embodiment is the same as the configuration of the image generation system S according to the first embodiment, and thus detailed description thereof is omitted. Further, in the flowchart shown in FIG. 11, the same processing as that in the flowchart shown in FIG. 10 is denoted by the same step number, and detailed description thereof is omitted.

  In the above-described fourth embodiment, the content of a song to be played back is analyzed, and a template suitable for the content is newly generated to generate an image. However, in the fifth embodiment described below, , Select a song and an image by a keyword specified by the viewer.

  That is, as shown in FIG. 11, in the image generation processing according to the fifth embodiment, first, when a search keyword for searching for a song to be played is input in the operation unit 8 (step S35). Then, the input content is compared with the coordinate space shown in FIG. 4, and the coordinate value in the coordinate space of the input keyword is defined (step S36).

  Next, a song having the coordinate value in the vicinity of the defined coordinate value is searched and read in a song database (not shown) (step S37), and the song is read in the vicinity of the defined coordinate value. A still image or moving image having a coordinate value is searched for and read in the still image recording unit 3 or the moving image information recording unit 4 (step S25), and the coordinate value is in the vicinity of the defined coordinate value. Are searched for and read in the template recording unit 2 (step S26).

  Thereafter, the processing in steps S27 to S32 in the fourth embodiment described above is executed in the same manner as in the fourth embodiment, and a series of image generation processing and music piece reproduction processing is completed.

  As described above, according to the image generation processing according to the fifth embodiment, as in the case of the first embodiment, a plurality of templates are accumulated in advance, and a template corresponding to a song to be actually output is stored. Since an image to be extracted and output together with the song is generated, when the viewer himself creates a continuous image reproduced along with the song, an image rich in change can be created without getting tired.

  In addition, a keyword for searching for a song to be output is input to search for the song, and a playback image is generated using the image corresponding to the song and the analysis result of the content of the song. Necessary music can be selected with a simple process from the stage of selecting a music.

  In the fifth embodiment described above, a keyword input in the past and a keyword input in the past are used together and analyzed to select a song corresponding to both and generate an image. You can also.

  In this case, since one song is searched using a plurality of keywords input at different timings, the song can be selected more appropriately reflecting the viewer's intention.

(VI) Sixth Embodiment Next, a sixth embodiment, which is another embodiment according to the present application, will be described with reference to FIG. FIG. 12 is a flowchart showing a music and image reproduction process according to the sixth embodiment.

  The configuration of the image generation system according to the sixth embodiment is the same as the configuration of the image generation system S according to the first embodiment, and thus detailed description thereof is omitted.

  In the first embodiment and the like described above, the case where the contents of a song to be reproduced is analyzed and a template suitable for the contents is newly generated to generate an image has been described. In the sixth embodiment described below, In this case, information on the mood of a viewer who is playing a song and watching an image at that time is used for selecting a song and generating an image.

  That is, as shown in FIG. 12, in the image generation processing according to the sixth embodiment, first, character information indicating the current mood of the viewer is input as mood information in the operation unit 8 (step S40). .

  Here, specifically, the mood information is any of “happy”, “fun”, “cut” or “sad” described in each quadrant in the coordinate space shown in FIG. 4 as character information. It is input at the operation unit 8.

  Next, a song having the coordinate value in the quadrant of the coordinate value of the inputted mood information is searched and read in a song database (not shown) (step S41), and in the vicinity of the coordinate value of the song. The still image or moving image having the coordinate value is searched for and read in the still image recording unit 3 or the moving image information recording unit 4 (step S42), and the coordinate value is set in the vicinity of the coordinate value of the song. The basic template is searched and read in the template recording unit 2 (step S43).

  In addition to this, the character information having the coordinate value in the vicinity of the coordinate value of the music is searched for and read in the character information recording unit 5 (step S44), and the coordinate value is in the vicinity of the coordinate value. Is retrieved and read in the sound effect recording unit 6 (step S45).

  Then, the image created using the image, character information and sound effect information selected in steps S42, S44 and S45 is displayed while reproducing the song selected in step S41 according to the template selected in step S43. (Step S46).

  Then, when the reproduction and display of the song and the image are completed, it is confirmed on the display unit 14 whether or not the reproduction of the song is finished (step S47). If not finished (step S47; NO), the previous step A process of setting an identification flag or the like is performed so that the music, image, template, character information, and sound effect information selected in the processes of S41 to S45 are not selected again (step S48), and another music corresponding to the same mood information is performed. Then, the process returns to step S41 so that the image is reproduced / displayed.

  On the other hand, when it is determined in step S47 that the reproduction of the song is to be ended (step S47; YES), the image, character information, and sound effect information reproduced immediately before are used as a new template corresponding to the mood information and the song. It is confirmed with the user whether to synthesize (step S49). If a new template is to be synthesized (step S49; YES), the new template is synthesized (step S50), and the series of processes is terminated.

  On the other hand, when a new template is not synthesized in the determination in step S49 (step S49; NO), the series of processes is terminated as it is.

  As described above, according to the image generation processing according to the sixth embodiment, as in the case of the first embodiment, a plurality of templates are stored in advance, and a template corresponding to a song that is actually output is stored. Since an image to be extracted and output together with the song is generated, when the viewer himself creates a continuous image reproduced along with the song, an image rich in change can be created without getting tired.

  In addition to this, according to the image generation processing according to the sixth embodiment, mood information indicating a viewer's mood is acquired, a song is extracted based on the mood information, and a corresponding template is specified to generate an image. Therefore, it is possible to output an appropriate song and image according to the mood of the viewer.

  Furthermore, since the template is edited based on the character or sound effect corresponding to the song to be output, and an image is generated using the edited template, the image is based on both the actually viewed image and the song. Can be generated.

(VII) Seventh Embodiment Next, a seventh embodiment, which is another embodiment according to the present application, will be described with reference to FIG. FIG. 13 is a flowchart showing a music and image reproduction process according to the seventh embodiment. In the flowchart shown in FIG. 13, the same processing as that in the flowchart shown in FIG.

  Furthermore, since the configuration of the image generation system according to the seventh embodiment is the same as the configuration of the image generation system S according to the first embodiment, detailed description thereof is omitted.

  In the first embodiment and the like described above, the case where the content of a song to be reproduced is analyzed and a template suitable for the content is newly generated to generate an image has been described. However, in the seventh embodiment described below, The result of detecting the change in the surrounding environment where the image generation system is installed by the sensor 18 is reflected in the generation of the image.

  That is, as shown in FIG. 13, in the image generation process according to the seventh embodiment, first, the threshold value preset in the sensor 18 exceeds the value of the environmental factor that is the detection target at that time. Therefore, it is constantly monitored whether or not the information is input from the sensor 18 (step S55). When there is no input (step S55; NO), the monitoring is continued as it is, while the monitoring is continued. When the information is input (step S55; YES), the same processing as that of the third embodiment is executed (steps S20 to S22 and S7 and S8).

  As described above, according to the image generation processing according to the seventh embodiment, as in the case of the first embodiment, a plurality of templates are stored in advance, and a template corresponding to a song to be actually output is stored. Since an image to be extracted and output together with the song is generated, when the viewer himself creates a continuous image reproduced along with the song, an image rich in change can be created without getting tired.

  In addition to this, according to the image generation processing according to the seventh embodiment, an image to be reproduced together with the song is generated using the additional information indicating the attribute of the outputted song and the image accumulated in advance. Thus, an image suitable for the outputted music can be generated more easily.

  In addition, since the image generation timing can be controlled in accordance with changes in the surroundings, the image can be generated in a more appropriate manner depending on the surrounding environment.

(VIII) Eighth Embodiment Next, an eighth embodiment which is another embodiment according to the present application will be described with reference to FIG. FIG. 14 is a flowchart showing the music and image reproduction processing according to the eighth embodiment. Further, in the flowchart shown in FIG. 14, the same processes as those in the flowchart shown in FIG.

  Furthermore, the configuration of the image generation system according to the eighth embodiment is the same as the configuration of the image generation system S according to the first embodiment, and thus detailed description thereof is omitted.

  In the fifth embodiment described above, the case where a song and an image are selected by a keyword specified by the viewer has been described. However, in the eighth embodiment described below, the surrounding environment in which the image generation system is installed is described. The result of detecting the change in the sensor 18 is reflected in the generation of the image.

  That is, as shown in FIG. 14, in the image generation processing according to the eighth embodiment, first, information indicating that the value of the environmental factor that is the detection target at that time has been input is input from the sensor 18. Whether or not is constantly monitored (step S56), and when there is no input (step S56; NO), it waits for the input as it is, while when there is the input (step S56; YES), At the timing when the input is made, for example, keywords are set at random (step S57), and the process proceeds to the subsequent steps after step S36.

  On the other hand, if it is determined in step S56 that the keyword is input (step S56; YES), the same processing as in the fifth embodiment is executed (steps S25 to S32, S36, and S39).

  As described above, according to the image generation processing according to the eighth embodiment, as in the case of the first embodiment, a plurality of templates are stored in advance, and a template corresponding to a song to be actually output is stored. Since an image to be extracted and output together with the song is generated, when the viewer himself creates a continuous image reproduced along with the song, an image rich in change can be created without getting tired.

  In addition, a keyword for searching for a song to be output is input to search for the song, and a playback image is generated using the image corresponding to the song and the analysis result of the content of the song. Necessary music can be selected with a simple process from the stage of selecting a music.

  In addition to this, according to the image generation processing according to the eighth embodiment, keywords can be set at a timing according to changes in the surroundings, so that an image can be generated appropriately corresponding to the surrounding environment. .

(IX) Ninth Embodiment Finally, a ninth embodiment, which is another embodiment according to the present application, will be described with reference to FIG. FIG. 15 is a flowchart showing the music and image reproduction processing according to the ninth embodiment. Further, in the flowchart shown in FIG. 15, the same processes as those in the flowchart shown in FIG.

  Furthermore, since the configuration of the image generation system according to the ninth embodiment is the same as the configuration of the image generation system S according to the first embodiment, detailed description thereof is omitted.

  In the sixth embodiment described above, a case has been described in which information related to the mood of a viewer who plays a song and views an image is used for song selection and image generation. In the ninth embodiment to be described, the result of detection by the sensor 18 of a change in the surrounding environment where the image generation system is installed is reflected in image generation.

  That is, as shown in FIG. 15, in the image generation processing according to the ninth embodiment, first, was information from the sensor 18 indicating that the value of the environmental factor that was the detection target at that time was exceeded. No is constantly monitored (step S60), and when there is no input (step S60; NO), it waits for the input as it is, while when there is the input (step S60; YES), At the timing of the input, for example, mood information is randomly set from four (step S61), and the process proceeds to the processing from the subsequent step S41 to step S50.

  As described above, according to the image generation processing according to the ninth embodiment, as in the case of the first embodiment, a plurality of templates are stored in advance, and a template corresponding to a song to be actually output is stored. Since an image to be extracted and output together with the song is generated, when the viewer himself creates a continuous image reproduced along with the song, an image rich in change can be created without getting tired.

  In addition to this, according to the image generation processing according to the ninth embodiment, mood information indicating the mood of the viewer is acquired, a song is extracted based on the mood information and an image is generated by designating a corresponding template. Therefore, it is possible to output an appropriate song and image according to the mood of the viewer.

  Furthermore, since the template is edited based on the character or sound effect corresponding to the song to be output, and the image is generated using the edited template, the image is based on both the actually viewed image and the song. Can be generated.

  Furthermore, since the mood information can be set at a timing corresponding to a change in the surroundings, it is possible to generate an image corresponding more appropriately to the surrounding environment.

  In each of the above-described embodiments, when a new template is created, it may be recorded in, for example, a nonvolatile portable memory. In this case, the template can be separated from the image generation system S, and the use as a template can be expanded. That is, the template can be carried for personal use and used for new image formation on the go, or the template can be shared with others to expand the range of use of one template. be able to.

  Further, the database 1 shown in FIG. 1 can be constructed not in the image generation system S but in a server on the Internet and read out each information from this server as necessary.

  Further, a program corresponding to the flowcharts of FIGS. 7 to 15 is recorded on an information recording medium such as a flexible disk or a hard disk, or is acquired and recorded via the Internet or the like, and is read by a general-purpose computer. It is possible to utilize the computer as the template control unit 12 according to the embodiment.

1 is a block diagram illustrating a schematic configuration of an image generation system according to a first embodiment. It is a figure which illustrates the kind of template which concerns on 1st Embodiment. It is a figure which illustrates additional information on an image concerning a 1st embodiment. It is a figure which illustrates the search coordinate space of 1st Embodiment. It is a figure which illustrates the keyword which concerns on 1st Embodiment, (a) is a figure which illustrates a free keyword, (b) is a figure which illustrates a coordinate corresponding | compatible keyword. It is a figure which illustrates the environmental factor which concerns on 1st Embodiment. It is a flowchart which shows the image generation process which concerns on 1st Embodiment. It is a flowchart which shows the image generation process which concerns on 2nd Embodiment. It is a flowchart which shows the image generation process which concerns on 3rd Embodiment. It is a flowchart which shows the image generation process which concerns on 4th Embodiment. It is a flowchart which shows the image generation process which concerns on 5th Embodiment. It is a flowchart which shows the image generation process which concerns on 6th Embodiment. It is a flowchart which shows the image generation process which concerns on 7th Embodiment. It is a flowchart which shows the image generation process which concerns on 8th Embodiment. It is a flowchart which shows the image generation process which concerns on 9th Embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Database 2 Template recording part 3 Still image recording part 4 Moving image recording part 5 Character information recording part 6 Sound effect information recording part 7 Search processing part 8 Operation part 9 Image input part 10 Receiving part 11 Song analysis part 12 Template control part 13 Template input / output unit 14 Display unit 15 Playback control unit 16 Internet 17 Speaker 18 Sensor

Claims (16)

In an image generation system that generates an image output in parallel with music,
An image template that is a template of the contents of the image to be output in parallel with a plurality of types of music having different attributes, and a plurality of preset templates corresponding to each type of music Storage means for storing image templates of
Extraction means for extracting one image template corresponding to the image to be actually output in parallel with the music actually output from the plurality of stored image templates as an extraction template;
Generation means for generating image information corresponding to the image to be actually output using the extracted extraction template;
An image generation system comprising: The image generation system according to claim 1,
Analyzing the content of the music that is actually output, further comprising analysis means for generating content information indicating the content,
The extraction means extracts the extraction template based on the generated content information,
Further, the generation means edits the extraction template based on the generated content information to generate the image information. The image generation system according to claim 2,
Image storage means for storing material image information corresponding to a material image as a material constituting the image information;
Material image information extraction means for extracting the material image information corresponding to the actually output music from the image storage means based on the generated content information;
Further comprising
The image generation system, wherein the generation unit edits the extraction template based on the generated content information and the extracted material image information to generate the image information. The image generation system according to claim 1,
The music information corresponding to the music includes keyword information corresponding to a keyword indicating the content of the music,
Further comprising keyword information extraction means for extracting the keyword information added to the music information corresponding to the music that is actually output from the music information;
The extraction means extracts the extraction template based on the extracted keyword information,
Further, the generation means generates the image information corresponding to the generated extraction template. The image generation system according to claim 1,
The music information corresponding to the music is added with additional information indicating at least the attribute of the music,
Image storage means for storing material image information corresponding to a material image as a material constituting the image information;
Additional information extraction means for extracting the additional information corresponding to the music that is actually output from the music information;
Further comprising
The extraction means extracts the extraction template based on the extracted additional information and the accumulated material image information,
Further, the generation means generates the image information corresponding to the generated extraction template. The image generation system according to claim 1,
Music information storage means for storing a plurality of music information corresponding to the music;
Image storage means for storing material image information corresponding to a material image as a material constituting the image information;
Input means used for inputting a keyword for searching for the music that is actually output;
Analyzing the content of the input keyword and generating keyword content information generating means corresponding to the content;
Music information extraction means for extracting from the music information storage means the music information corresponding to the music that is actually output based on the generated keyword content information;
Image information extraction means for extracting the material image information corresponding to the extracted music information from the image storage means;
Analyzing means for analyzing the content of the extracted music information and generating music content information indicating the content of the music information;
Further comprising
The extraction means extracts the extraction template based on the extracted music information,
The image generation system characterized in that the generation means generates the image information using the extracted extraction template based on the generated music content information. The image generation system according to claim 6.
The keyword content information generating unit analyzes the contents of a plurality of keywords input at different timings, and generates one keyword content information corresponding to each content. The image generation system according to claim 1,
Music information storage means for storing a plurality of music information corresponding to the music;
Image storage means for storing material image information corresponding to a material image that is a material constituting the image information;
Obtaining means for obtaining mood information indicating a mood of a viewer who views the music and images output;
Analyzing the content of the acquired mood information, and generating mood content information generating means corresponding to the content;
Music information extraction means for extracting from the music information storage means the music information corresponding to the music that is actually output based on the generated mood content information;
Image information extraction means for extracting the material image information corresponding to the music that is actually output from the image storage means based on the generated mood content information;
Further comprising
The extraction means extracts the extraction template based on the generated mood content information,
The image generating system, wherein the generating means generates the image information using the extracted extraction template. The image generation system according to claim 1,
Character information storage means for storing character information corresponding to characters output in parallel with the music and the image;
Sound effect information storage means for storing sound effect information corresponding to the sound output in parallel with the music and the image;
Acquisition means for acquiring mood information indicating a mood of a viewer who views the output music and images;
Analyzing the content of the acquired mood information, and generating mood content information corresponding to the content; mood content information generating means;
Music information extraction means for extracting from the music information storage means the music information corresponding to the music that is actually output based on the generated mood content information;
Character information extraction means for extracting the character information corresponding to the music that is actually output based on the generated mood content information from the character information storage means;
A sound effect information extracting means for extracting the sound effect information corresponding to the actually output music from the sound effect information accumulating means based on the generated mood content information;
Further comprising
The generation means edits the extraction template based on the generated content information, the extracted character information, and the extracted sound effect information, and generates the image information. Image generation system. The image generation system according to claim 8 or 9,
The mood content information generation means analyzes the contents of a plurality of mood information acquired at different timings, and generates one mood content information corresponding to each content. The image generation system according to any one of claims 2, 3, 6 to 10,
The storage unit stores the extraction template used for generating the image information as a new image template. In the image generation system according to any one of claims 1 to 11,
An environment change detecting means for detecting a change in the surrounding environment in which the image system is operating and generating a change detection signal;
An image generation system, wherein a process for generating the image information is executed based on the generated change detection signal. The storage means in the image generation system according to any one of claims 1 to 11,
Recording means for causing the storage means to record the image template;
A storage device comprising: An image template that is a template of the contents of the image to be output in parallel with a plurality of types of music having different attributes, and a plurality of preset templates corresponding to each type of music An image generation method that is executed in an image generation system that includes storage means for storing the image template and generates the image using the image template,
An extraction step of extracting, as an extraction template, one image template corresponding to the image that is to be actually output in parallel with the music that is actually output, from among the plurality of stored image templates;
Using the extracted extraction template to generate image information corresponding to the image to be actually output;
An image generation method comprising: A computer included in an image generation system that generates an image output in parallel with music,
An image template that is a template for the contents of the image to be output in parallel with a plurality of types of music having different attributes, and a plurality of preset templates corresponding to each type of the music Storage means for storing image templates of
Extraction means for extracting one image template corresponding to the image that is to be actually output in parallel with the music that is actually output from the plurality of stored image templates, as an extraction template; and ,
Generating means for generating image information corresponding to the image to be actually output using the extracted extraction template;
An image generation program characterized by being caused to function as a computer program.   16. An information recording medium in which the image generation program according to claim 15 is recorded so as to be readable by the computer.

Images