JP4219906B2 - Video content creation device and video content creation program - Google Patents

Description

  The present invention relates to a video content creation device, and more particularly to a novel video content creation device that creates video content such as a memory video using past photographs of persons with dementia.

  With the rapid increase in the elderly population, it is urgent to provide adequate care for people with dementia (or people with dementia. However, in the following explanation, the term dementia will be used for convenience) It can be said that. However, families who are generally cared for as a caregiver for dementia cannot keep an eye on the dementia for 24 hours. Abuse of dementia due to the stress has recently become a social problem. It has become. In order to reduce the stress of both dementia and caregivers, it is important to bring out psychological stability of the dementia. For this purpose, there is a recollection method that lets you watch old movies, music, and photos of the person. However, it is difficult to do it at home because it requires someone to speak.

For this reason, Non-Patent Document 1 proposes a technique using a memorized video. The memory video is a slide show video of old photos of people with dementia plus BGM and narration.
Yasuda et al .: Created a memorable photo video for people with dementia and evaluated concentration. 28th Annual Meeting of the Society for Higher Brain Dysfunction (2004)

  Although the memorized video has been confirmed to be effective in creating a stable mental state of dementia, the creation of the memorized video requires video editing know-how and is not easy for ordinary caregivers to do. .

  In addition, it is desirable that such a memorized video be viewed by a person with dementia without losing interest and concentration as much as possible.

  Therefore, a main object of the present invention is to provide a video content creation device capable of creating video content such as a memory video using a past photograph of a demented person relatively easily.

  Another object of the present invention is to provide a video content creation device capable of creating video content such as a memory video that is easily viewed by a person with dementia.

The invention of claim 1 is a photographic data input means for inputting photographic image data, a meta information giving means for giving meta information related to the photographic image data, and a storage means for storing the photographic image data together with the related meta information. A search means for searching for photographic image data from the storage means using the meta information, and a generating means for generating a series of video contents by editing the photographic image data searched by the search means according to the meta information, look including video effect applying means for applying a video effect based on the meta information, the meta information includes a region between the interval information of the region that is set in one of the photographic image data, the image effect applying means interregion interval information This is a video content creation apparatus that applies different video effects depending on the video content.

  According to the first aspect of the present invention, for example, photographic image data is input to the computer (12) from a photographic data input device (20. Reference numerals indicating parts or elements corresponding to the embodiments; the same applies hereinafter) including an image scanner. . In the computer, means for giving meta information to the photographic image data, for example, Dublin Core, Image Regions, FOAF, etc. are provided. The meta information related to the inputted photographic image data is given by these meta information giving means, and the computer stores the photographic image data in the storage means (22) together with the related meta information. The search means (12, S2) searches the storage means or the photographic image data using the meta information. The generation means (12, S6) generates video content such as a slide show in the AVI format or the Flash movie format, for example, by editing the retrieved photographic image data according to the meta information. Here, the video effect applying means (12, S22, S36) included in the generating means applies video effects such as zoom-up, pan, fade-out, fade-in, etc. according to the meta information. And if the picture is an old picture of a person with dementia, the video content can be called a memory video.

In the invention of claim 1, since the photographic image data is edited using the meta information, even if the user is not very familiar with the production of the video content, for example, using an old photograph of a dementia patient, Video content such as video can be created. Moreover, since various video effects according to the meta information are applied, it is easy to maintain the interest and concentration of the demented person who watches the memory video, and further improvement of the memory video effect can be expected.
Further, when a region is set in one photographic image, meta information of the region, for example, the coordinate position, height, width, etc. of the region is registered by the meta information adding means. Therefore, such region information includes interval information between the regions. Then, the video effect applying means (12, S22) applies different video effects according to the inter-region interval information. For example, when the inter-region interval is equal to or smaller than a predetermined threshold, after zooming up one region, the subsequent region is panned. When the inter-region interval is equal to or greater than a predetermined threshold, instead of panning, one region is zoomed up, then that region is faded out, and the subsequent region is faded in. Since the appropriate video effect is automatically given using the inter-region interval information of the meta information, a troublesome operation for giving the video effect is not required.

According to a second aspect of the invention, the meta information includes picture information that is set in each photographic image data, the image effect applying means for applying a video effect according to the photographic information, create video content according to claim 1, wherein Device.

In the invention of claim 2 , the video effect applying means (12, S36) gives a video effect according to the photographic information. Therefore, since an appropriate video effect is automatically given using the photographic information, a troublesome operation for giving the video effect is not required.

The invention of claim 3 is the video content creation device according to claim 2 , wherein the photo information includes date information, and the video effect applying means applies different video effects according to the age difference between the photos based on the date information. .

According to the third aspect of the present invention, when a plurality of photograph images are captured, the age difference between the photographs can be detected based on the photographing date included in the meta information providing means. The video effect applying means (12, S36) applies different video effects according to the age difference between the photographs. For example, if the age difference is less than or equal to a predetermined threshold, the next photo is faded in while fading out the previous photo, and they are overlapped. If the age difference is greater than or equal to a predetermined threshold, the next photo is slid in. In this way, since an appropriate video effect is automatically given using the age difference between each photograph, a troublesome operation for giving the video effect is not required.

According to a fourth aspect of the present invention, there is provided the video content creating apparatus according to the second or third aspect , wherein the photographic information includes person information, and the video effect applying means applies the video effect according to the person information.

In the invention of claim 4 , when it is found from the person information that the subject is a person who is memorable to the viewer, the computer applies the video effect only to the region in which the subject is present, for example.

A fifth aspect of the present invention is the video content creation device according to any one of the second to fourth aspects, wherein the photographic information includes color tone information, and the video effect applying means applies the video effect according to the color tone information.

According to the invention of claim 5 , when the color tone information is detected, for example, when the photograph changes from monochrome to color, for example, a color photograph is first displayed in monochrome tone and then gradually shifted to color display. Gives an effect.

The invention of claim 6 relates to a picture data input step for inputting picture image data, a meta information giving step for giving meta information in relation to the picture image data, and picture image data related to the computer of the video content creation device. Storing with the meta information to be stored, a search step for searching the photographic image data from the storage means using the meta information, and editing the photographic image data searched by the search means according to the meta information to generate a series of video contents A generation step is executed, and in the generation step, a video effect application step for applying a video effect based on the meta information is executed . The meta information is inter-region interval information of a region set in one photo image data. Between the regions in the video effect application step Applying a video effect vary depending on the information, a video content creation program.
The invention of claim 7 is a storage means for storing photographic image data together with related meta information, a search means for searching photographic image data from the storage means using the meta information, and a thumbnail image according to the photographic image data searched by the search means. A display means for displaying a GUI screen including a thumbnail image display area for displaying and a reproduction order setting area having a photo list field, and dragging and dropping the thumbnail image displayed in the thumbnail image display area to the photo list field By selecting multiple photos to be actually used from the searched photo data and changing the list loading order in the photo list column, you can change the multiple playback order, and the photos loaded in the photo list column Read out photographic image data and related meta information from the storage means Readout means, and generation means for generating a series of video contents by editing the photographic image data read out by the readout means in accordance with the meta information, the meta information being region information of a region set in one photographic image data And generating means includes positioning means for aligning an image of the region at a predetermined position in accordance with the region information, and further generating means includes video effect applying means for applying a video effect based on the region information. Device.
The invention of claim 8 is a photographic data input means for inputting photographic image data, a meta information giving means for giving meta information in relation to the photographic image data, and a storage means for storing the photographic image data together with related meta information. A search means for searching for photographic image data from the storage means using the meta information, and a generating means for generating a series of video contents by editing the photographic image data searched by the search means according to the meta information, Including video effect applying means for applying a video effect based on meta information, the meta information including region information of a region set in one photographic image data, and the video effect applying means corresponding to the region information. This is a video content creation device to which is applied.

  According to the present invention, for example, by adding meta information to a past photo of a person with dementia and editing a photo image using the meta information, video content such as a memory video can be created relatively easily. can do. Moreover, since various video effects according to the meta information are applied, it is easy to maintain the interest and concentration of the demented person who watches the memory video, and further improvement of the memory video effect can be expected.

  The above object, other objects, features and advantages of the present invention will become more apparent from the following detailed description of embodiments with reference to the drawings.

  With reference to FIG. 1, a video content creation apparatus 10 of this embodiment includes a computer 12. As the computer 12, a personal computer or a workstation can be used.

  The computer 12 is provided with an internal memory 14 such as a hard disk or a RAM. In the internal memory 14, as a tool for inputting meta information, for example, Dublin Core (http: //Doublelincore.org), Image Regions (http://www.w3.org), FOAF (http://www.foaf-project.org), Jena2 (2.1) (http://jena.sourceforge.net) ) Etc. are set. All of these function as means for inputting, registering or giving meta information related to photographic image data.

  Here, meta information means structured information regarding data, and in the case of the embodiment, is information that structurally describes details of photographic image data to be handled.

  Dublin Core is known as a representative tool for handling meta information. In addition, a person's face and object area (hereinafter referred to as "region") in a photo is specified and used as meta information. Use Image Regions to hold. If the region is a person, add meta information of the person using FOAF. Regions are used to add effects such as zooming and panning between regions to still photos. Jena2 (2.1) is used to store the meta information in the database in the RDF format.

  Although not shown, the computer 12 includes a graphic board, a processor, a sound board, or a processor, through which a GUI screen and other graphics (video) are displayed on the monitor 16 and from the speaker 18 to the BGM or the like. Audio is output.

  Further, a photographic data input device 20 is connected to the computer 12. The photograph data input device 20 includes at least one of an image scanner, a digital camera (digital camera), the Internet (Web), and the like. The image scanner scans past photographs of persons with dementia and inputs color or monochrome photographic image data. Digital cameras can input photographic image data taken in real time, and can also be used to take past photographs and input those photographic image data. The Internet can be used to input photographic image data of a past photo of a dementia person transmitted from a remote place, and to capture photographic image data of an event related to the past of the dementia person as needed. Still other types of photo data input devices may be used.

  Further, a database 22 is coupled to the computer 12 via an interface 24. In this embodiment, a relational database called PostgreSQL 7.4 (http://www.postgresql.org) is used as the database 22.

  Although not shown, the computer 12 naturally has input means such as a keyboard and a mouse.

  Here, prior to the description of the operation of the embodiment in FIG. 1, the application of the video effect which is a feature of this embodiment will be described.

The inventors have assumed what kind of video effects (effects given to each photo, transitions between photos) that can be added to multiple creators with video editing experience when converting from photos to memory videos. An interview was conducted to see if this could be done. The results were the following A-1 to A-3 and B-1 to B-5.
(A) Effect A-1: A so-called Ken Burns effect (http://en.wikipedia.org) called zoom-up or panning on a rectangular area including a human face in a photograph (hereinafter referred to as a region). / Wiki / Ken_Burns).
A-2: Instead of panning, the zoomed-up region is faded out, and the next region is faded in.
A-3: First, a color photograph is displayed in monochrome, and then gradually changed to color display.
(B) Transition B-1: Fade out the next photo while fading out the previous photo, and overlap them.
B-2: Slide in the next photo.
B-3: Dissolve the previous photo and move to the next photo.
B-4: The previous photo is page peeled (an effect that rises from the lower right corner) and transitions to the next photo.
B-5: The previous photo is rotated about the vertical axis at the center to transition to the next photo.

  Based on the above results, we created a template to render the memories video as shown in Table 1. The items listed in “Used information” in Table 1 were designed as annotations (ontology) as annotations (anotation: “meta information”). Semantic Web (this is a next-generation Web technology that improves search performance and enhances convenience as a framework for adding annotations. Definition) ”is used. That is, the annotation is described in RDF (Resource Description Framework). This is because, in the future, when considering information exchange in which a photograph annotated by another person is used for a memory video of oneself or family, the affinity with the Web is taken into consideration.

  Furthermore, the inventors have established a policy of using the existing vocabulary as much as possible, and in the experiment for the embodiment of FIG. 1, the date and event of shooting are standard vocabulary related to bibliographic information. It was described in the above Dublin Core. In addition, in order to describe the information of the person in the photograph, the above-mentioned FOAF, which is a standard vocabulary for describing the information of the person, was used. In order to describe a human region in a photograph, the above-described Image Region is used, and the color tone of the photograph is Exif (http://it.jeta.or.jp/document/publica/standard/exif/english). /Jeida49e.htm) color space information. Also, regarding the relationship with the person (subject) in the photo, the knows property is defined in FOAF, but in creating a memory video, it is necessary to define the relationship between the person and the subject in more detail, so FOAF knows RELATIONSHIP (http://vocab.org/relationship/reference) defined by extending the attributes of the above was used. This was used to describe parent-child relationships, relative relationships, etc. The sample is shown in FIG.

  Furthermore, for the effects and transitions listed in Table 1, the ontology of the video effects used for creating the memory video is defined as shown in Table 2.

  Then, a conversion rule from photo annotation to video effect is described and used as a template. Table 3 shows a representation example of the rendering template “RT-8” in Table 1. However, a template is created in the same manner for other identifiers shown in Table 1.

  Here, the rendering template itself is also described as an RDF statement, and it is assumed that the rendering template is searched using the above-described RDQL. Note that the rendering ontology has a low necessity for interoperability (interoperability), so another unique format may be defined.

  In the embodiment described below, the video effects (effects and transitions) shown in Table 1 or Table 2 are given to the video content called the memory video using the template shown in Table 3 described above.

  In order to create a memory video using the video content creation apparatus 10 of the embodiment of FIG. 1, the computer 12 and the like operate according to the procedure shown in FIG.

  First, in step S1, photographic image data is input and meta information is registered. Specifically, photographic image data of a past photograph of the demented person is mainly input to the computer 12 using the photographic data input device 20 of FIG.

  At this time, a GUI (Graphical User Interface) screen 26 shown in FIG. The GUI 26 includes a photo display editing area 28 that occupies most of the left side of the monitor screen. The photo display / editing area 28 displays a photo of the photo image data input thereto, and is used for editing work such as designating a region. The GUI 26 includes a thumbnail display area 30 formed on the right side of the monitor screen. A thumbnail image is displayed in the thumbnail display area 30 in order to select a photo to be used from photographic image data that has been input and searched as described below.

  The GUI 26 includes a first meta information input area 32 formed below the monitor screen, a second meta information input area 34 formed substantially at the center of the monitor screen, and a playback order formed on the lower right side of the monitor screen. A setting area 36 is provided. In the meta information input area 32, meta information defined in Dublin Core related to all the photographs displayed in the area 28 is input. Further, the meta information input area 34 is an area that appears at the time of input in a pop-up format, and is used to input meta information when the region is a person using FOAF. The playback order setting area 36 is used to set the order in which the photos taken in the memory video are played back, and the order can be changed by drag and drop as will be described later.

  The photo input and meta information registration in step S1 will be specifically described. Referring to FIG. 4, when photographic image data of a photograph (a drawing is a line drawing in the drawing but is actually a photograph. The same applies to other drawings) as shown in FIG. The computer 12 displays the photograph (still image) represented by the image data in the photograph display editing area 28 of the GUI 26 of the monitor 16. At the same time, the computer 12 displays a thumbnail image of the photo in the thumbnail display area 30.

  Then, using a mouse or the like (not shown), if two regions, region 1 and region 2, are set in the photo, region 1 and region 2 are respectively displayed in the photo display editing area 28 as shown in FIG. The specified rectangular frames 29a and 29b are displayed. When such region 1 and region 2 are designated, meta information of each region shown in FIG. 6 is set in a format defined by Image Regions. That is, the coordinates (x11, y11) of the origin of the region 1 (the upper left corner of the rectangular frame) and the diagonal coordinates (x12, y12) thereof are registered, and the height h1 and width w1 of the region 1 are also set as meta. Registered as information. Similarly, for the region 2, the origin coordinates, diagonal coordinates, height, and width are registered as x21, y21, x22, y22, h2, and w2, respectively.

  Thus, if a photograph is input in step S1 and meta information is registered, a database as shown in FIG. 7, for example, is created. In FIG. 7, a photo display editing area 28 shown in FIG. 5 in which an actual photo is displayed and a region is designated is drawn on the right. Then, there is “dc:” in the ellipse, and the data in the rectangle related to it is meta information registered in Dublin Core. For example, “dc: date” has the date “200407716 (July 16, 2004)”, “dc: title” has the name “at Disney Animal Kingdom”, and “dc: description” has the description The sentence “They are very happy.” Is registered as meta information.

  The meta information “imgReg:” in the ellipse is meta information when a region is designated by Image Regions. “ImgReg: has region” is meta information indicating that the region is set, “imgReg: Rectangle” indicates that the region is rectangular, “imgReg: regionDepict” is a description of the region, and “imgReg: “BoundingBox” includes the origin position and size (height h, width w) of the region. “ImgReg: coords” is the coordinates of the origin position and the diagonal position of the region.

  The meta information given by “foaf: gender” is the gender (“female (female)” in the example) when the region is a person, and the meta information given by “foaf: name” is the name (in the example "Haruka (Haruka)"), the meta information given by "foaf: Person" shows the relationship between the viewer (the person watching this memorable video) and the person in the photo. In the example, "grandchild" "It can be seen that it is.

  In addition, each meta information shown in FIG. 7 is shown below, respectively.

xmlns: imgReg = “http://www.w3.org/2004/02/image-regions#”
xmlns: foaf = “xmlns.com/foaf/0.1”
xmlns: dc = “http://purl.org/dc/elements/1.1/”
When the photo input and meta information registration are performed in step S1 of FIG. 2, the computer 12 stores the photo and the meta information assigned thereto in the database 22. In the above description, one photo and its meta information have been described. However, every time photographic image data is input by the input device 20, meta information as shown in FIG. Photo data to which meta information is added is stored in the database 22.

  Thereafter, in step S2 in FIG. 2, the photograph used at that time is retrieved from the photographs input in step S1. However, the meta information described above is used in the photo search. For example, the following can be used as search conditions for photographs using meta information.

  First, in FOAF, when the region type is a person, meta information that identifies the person is added, so use this FOAF meta information to search for "photos of a specific person" be able to. A plurality of persons can be searched simultaneously, but in this case, a photograph in which all of the searched names are shown is targeted.

  When using the Dublin Core meta information, you can search by “shooting date”. For example, in “From (first specified date) to To (second specified date)”, it is possible to search for all photos taken after the first specified date and before the second specified date. With “From (specified date)”, you can search all photos taken after the specified date. Similarly, “To (specified date)” can be used to search for all photos taken before the specified date. In addition, it is possible to search for “photos whose specific properties include specific values”. For example, photos with “Disney” in “dc: title”. However, a plurality of images can be selected or searched at the same time, but at the time of selecting a plurality of images, only photographs that satisfy the set search conditions at the same time are targeted.

  In this way, when the selection of the photos is completed in step S2, next, in step S3, the photos to be actually used are selected from the searched photos, and when a plurality of photos are selected, the reproduction order of the plurality of photos is selected. Specify or determine The GUI 26 for this step S3 is shown in FIG. However, for the sake of convenience, it should be noted that FIG. 8 uses a photo different from the photo before FIG.

  For example, when step S3 is selected by a menu (not shown), the computer 12 displays the GUI 26 shown in FIG. In the GUI 26 of FIG. 8, a reproduction order setting area 36 shown in FIG. 3 is formed at the lower right of the monitor screen. An enlarged view of the reproduction order setting area 36 is shown in FIG.

  The reproduction order setting area 36 includes a photo list field 38, and as shown in FIG. 8, a photo to be used can be selected by dragging and dropping the thumbnail image displayed in the thumbnail display area 30 to the photo list field 38. . In other words, since all thumbnail images of the photos searched in the photo search step of step S2 are displayed in the thumbnail display area 30, by dragging and dropping the photos in the list image to the photo list column 38, the photos can be selected. You can choose. In the photo list column 38, columns of name (name), title (title), and date (date) are set.

  Further, in the playback order setting area 36, as shown in FIG. 9, a sort designation field 40, a generation button 42 for instructing generation of a slide show of a memorable video, a delete button 44 used for deleting a photo, and a selection A preview button 46 to be pressed (clicked) when previewing the photograph taken is provided. When the cursor is placed on any photo in the photo list field 38 and the preview button 46 is operated in this state, a preview image is displayed in the preview area 48 shown in FIG.

  In the sort designation field 40, properties for sorting the selected photos, such as registration date, are set. Then, when a specific property is set in the sort specification field 40 by, for example, a pull-down menu, the computer 12 sorts the photos in ascending order of the value for the property, for example.

  Furthermore, since the images are played back in the order of the photo list registered in the photo list column 38, if you want to change the playback order, drag and drop one line in the photo list column 38 to move the list. What is necessary is just to change order, ie, the order of reproduction | regeneration.

  Subsequently, in step S4 of FIG. 2, a BGM playlist to be played at the time of reproduction of the photo is registered or selected. In this embodiment, for example, an MP3 format music file is supported, and if a music used as a BGM is registered in the playlist, the BGM is reproduced along the playlist. If, for example, the M3U format is used as a playlist, a playlist created with general software such as Winamp (application name) can be used as it is.

  In step S4, first, “play list selection or registration” is selected in a menu (not shown). Then, the GUI 26 shown in FIG. 10 is displayed on the monitor 16. When the playlist creation button 52 of the GUI 26 is clicked, a new playlist registration field 54 shown in FIG. 11 is displayed. In the registration field 54, a name field 56 for setting the name of the playlist (“Sample” in the illustrated example) and a determination (acceptance) button 58 when the input name is acceptable are formed. When a name is entered in the name field 56 and the OK button 58 is clicked, an empty play list 60 shown in FIG. 12 is displayed.

  The MP3 file list 62 shown in FIG. 13 is displayed, and by dragging and dropping an appropriate music piece in the MP3 file list 62 to the play list 60, the music pieces used as BGM can be sequentially selected or registered in the play list 60. . In the example shown in the figure, nursery rhymes are mainly selected using the music list 62 (FIG. 13) called “MY album”.

  In this way, steps such as inputting photo data, registering meta information, searching for photos, selecting, selecting a song are executed, and when the generation button 42 shown in FIG. 9 is pressed, generation of a memory video is started. . Therefore, in step S5 of FIG. 2, it is determined whether or not the generation button 42 has been clicked. If "YES", the process proceeds to the next step S6 as it is, but if "NO", the process can be resumed by returning to any step S1-S4 in FIG.

  That is, each step S1-S4 in FIG. 2 can be executed any number of times at an arbitrary time by selecting from the menu, so that the operator needs as much as necessary when there is time. You just have to do this. In any case, since the result of the previous work is stored in the database 22, in this work, first, the data up to the previous time is read from the database 22, and then the process that continues or changes the process. I do.

  When it is detected in step S5 that the generation button 42 is turned on, in the next step S6, a slide show video (RVML) of a memory video is generated.

  Here, “RVML” is a kind of XML designed so that swf can be completely expressed, and all versions of swf moving images can be expressed as RVML. However, swf is basically only a column of tags excluding a file header having information such as a Flash version and a frame rate. For example, a pattern in which a figure is defined by a definition tag, the figure is arranged in a frame by an operation tag, and the current frame is drawn on the screen by a display tag corresponds to one frame, and this is repeated.

  The generated RVML can create a Flash movie format memory video by using KineticFusion (http://www.kinesissoftware.com), which is a Flash tool, from the RVML.

  Specifically, step S6 of FIG. 2 is executed according to the procedure shown in FIG. In the first step S11, in response to the operation of the generation button 42, the computer 12 first starts playing the music of the playlist 60 (FIG. 14) as BGM. Thereafter, the music is sequentially switched along the playlist 60 and continuously reproduced. That is, the computer 12 reads out music data registered in the playlist 60 and processes the data with a sound board or a processor, so that the music in the playlist is reproduced as sound from the speaker 18 (FIG. 1).

  In the next step S13, the computer 12 sets the width of the photo display editing area 28 (FIG. 3) of the GUI 26 formed on the display screen of the monitor 16 as Sw and the height as Sh.

  In step S15, the computer 12 increments the number of photographs n (n = n + 1). In the next step S17, the nth photo listed in the photo list field 38 (FIG. 9) is displayed in the display editing area 28 in a fade-in manner. That is, the computer 12 reads the nth photo in the photo list column 38 and the meta information associated therewith from the database 22 (FIG. 1), and displays the nth photo. However, since n = 1 at the beginning, the first photo is displayed in a fade-in manner. Thereafter, after a fixed time pause in step S19, in the next step S21, the computer 12 determines whether or not a region is designated in the nth photo, for example, whether there is meta information of Image Regions. If the region is designated, in the next step S23, the computer 12 increments the region number m (m + 1). In step S25, the meta information of Image Regions of the mth region is referred to. This meta information includes position data and size data of the mth region. Therefore, in the next step S27, the computer 12 uses the meta information and the previously set display area height Sh and width Sw to align the mth region with the center of the area 28. .

  As an example, if the width of the region is w1 and the height is h1, the enlargement ratio is min [Sw / w1, Sh / h1], and the frame is displayed frame by frame until it just fits in the center of the display screen 28. The photographic image is displayed on the screen 28 by moving dX on the horizontal axis and dY on the vertical axis, and enlarged by dZ.

  However, in step S27, the image of the region is not only aligned with the center of the display area, but may be aligned with other positions such as upper left and lower right, for example.

  Thereafter, after the pause in step S29, in the next step S31, the computer 12 determines whether or not there are no remaining regions. That is, since the number of regions can be known from the image region meta information, in step S31, it may be determined whether the result incremented in step S23 is equal to the number of regions.

  If there are remaining regions, in the next step S32, the computer 12 assigns video effects A-1 and A-2 indicated by identifiers RT-1, RT-2 and RT-4 shown in Table 1.

  Specifically, the computer 12 uses a region coordinate value (imgReg: coords) and a region boundary value (imgReg: boundingBox) in the meta information as shown in FIG. The interval and the Y direction (vertical direction) interval are calculated. Then, it is determined whether the inter-region spacing is greater than or less than a predetermined threshold. When either the X direction (horizontal direction) interval or the Y direction (vertical direction) interval is equal to or less than the threshold value, after zooming in on the region including the face of the person in the photo to use the Cambridge effect, Pan to the next region. Conversely, when either the X direction (horizontal direction) interval or the Y direction (vertical direction) interval is equal to or greater than the threshold, the zoomed-up region is faded out and the next region is faded in instead of panning.

  Also, if the subject information indicated by foaf: person shown in FIG. 7 is a subject that is memorable to the viewer, it zooms up only to the region of that subject, and then pans or fades out and fades in according to the inter-region spacing. Perform the process.

  However, the conversion templates exemplified in Table 3 are used in order to automatically give A-1 and A-2 video effects such as identifiers RT-1, -2, and -4 in step S32.

  Then, it returns to previous step S23 and repeats steps S23-S32.

  If “NO” in the step S31, that is, if all the regions of the nth photo in the photo list column 38 have been processed, in the next step S33, the computer 12 displays the nth photo in the photo display editing area. Fade out from 28.

  Subsequently, in step S35, the computer 12 determines whether there is no remaining photo to be processed. That is, since the number of photos in the photo list shown in FIG. 9 is known in advance, in this step S35, it may be determined whether or not the result incremented in step S15 is equal to the number.

  If the photograph still remains, in the next step S36, the computer 12 causes the video effect A-3 indicated by the identifiers RT-3, RT-5, RT-6, RT-7, and RT-8 shown in Table 1. Or B-1 and B-5.

  Specifically, the computer 12 acquires meta information of a color tone of a photograph (not shown in FIG. 7) from the above-described Exif color space information among the meta information, and the color information is obtained from the previous photo. Judge whether the current photo shows a color change such as color in monochrome. If there is the color change information, the computer 12 gives the video effect that the current color photograph is first displayed in monochrome and then gradually changed to color display.

  Further, the computer 12 acquires data of the shooting date (date) from the meta information shown in FIG. 7 and calculates the age difference between the previous photo and the current photo. Then, it is determined whether the age difference is greater than or equal to a predetermined threshold value. When the age difference from the previous photo is less than or equal to the threshold value, the computer 12 gives the video effect of fading in the next photo while fading out the previous photo and overlapping them. When the age difference from the previous photo is equal to or greater than the threshold, the computer 12 gives the video effect that the previous photo is rotated about the vertical axis at the center to transition to the current photo.

  Further, the computer 12 determines whether or not the scenario is consistent with reference to the shooting date and the event (title) in the meta information shown in FIG. For example, it may be determined whether the photos are taken in chronological order, or if they are of the same age, the order of the seasons is followed. When it is determined that the scenario is consistent, a video effect is applied in which the next photo is faded in while the previous photo is faded out, and they are overlapped.

  In addition, when the age difference of the photograph calculated as described above is below a predetermined threshold and is a turning point of the scenario, for example, you have entered a school, married, a child was born, etc. In this case, the computer 12 gives the video effect that the previous photo is rotated about the vertical axis at the center to transition to the current photo.

  However, the conversion template exemplified in Table 3 is used to automatically give the video effect in step S36.

  When it is determined in the previous step S35 that there is a remaining number of photos, the process returns to the previous step S15, and steps S15 to S36 are repeatedly executed. If “NO” in the step S35, the computer 12 stops the BGM in a step S37 and ends.

In this way, a series of video content (recollection video) is generated (rendered) by editing the photographic image data using the meta-information related to the photographic image data and adding the video effect according to step S6, that is, FIG. . When the rendering is completed, the rendering result is stored in the internal memory 14 (FIG. 1) such as the hard disk of the computer 12 in the RVML format described above. Then, it can be played back on another personal computer or the like by converting the RVML into a Flash movie or the like and storing it in another storage medium. That is, a series of video contents is stored in some storage device or storage medium and used for the care of a person with dementia.
[Evaluation experiment]
In order to verify the validity of the annotation ontology and the rendering template, an experiment was conducted to evaluate the difference in impression between the memories video generated according to the rendering template described in Table 3 and those created by the creator. Since the impression evaluation experiment was based on a questionnaire survey, healthy subjects were used as subjects. The outline and results of the evaluation experiment are described below.

  First, 3 creators who have a certain video editing experience, each using about 15 photo materials, and about 5 minutes of recollection video for each one, each theme, Adobe video editing software Premiere (product Name).

  Next, each creator gave meta information such as a photo date, title, and region to the above-described photo using the authoring tool of the embodiment. Each creator was informed in advance that the authoring tool would use a rendering technique such as the Cambridge effect. However, the video rendered based on the meta information that they actually gave is not known until the annotation work is completed. This is to prevent the creator from annotating the rendering effect as much as possible. Then, based on such annotations, a slide show with video effects (video with thought) was created according to the embodiment.

  For comparison, we also prepared a slideshow that was created without applying video effects to the photos used for the memories video.

  Next, these contents were presented to the subject and the impression was evaluated. As the evaluation method, the questionnaire format used in the SD method (Toyohiko Iwashita: Image measurement by the SD method. Kawashima Shoten) was adopted. Specifically, for 22 adjectives that are considered appropriate for impression evaluation of memories video (for example, like, hateful, featured, etc.) I was allowed to evaluate 11 levels of what impressions were obtained. The contents were presented to 21 healthy evaluators in a procedure that considered as little order effect as possible, and evaluated for each content viewing. The results were analyzed using MANOVA (Analysis of Variance: Krzanowski, W. J .: Principles of Multivariate Analysis. Oxford University Press, 1988.) to determine whether there was a significant difference in the impressions of the above three types of video content.

From this impression evaluation experiment, the memory video automatically generated by the authoring tool of the example and that produced by video editing software by creators with video editing experience gave the same impression to the subjects. I understood it.
Evaluation of the effect of the memory video for people with dementia The effect of the memory video automatically generated by the authoring tool of the example on the actual person with dementia was evaluated. It shows that people with dementia tend to be more concentrated and entertained compared to the content.

It is a block diagram which shows the video content production apparatus of one Example of this invention. It is a flowchart which shows the whole operation | movement of FIG. 1 Example. It is an illustration figure which shows an example of GUI of FIG. 1 Example. FIG. 3 is an illustrative view showing one example of GUI display at the time of photo input and meta information registration used in step S1 of FIG. 2; FIG. 5 is an illustrative view showing segmentation in FIG. 4. It is an illustration figure which shows the meta information of the region of FIG. It is an illustration figure which illustrates the relationship between a photograph and meta information. FIG. 5 is an illustrative view showing one example of a GUI during a photo selection and playback order setting operation in step S3 of FIG. 2; FIG. 9 is an illustrative view showing a reproduction order setting area in the GUI of FIG. 8. FIG. 3 is an illustrative view showing a part of GUI at the time of creating a playlist in step S4 of FIG. 2; It is an illustration figure which shows the new play list registration column formed in GUI of FIG. It is an illustration figure which shows a part of play list formed in GUI of FIG. It is an illustration figure which shows an example of the music list used for play list registration. It is an illustration figure which shows an example of the registered play list. It is a flowchart which shows in detail the memory | video video generation operation | movement in FIG.2 S6.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Video content creation apparatus 12 ... Computer 14 ... Internal memory 16 ... Monitor 20 ... Photo data input device 22 ... Database 26 ... GUI
28 ... Photo display edit area 30 ... Thumbnail display area 32, 34 ... Meta information input area 36 ... Playback order setting area 38 ... Photo list field 42 ... Generate button 60 ... Play list

Claims (8)

Photo data input means for inputting photo image data,
Meta information giving means for giving meta information in relation to the photographic image data;
Storage means for storing the photographic image data together with the related meta information;
Search means for searching photographic image data from the storage means using the meta information, and generating means for generating a series of video contents by editing the photographic image data searched by the search means according to meta information,
It said generating means viewed contains a video effect applying means for applying a video effect based on the meta information,
The meta information includes inter-region interval information of a region set in one photo image data,
The video content creation device, wherein the video effect applying unit applies different video effects according to the inter-region interval information . The meta information includes photo information set in each photo image data,
The video content creation apparatus according to claim 1, wherein the video effect applying unit applies a video effect according to the photographic information. The photo information includes date information,
3. The video content creation apparatus according to claim 2 , wherein the video effect applying unit applies different video effects according to the age difference between the photos based on the date information. The photo information includes person information,
4. The video content creation apparatus according to claim 2 , wherein the video effect applying unit applies the video effect according to the person information. The photographic information includes color tone information,
5. The video content creation device according to claim 2 , wherein the video effect applying unit applies a video effect according to the color tone information. In the computer of the video content creation device,
Photo data input step for inputting photo image data,
Meta information giving step for giving meta information in relation to the photographic image data,
Storing the photographic image data together with the associated meta information;
A search step of searching for photographic image data from the storage means using the meta information, and a generation step of generating a series of video content by editing the photographic image data searched by the search means according to the meta information, In the generating step, a video effect applying step for applying a video effect based on the meta information is executed ,
The meta information includes inter-region interval information of a region set in one photo image data,
A video content creation program that applies different video effects according to the inter-region interval information in the video effect applying step . Storage means for storing photographic image data together with related meta information;
Search means for searching photographic image data from the storage means using the meta information; and
Display means for displaying a GUI screen including a thumbnail image display area for displaying thumbnail images according to the photo image data searched by the search means and a playback order setting area having a photo list field;
By dragging and dropping the thumbnail image displayed in the thumbnail image display area to the photo list field, a plurality of photos to be actually used are selected from the searched photo data,
The plurality of playback orders can be changed by changing the list loading order in the photo list column,
Reading means for reading out the photographic image data of the photo mounted in the photo list column and the meta information related thereto from the storage means, and
Generating means for generating a series of video contents by editing the photographic image data read by the reading means according to meta information;
The meta information includes region information of a region set in one photographic image data, and the generation unit includes an alignment unit that aligns an image of the region at a predetermined position according to the region information.
The video content creating apparatus, wherein the generating means includes video effect applying means for applying a video effect based on the region information. Photo data input means for inputting photo image data,
Meta information giving means for giving meta information in relation to the photographic image data;
Storage means for storing the photographic image data together with the related meta information;
  Search means for searching photographic image data from the storage means using the meta information; and
Generating means for generating a series of video content by editing photographic image data searched by the search means according to meta information;
The generating means includes video effect applying means for applying a video effect based on the meta information,
The meta information includes region information of a region set in one photo image data,
The video content creation device, wherein the video effect applying means applies a video effect according to the region information.

Images