JP2006129389A - Video contents creating apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a video contents creating apparatus capable of comparatively easily creating video contents like memory video using photos, in the past, of a patient of dementia. <P>SOLUTION: A video contents creating apparatus 10 includes a computer 12, and the computer 12 registers, on a database 22, photographic image data inputted from a photographic data input device 20 and meta-information that is set in relation to the photographic image data. Photos are retrieved based on the meta-information, photos to be used for the memory video are then selected, and the order of reproducing the photos is determined. A BGM play list is registered, and the photos and BGM are then played back for producing the memory video. At such a time, if there are regions, the regions are displayed on a monitor 16 while being positioned sequentially according to meta-information of the regions. A result of the production is stored in a Flash movie form, for example. Thus, by relating and registering the photos and the meta-information thereof, according to the meta-information, video contents like memory video can be comparatively easily created. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a video content creation device, and more particularly to a novel video content creation device that creates video content such as a memory video using past photographs of persons with dementia.

  With the rapid increase in the elderly population, it can be said that it is an urgent issue to provide full nursing care for people with dementia. However, families who are generally cared for as a caregiver for dementia cannot keep an eye on the dementia for 24 hours. Abuse of dementia due to the stress has recently become a social problem. It has become. In order to reduce the stress of both dementia and caregivers, it is important to bring out psychological stability of the dementia. For this purpose, there is a recollection method that lets you watch old movies, music, and photos of the person. However, it is difficult to do it at home because it requires someone to speak.

For this reason, Non-Patent Document 1 proposes a technique using a memorized video. Memories video is a slideshow video of old photos of people with dementia plus background music and narration.
Yasuda et al .: Created a memorable photo video for people with dementia and evaluated concentration. 28th Annual Meeting of the Society for Higher Brain Dysfunction (2004)

  Although the memorized video has been confirmed to be effective in creating a stable mental state of dementia, the creation of the memorized video requires video editing know-how and is not easy for ordinary caregivers to do. .

  Therefore, a main object of the present invention is to provide a video content creation device capable of creating video content such as a memory video using a past photograph of a demented person relatively easily.

  The invention of claim 1 is a photographic data input means for inputting photographic image data, a meta information giving means for giving meta information related to the photographic image data, and a storage means for storing the photographic image data together with the related meta information. Video content creation comprising: search means for retrieving photographic image data from storage means using meta information; and generation means for generating a series of video content by editing photographic image data retrieved by the search means in accordance with the meta information Device.

  According to the first aspect of the present invention, for example, photographic image data is input to the computer (12) from a photographic data input device (20. Reference numerals indicating parts or elements corresponding to the embodiments; the same applies hereinafter) including an image scanner. . In the computer, means for giving meta information to the photographic image data, for example, Dublin Core, Image Regions, FOAF, etc. are provided. The meta information related to the inputted photographic image data is given by these meta information giving means, and the computer stores the photographic image data in the storage means (22) together with the related meta information. The search means (12, S2) searches the storage means or the photographic image data using the meta information. The generation means (12, S6) generates video content such as a slide show in the AVI format or the Flash movie format, for example, by editing the retrieved photographic image data according to the meta information. If the photo is an old photo of a person with dementia, the video content can be called a memory video.

  In the invention of claim 1, since the photographic image data is edited using the meta information, even if the user is not very familiar with the production of the video content, for example, an old photograph of a person with dementia can be used in a relatively simple manner. Video content such as video can be created.

  According to a second aspect of the present invention, the meta information includes region information of a region set in one photo image data, and the generation unit includes an alignment unit that aligns an image of the region at a predetermined position in accordance with the region information. A video content creation device according to claim 1.

  In the invention of claim 2, when a region is set in one photographic image, the meta information adding means registers the meta information of the region, for example, the coordinate position, height, width, etc. of the region. The alignment means (12, S27) positions the region at the center of the display area, for example, using the meta information such as height and width information. Therefore, the region image is automatically edited at an appropriate position, so that no troublesome operation is required.

  The invention of claim 3 further includes playlist registration means for registering a playlist indicating music, and the generating means includes means for reproducing the music registered in the playlist as BGM according to reproduction of a photograph by photographic image data. Item 3. The video content creation device according to Item 1 or 2.

  In the invention of claim 3, the playlist registration means (12, S4) registers, for example, MP3 music data in the playlist, and the reproduction means (12, S11) uses the music registered in the playlist as a photographic image. Play as BGM along with playback. Therefore, it is possible to automatically add BGM without troublesome operations simply by registering a music piece in a playlist.

  According to the present invention, for example, by adding meta information to a past photo of a person with dementia and editing a photo image using the meta information, video content such as a memory video can be created relatively easily. can do.

  The above object, other objects, features and advantages of the present invention will become more apparent from the following detailed description of embodiments with reference to the drawings.

  With reference to FIG. 1, a video content creation apparatus 10 of this embodiment includes a computer 12. As the computer 12, a personal computer or a workstation can be used.

  The computer 12 is provided with an internal memory 14 such as a hard disk or a RAM. In the internal memory 14, as a tool for inputting meta information, for example, Dublin Core (http: //Doublelincore.org <http://dublincore.org>), Image Regions (http://www.w3.org), FOAF (http://www.foaf-project.org), Jena2 (2.1) ( http://jena.sourceforge.net) and the like are set. All of these function as means for inputting, registering or giving meta information related to photographic image data.

  Here, meta information means structured information regarding data, and in the case of the embodiment, is information that structurally describes details of photographic image data to be handled.

  Dublin Core is known as a representative tool for handling meta information. In addition, a person's face and object area (hereinafter referred to as "region") in a photo is specified and used as meta information. Use Image Regions to hold. If the region is a person, add meta information of the person using FOAF. Regions are used to add effects such as zooming and panning between regions to still photos. Jena2 (2.1) is used to store the meta information in the database in the RDF format.

  Although not shown, the computer 12 includes a graphic board, a processor, a sound board, or a processor, through which a GUI screen and other graphics (video) are displayed on the monitor 16 and from the speaker 18 to the BGM or the like. Audio is output.

  Further, a photographic data input device 20 is connected to the computer 12. The photograph data input device 20 includes at least one of an image scanner, a digital camera (digital camera), the Internet (Web), and the like. The image scanner scans past photographs of persons with dementia and inputs color or monochrome photographic image data. Digital cameras can input photographic image data taken in real time, and can also be used to take past photographs and input those photographic image data. The Internet can be used to input photographic image data of a past photo of a dementia person transmitted from a remote place, and to capture photographic image data of an event related to the past of the dementia person as needed. Still other types of photo data input devices may be used.

  Further, a database 22 is coupled to the computer 12 via an interface 24. In this embodiment, a relational database called PostgreSQL 7.4 (http://www.postgresql.org) is used as the database 22.

  Although not shown, the computer 12 naturally has input means such as a keyboard and a mouse.

  In order to create a memory video using the video content creation apparatus 10 of the embodiment of FIG. 1, the computer 12 and the like operate according to the procedure shown in FIG.

  First, in step S1, photographic image data is input and meta information is registered. Specifically, photographic image data of a past photograph of the demented person is mainly input to the computer 12 using the photographic data input device 20 of FIG.

  At this time, a GUI (Graphical User Interface) screen 26 shown in FIG. The GUI 26 includes a photo display editing area 28 that occupies most of the left side of the monitor screen. The photo display / editing area 28 displays a photo of the photo image data input thereto, and is used for editing work such as designating a region. The GUI 26 includes a thumbnail display area 30 formed on the right side of the monitor screen. A thumbnail image is displayed in the thumbnail display area 30 in order to select a photo to be used from photographic image data that has been input and searched as described below.

  The GUI 26 includes a first meta information input area 32 formed below the monitor screen, a second meta information input area 34 formed substantially at the center of the monitor screen, and a playback order formed at the lower right of the monitor screen. A setting area 36 is provided. In the meta information input area 32, meta information defined in Dublin Core related to all the photographs displayed in the area 28 is input. Further, the meta information input area 34 is an area that appears at the time of input in a pop-up format, and is used to input meta information when the region is a person using FOAF. The playback order setting area 36 is used to set the order in which the photos taken in the memory video are played back, and the order can be changed by drag and drop as will be described later.

  The photo input and meta information registration in step S1 will be specifically described. Referring to FIG. 4, when photographic image data of a photograph (a drawing is a line drawing in the drawing but is actually a photograph. The same applies to other drawings) as shown in FIG. The computer 12 displays the photograph (still image) represented by the image data in the photograph display editing area 28 of the GUI 26 of the monitor 16. At the same time, the computer 12 displays a thumbnail image of the photo in the thumbnail display area 30.

  Then, using a mouse or the like (not shown), if two regions, region 1 and region 2, are set in the photo, region 1 and region 2 are respectively displayed in the photo display editing area 28 as shown in FIG. The specified rectangular frames 29a and 29b are displayed. When such region 1 and region 2 are designated, meta information of each region shown in FIG. 6 is set in a format defined by Image Regions. That is, the coordinates (x11, y11) of the origin of the region 1 (the upper left corner of the rectangular frame) and the diagonal coordinates (x12, y12) thereof are registered, and the height h1 and width w1 of the region 1 are also set as meta. Registered as information. Similarly, for the region 2, the origin coordinates, diagonal coordinates, height, and width are registered as x21, y21, x22, y22, h2, and w2, respectively.

  Thus, if a photograph is input in step S1 and meta information is registered, a database as shown in FIG. 7, for example, is created. In FIG. 7, a photo display editing area 28 shown in FIG. 5 in which an actual photo is displayed and a region is designated is drawn on the right. Then, there is “dc:” in the ellipse, and the data in the rectangle related to it is meta information registered in Dublin Core. For example, “dc: date” has the date “200407716 (July 16, 2004)”, “dc: title” has the name “at Disney Animal Kingdom”, and “dc: description” has the description The sentence "They are very happy to ride the car in the wildlife park" is registered as meta information. In “dc: identifier”, the region number (“1” in the example) is registered, and in “dc: type”, the type of photograph in the region (“person (person)” in the example) is registered.

  The meta information “imgReg:” in the ellipse is meta information when a region is designated by Image Regions. “ImgReg: has region” is meta information indicating that the region is set, “imgReg: Rectangle” indicates that the region is rectangular, “imgReg: regionDepict” is a description of the region, and “imgReg: “BoundingBox” includes the origin position and size (height h, width w) of the region. “ImgReg: coords” is the coordinates of the origin position and the diagonal position of the region. Further, the meta information given by “foaf:” includes the gender (“female (female)” in the example) and name (“Haruka (haruka)” in the example) when the region is a person.

  In addition, each meta information shown in FIG. 7 is shown below, respectively.

xmlns: imgReg = “http://www.w3.org/2004/02/image-regions#”
xmlns: foo == "xmlns.com/foaf/0.1"
xmlns: dc = “http://purl.org/dc/elements/1.1/”
When the photo input and meta information registration are performed in step S1 of FIG. 2, the computer 12 stores the photo and the meta information assigned thereto in the database 22. In the above description, one photo and its meta information have been described. However, every time photographic image data is input by the input device 20, meta information as shown in FIG. Photo data to which meta information is added is stored in the database 22.

  Thereafter, in step S2 in FIG. 2, the photograph used at that time is retrieved from the photographs input in step S1. However, the meta information described above is used in the photo search. For example, the following can be used as search conditions for photographs using meta information.

  First, in FOAF, when the region type is a person, meta information that identifies the person is added, so use this FOAF meta information to search for "photos of a specific person" be able to. A plurality of persons can be searched simultaneously, but in this case, a photograph in which all of the searched names are shown is targeted.

  When using the Dublin Core meta information, you can search by “shooting date”. For example, in “From (first specified date) to To (second specified date)”, it is possible to search for all photos taken after the first specified date and before the second specified date. With “From (specified date)”, you can search all photos taken after the specified date. Similarly, “To (specified date)” can be used to search for all photos taken before the specified date. In addition, it is possible to search for “photos whose specific properties include specific values”. For example, photos with “Disney” in “dc: title”. However, a plurality of images can be selected or searched at the same time, but at the time of selecting a plurality of images, only photos that satisfy the set search conditions are targeted.

  In this way, when the selection of the photos is completed in step S2, next, in step S3, the photos to be actually used are selected from the searched photos, and when a plurality of photos are selected, the reproduction order of the plurality of photos is selected. Specify or determine The GUI 26 for this step S3 is shown in FIG. However, for the sake of convenience, it should be noted that FIG. 8 uses a photo different from the photo before FIG.

  For example, when step S3 is selected by a menu (not shown), the computer 12 displays the GUI 26 shown in FIG. In the GUI 26 of FIG. 8, a reproduction order setting area 36 shown in FIG. 3 is formed at the lower right of the monitor screen. An enlarged view of the reproduction order setting area 36 is shown in FIG.

  The reproduction order setting area 36 includes a photo list field 38, and as shown in FIG. 8, a photo to be used can be selected by dragging and dropping the thumbnail image displayed in the thumbnail display area 30 to the photo list field 38. . In other words, since all thumbnail images of the photos searched in the photo search step of step S2 are displayed in the thumbnail display area 30, by dragging and dropping the photos in the list image to the photo list column 38, the photos can be selected. You can choose. In the photo list column 38, columns of name (name), title (title), and date (date) are set.

  Further, in the playback order setting area 36, as shown in FIG. 9, a sort designation field 40, a generation button 42 for instructing generation of a slide show of a memorable video, a delete button 44 used for deleting a photo, and a selection A preview button 46 to be pressed (clicked) when previewing the photograph taken is provided. When the cursor is placed on any photo in the photo list field 38 and the preview button 46 is operated in this state, a preview image is displayed in the preview area 48 shown in FIG.

  In the sort designation field 40, properties for sorting the selected photos, such as registration date, are set. Then, when a specific property is set in the sort specification field 40 by, for example, a pull-down menu, the computer 12 sorts the photos in ascending order of the value for the property, for example.

  Furthermore, since the images are played back in the order of the photo list registered in the photo list column 38, if you want to change the playback order, drag and drop one line in the photo list column 38 to move the list. What is necessary is just to change order, ie, the order of reproduction | regeneration.

  Subsequently, in step S4 of FIG. 2, a BGM playlist to be played at the time of reproduction of the photo is registered or selected. In this embodiment, for example, an MP3 format music file is supported, and if a music used as a BGM is registered in the playlist, the BGM is reproduced along the playlist. If, for example, the M3U format is used as a playlist, a playlist created with general software such as Winamp (application name) can be used as it is.

  In step S4, first, “play list selection or registration” is selected in a menu (not shown). Then, the GUI 26 shown in FIG. 10 is displayed on the monitor 16. When the playlist creation button 52 of the GUI 26 is clicked, a new playlist registration field 54 shown in FIG. 11 is displayed. In the registration field 54, a name field 56 for setting the name of the playlist (“Sample” in the illustrated example) and a determination (acceptance) button 58 when the input name is acceptable are formed. When a name is entered in the name field 56 and the OK button 58 is clicked, an empty play list 60 shown in FIG. 12 is displayed.

  The MP3 file list 62 shown in FIG. 13 is displayed, and by dragging and dropping an appropriate music piece in the MP3 file list 62 to the play list 60, the music pieces used as BGM can be sequentially selected or registered in the play list 60. . In the example shown in the figure, nursery rhymes are mainly selected using the music list 62 (FIG. 13) called “MY album”.

  In this way, steps such as inputting photo data, registering meta information, searching for photos, selecting, selecting a song are executed, and when the generation button 42 shown in FIG. 9 is pressed, generation of a memory video is started. . Therefore, in step S5 of FIG. 2, it is determined whether or not the generation button 42 has been clicked. If "YES", the process proceeds to the next step S6 as it is, but if "NO", the process can be resumed by returning to any step S1-S4 in FIG.

  That is, each step S1-S4 in FIG. 2 can be executed any number of times at an arbitrary time by selecting from the menu, so that the operator needs as much as necessary when there is time. You just have to do this. In any case, since the result of the previous work is stored in the database 22, in this work, first, the data up to the previous time is read from the database 22, and then the process that continues or changes the process. I do.

  When it is detected in step S5 that the generation button 42 is turned on, in the next step S6, a slide show video (RVML) of a memory video is generated.

  Here, “RVML” is a kind of XML designed so that swf can be completely expressed, and all versions of swf moving images can be expressed as RVML. However, swf is basically only a column of tags excluding a file header having information such as a Flash version and a frame rate. For example, a pattern in which a figure is defined by a definition tag, the figure is arranged in a frame by an operation tag, and the current frame is drawn on the screen by a display tag corresponds to one frame, and this is repeated.

  The generated RVML can create a Flash movie format memory video by using KineticFusion (http://www.kinesissoftware.com), which is a Flash tool, from the RVML.

  Specifically, step S6 of FIG. 2 is executed according to the procedure shown in FIG. In the first step S11, in response to the operation of the generation button 42, the computer 12 first starts playing the music of the playlist 60 (FIG. 14) as BGM. Thereafter, the music is sequentially switched along the playlist 60 and continuously reproduced. That is, the computer 12 reads out music data registered in the playlist 60 and processes the data with a sound board or a processor, so that the music in the playlist is reproduced as sound from the speaker 18 (FIG. 1).

  In the next step S13, the computer 12 sets the width of the photo display editing area 28 (FIG. 3) of the GUI 26 formed on the display screen of the monitor 16 as Sw and the height as Sh.

  In step S15, the computer 12 increments the number of photographs n (n = n + 1). In the next step S17, the nth photo listed in the photo list field 38 (FIG. 9) is displayed in the display editing area 28 in a fade-in manner. That is, the computer 12 reads the nth photo in the photo list column 38 and the meta information associated therewith from the database 22 (FIG. 1), and displays the nth photo. However, since n = 1 at the beginning, the first photo is displayed in a fade-in manner. Thereafter, after a fixed time pause in step S19, in the next step S21, the computer 12 determines whether or not a region is designated in the nth photo, for example, whether there is meta information of Image Regions.

  If the region is designated, in the next step S23, the computer 12 increments the region number m (m + 1). In step S25, the meta information of Image Regions of the mth region is referred to. This meta information includes position data and size data of the mth region. Therefore, in the next step S27, the computer 12 uses the meta information and the previously set display area height Sh and width Sw to align the mth region with the center of the area 28. .

  As an example, if the width of the region is w1 and the height is h1, the enlargement ratio is min [Sw / w1, Sh / h1], and the frame is displayed frame by frame until it just fits in the center of the display screen 28. The photographic image is displayed on the screen 28 by moving dX on the horizontal axis and dY on the vertical axis, and enlarged by dZ.

  However, in step S27, the image of the region is not only aligned with the center of the display area, but may be aligned with other positions such as upper left and lower right, for example.

  Thereafter, after the pause in step S29, in the next step S31, the computer 12 determines whether or not there are no remaining regions. That is, since the number of regions can be known from the image region meta information, in step S31, it may be determined whether the result incremented in step S23 is equal to the number of regions. If there is a remaining region, the process returns to the previous step S23, and steps S23-S31 are repeatedly executed.

  If “NO” in the step S31, that is, if all the regions of the nth photo in the photo list column 38 have been processed, in the next step S33, the computer 12 displays the nth photo in the photo display editing area. Fade out from 28.

  Subsequently, in step S35, the computer 12 determines whether there is no remaining photo to be processed. That is, since the number of photos in the photo list shown in FIG. 9 is known in advance, in this step S35, it may be determined whether or not the result incremented in step S15 is equal to the number. If there is a remaining number of photographs, the process returns to the previous step S15, and steps S15 to S35 are repeated. If “NO” in the step S35, the computer 12 stops the BGM in a step S37 and ends.

In this way, a series of video content (recollection video) is generated (rendered) by editing the photographic image data using the meta information associated therewith according to step S6, ie, FIG. When the rendering is completed, the rendering result is stored in the internal memory 14 (FIG. 1) such as the hard disk of the computer 12 in the RVML format described above. Then, it can be played back on another personal computer or the like by converting the RVML into a Flash movie or the like and storing it in another storage medium. That is, a series of video contents is stored in some storage device or storage medium and used for the care of a person with dementia.
[Evaluation experiment]
The inventors conducted a preliminary evaluation experiment on the effect of the memory video generated by the apparatus 10 of the example in which only the fade-in and the fade-out were implemented with respect to the transition between photographs. A simple profile of the subject is shown in Table 1.

  In the experiment, in addition to the memory video, we prepared four types of video clips: hobbies, songs and news unrelated to the taste of the subject. Each clip was about 7-8 minutes. The subject was photographed with facial expressions from the direction of the monitor screen, and a video of the subject's whole body from behind. Next, the video taken at the time of the experiment was seen, and the degree of the subject's concentration or enjoyment was evaluated in five subjective levels based on the subject's facial expression and body reaction for 1 minute from the start of viewing each clip. The evaluation was performed by 5 people who were not related to the subjects. Table 2 shows the average of the scores of the five people. The degree of concentration and enjoyment by all subjects was higher for memorable videos than for other content.

It is a block diagram which shows the video content production apparatus of one Example of this invention. It is a flowchart which shows the whole operation | movement of FIG. 1 Example. It is an illustration figure which shows an example of GUI of FIG. 1 Example. FIG. 3 is an illustrative view showing one example of GUI display at the time of photo input and meta information registration used in step S1 of FIG. 2; FIG. 5 is an illustrative view showing segmentation in FIG. 4. It is an illustration figure which shows the meta information of the region of FIG. It is an illustration figure which illustrates the relationship between a photograph and meta information. FIG. 5 is an illustrative view showing one example of a GUI during a photo selection and playback order setting operation in step S3 of FIG. 2; FIG. 9 is an illustrative view showing a reproduction order setting area in the GUI of FIG. 8. FIG. 3 is an illustrative view showing a part of GUI at the time of creating a play list in step S4 of FIG. 2; It is an illustration figure which shows the new play list registration column formed in GUI of FIG. It is an illustration figure which shows a part of play list formed in GUI of FIG. It is an illustration figure which shows an example of the music list used for play list registration. It is an illustration figure which shows an example of the registered play list. FIG. 3 is a flowchart showing in detail a memory video generation operation in step S6 of FIG. 2.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Video content creation apparatus 12 ... Computer 14 ... Internal memory 16 ... Monitor 20 ... Photo data input device 22 ... Database 26 ... GUI
28 ... Photo display edit area 30 ... Thumbnail display area 32, 34 ... Meta information input area 36 ... Playback order setting area 38 ... Photo list field 42 ... Generate button 60 ... Play list

Claims (3)

Photo data input means for inputting photo image data,
Meta information giving means for giving meta information in relation to the photographic image data;
Storage means for storing the photographic image data together with the related meta information;
A video comprising: search means for searching for photographic image data from the storage means using the meta information; and generation means for generating a series of video contents by editing the photographic image data searched by the search means according to meta information. Content creation device. The meta information includes region information of a region set in one photo image data,
The video content creation device according to claim 1, wherein the generation unit includes an alignment unit that aligns an image of the region at a predetermined position in accordance with the region information. It further comprises a playlist registration means for registering a playlist showing the music,
The video content creation device according to claim 1 or 2, wherein the generation means includes means for reproducing the music registered in the playlist as BGM in accordance with reproduction of a photograph by the photograph image data.

Images