JP5089641B2 - Video image printing apparatus and video image printing program - Google Patents

Description

  The present invention uses a moving image data and a still image data photographed by a video camera or the like to output on a sheet with high listability and to support access to moving image data based on the output. In particular, the present invention relates to a system that can improve appreciation of output results.

  In general, in order to confirm the content of moving image data, it is necessary to spend the same time as the time of shooting, and to play it back using a playback device such as a video deck, a DVD player, or recently a personal computer. In addition, in order to access a specific scene in moving image data, the time for physical operations such as rewinding and fast-forwarding a videotape has been reduced in recent playback of DVDs and MPEG files. However, it is still necessary to find a position to play back over time.

  As a prior art for such a problem, there is a technique of Patent Document 1. That is, in Patent Document 1, moving image data is analyzed by cut point detection, camera work detection, temporal or frequency feature detection related to sound, and image data that meets a predetermined condition is placed on a sheet and printed. , We provide technology that allows you to easily and quickly grasp the contents of video data.

  Further, Patent Document 1 provides a technique for easily accessing the scene of the moving image data by simultaneously printing with attached information and reading the attached information with a barcode reader or the like. .

  In addition, as an example of the elemental technology that can be used when implementing the present invention, Patent Document 2 discloses position information of a shooting location using an GPS (Global Positioning System) in an imaging device for shooting an image. A technique for acquiring and recording the position information in synchronization with image information is disclosed.

  Similarly, in Patent Document 3, as an example of an elemental technique that can be used when carrying out the present invention, a digest video that includes a scene in a video suitable as a digest scene and that fits within a predetermined time is disclosed. Techniques for creating are disclosed.

  Further, Patent Document 4 discloses a technique for accurately detecting the video time of the moving object up frame in which the moving object is greatly captured and the up shot of the moving object.

Japanese Patent No. 3108585 Japanese Patent Laid-Open No. 7-123348 JP 2006-148397 A JP 2006-244074 A

  However, in Patent Document 1, an image extracted from moving image data is arranged for all the analyzed results with an emphasis on listability, and appreciation of output results is not necessarily considered.

  In general, the quality of still image data extracted from moving image data is difficult in terms of image resolution and the like. As described in the embodiment of Patent Document 1, when printing a larger amount of image data on a limited paper surface, there is no problem even if the resolution is low, but appreciation as a still image is possible. If you try to print as large as possible on the paper to improve the image quality, the roughness of the image quality will be noticeable.

  As for moving image data, generally, moving image data taken by a person who is not a professional photographer generally contains many scenes taken indiscriminately and replays all moving image data. That is, the appreciation becomes low. Further, if these data are stored as they are, there will be a problem from the viewpoint of the capacity of a storage medium such as a hard disk drive (HDD).

  In order to solve the above problems, the present invention prints a representative image representing a digest of a moving image on a paper medium. In particular, as this representative image, an image with high resolution and high appreciation that allows easy understanding of the content of the digest moving image. The print data can be printed, and the digest of the moving image data can be easily accessed from the sheet of the output result.

  In order to solve the above problems, in the present invention, a digest is created from the original moving image data, uploaded to the server, and image data used for output to paper is at the same time or the same as possible. Use still image data taken at a location.

  Specifically, the present invention relates to a moving image data storage unit that stores moving image data, a still image data storage unit that stores a plurality of still image data in which a shooting target of the moving image data is captured, and the moving image Additional information data storage means for storing additional information including moving image data stored in the data storage means and shooting time of the still image data stored in the still image data storage means; and the moving image data storage means A digest moving image is calculated by calculating a feature amount of a moving image from stored moving image data, extracting one or a plurality of characteristic sections of the moving image data based on the calculated feature amount, and connecting them. A digest creation means for creating data, and an access destination to which the digest moving image data can be accessed through a communication line are described. Based on the digest storage means for storing at a position associated with the data, the additional information data read from the additional information storage means, and the section information used for creating the digest of the moving image obtained from the digest creation means, The additional information of the moving image data relating to the section used for the digest creation is compared with the additional information of the still image data, and the still image data representing the digest is selected according to a predetermined comparison result condition, and the representative still image data is selected. Prints together representative image creation means to be image data, representative still image data created by the representative image creation means, and data describing the access destination of the digest moving image data stored by the digest storage means And a printing unit.

  When a representative image representing a digest of a moving image is printed on a paper medium, since the resolution of the moving image is generally low, in the present invention, still images obtained by capturing the same object are used. As a result, it is possible to obtain a print result with a high appreciation property that makes it easy to grasp the contents of the digest.

  Further, in the above invention, the representative image creation means selects one frame image from the moving image data when the still image data corresponding to the condition of the comparison result does not exist, It has the means to do, It is characterized by the above-mentioned. As a result, even when there is no appropriate still image data that matches a predetermined condition, it is possible to print an image representing a digest scene of a moving image.

  In the above invention, the printing means converts the data describing the access destination into a barcode, and the character string of the data describing the access destination alone or together with the barcode is used as the representative still image. A means for printing together with the data is provided. This makes it possible to easily access the digest moving image data from a portable terminal with a barcode reader equipped with a camera or the like.

  In the above invention, the condition of the comparison result used by the representative image creating means for selecting the still image data is that the difference between the shooting time of the specific scene of the moving image data and the shooting time of the still image data is the smallest. Or a condition that the threshold value is smaller than a predetermined threshold value or both. By selecting a still image at a shooting time close to the digest moving image section as a representative image, it is possible to print an appropriate still image representing the contents of the digest moving image section.

  Furthermore, in the above invention, the additional information stored in the additional information storage means includes shooting position information of moving image data stored in the moving image data storage means and still image data stored in the still image data storage means. The comparison result condition including the shooting position information of the image data and used by the representative image creating means for selecting the still image data is that the shooting position of the specific scene of the moving image data and the shooting position of the still image data are Including a condition that the difference is minimum or less than a predetermined threshold value, or both. As a result, a more appropriate still image representing the contents of the digest moving image section can be selected as the representative image.

  Further, in the above invention, the digest accumulation means includes a moving image file for each section and a moving image file obtained by connecting the characteristic files extracted from the moving image data by the digest creating means. Are individually stored so as to be associated with data describing different access destinations. As a result, the user who refers to the digest moving image data can easily access the individual digest section or the entire digest.

  According to the present invention, it is possible to easily access a digest moving image of moving image data from a sheet while improving appreciation as a result of output on the sheet. Note that the image data output on the paper does not necessarily exist as one frame of a digest moving image that can be accessed and reproduced, but an image that is very closely related in time or position is selected. Therefore, there is also an effect of preventing the user from feeling uncomfortable.

  Furthermore, by using digest data, the required media capacity can be reduced without degrading appreciation.

It is a block diagram which shows the outline | summary of this invention. It is explanatory drawing of image data and additional information. It is a figure which shows the example of additional information. It is a figure which shows the example which accumulate | stores the moving image of each area of a digest, and the moving image which connected them as another file, respectively. It is a process flowchart of a representative image creation part. It is a process flowchart of a representative image creation part. It is a flowchart of an imaging time calculation process. It is a flowchart of still image file name acquisition processing (part 1). It is a flowchart of still image file name acquisition processing (part 2). It is a figure which shows the example of a print.

  FIG. 1 is a block diagram showing an outline of the present invention. In FIG. 1, reference numeral 1 denotes a video image printing apparatus constituted by a computer having a CPU and a memory. Reference numeral 2 denotes a camera such as a digital camera or digital video camera, 3 denotes an output result on the paper surface by the video image printing apparatus 1, and 4 denotes a mobile phone with a barcode reader. Although FIG. 1 also shows a camera 2, a mobile phone 4 and the like used by a user of the system according to the present invention, these are not essential components for the present invention.

  Hereinafter, embodiments will be described according to an example in which the present invention is realized by a dedicated software program mainly using a computer.

  The video image printing apparatus 1 includes a moving image data storage unit 10 that stores moving image data, a still image data storage unit 11 that stores still image data, and an additional information data storage unit 12 that stores additional information of moving images and still images. , A data retrieval unit 13 capable of retrieving additional information and still image data, a digest creating unit 14 for creating a digest of a moving image, and representing a digest by using section information used for creating a digest from the original moving image data A representative image creation unit 15 for acquiring still image data suitable for the purpose, a digest storage unit 16 for storing the created digest moving image data so as to be accessible through a communication line, and a digest data storage for storing the stored digest moving image data Unit 17, representative still image data and digest video data The access destination describing the data, and a printing unit 18 that together print.

  In recent years, the name of the camera 2 that is generally marketed as a digital camera or digital video camera changes depending on whether it is mainly for taking still images or moving images, but it can actually take both still images and moving images. It has a function. In general, still image data can be captured with higher definition than moving image data. Captured still images and moving image data are stored in a memory card or a disk device.

  Here, an embodiment will be described that uses image data captured by a digital camera that saves a captured moving image data as an MPEG file and a captured still image as a JPEG file in a memory card. In this case, the shooting time can be acquired as a time stamp of these files by a program on the computer constituting the video image printing apparatus 1. That is, for an MPEG file (moving image data), it is considered that the creation time of the file indicates the date and time when the shooting of the moving image is started, and the last update time of the file indicates the date and time when shooting of the moving image is completed. For files (still image files), the two usually point to the same time, which is considered to indicate the date and time when the still image was taken.

  In the case of using such photographed image data, the data search unit 13 first displays input data as pre-processing when data is taken into the video image printing apparatus 1 from a memory card or the like. 2. Convert to file and search table data as shown in 2. That is, for MPEG data, a plurality of moving images as shown in FIG. 2A are combined in time order so as to become one moving image as shown in FIG. File name: Mall.mpg), and table data of the moving image additional information 121 that can search the actual shooting time for each scene as shown in FIG. 2B from the time stamp of the original MPEG file. create. The JPEG data is copied as it is, and the table data of the still image additional information 122 that can search the actual shooting time for each still image data as shown in FIG.

  By creating such table data, it is possible to provide a function to search shooting time information as additional information for each scene of moving image data, or to search still image data having shooting time data closest to a specified time. The additional information search function such as searching for still image data that falls within the range of the specified time and the specified threshold number of seconds can be easily realized.

  Further, as another embodiment, it is possible to use a technique as described in Patent Document 2 (imaging device with a position measurement mechanism). For such cameras, moving image data and still image data can be obtained in the same manner as in the above example, and in addition, for a JPEG file (still image data), the latitude and longitude indicating the shooting position. For the MPEG file (moving image data), the latitude and longitude can be acquired with sufficient frequency while the moving image is captured.

  With respect to data from such a camera, the data search unit 13 performs the above-described processing as a pre-processing and also converts the positional information to create a table of additional information as shown in FIG. It is possible. That is, for the moving image additional information 121 shown in FIG. 3A, a table of moving image additional information 121 ′ with position information as shown in FIG. 3B can be created. Further, a table of still image additional information 122 ′ with position information as shown in FIG. 3C can be created. In this case, in addition to the functions described above, an additional information search function such as searching for still image data having position information for each scene of moving image data or position information included in a specified range can be easily realized. be able to.

  About the digest production | generation part 14, it can implement by using the method of patent document 3, for example. That is, by using the program described in Patent Document 3 for the acquired MPEG data (combined in the above embodiment), several digest scene sections are automatically extracted, and these are extracted. It is possible to create a digest video that joins together. In the method described in Patent Document 3, since it is known which sections of the moving image data are joined in the middle of the processing, the representative image creation unit 15 (to be described later) uses the section information (usually a plurality of sections). It is assumed that data is written in a memory area or a temporary file on the video image printing apparatus 1 so that it can be acquired.

  The digest storage unit 16 can be implemented using a Web server that is generally widely implemented. That is, by saving the created digest moving image data as a file (digest data storage unit 17) in a folder published as a Web server, an Internet terminal device including a mobile phone or Web browser software can receive URL and By specifying a called access destination character string, it is possible to reproduce and view the digest moving image data stored in the digest data storage unit 17.

  In addition, when storing the digest moving images, one moving image file is provided for each section of the digest scene extracted by the digest creating unit 14, and the whole of them is combined into one moving image file. The embodiment of accumulating separately from the one is preferred.

  FIG. 4 shows an example in which the moving images in each section of the digest and the moving images obtained by connecting them are stored as separate files. As shown in FIG. 4A, when moving images of the respective sections D01 to D06 are extracted as digests from the original moving image data, these are respectively converted into D01. mpg to D06. As a moving image file separately stored as a moving image file of mpg, a moving image file obtained by connecting these files is called Dall. Store as mpg. By doing so, the user can access individual digest sections or the entire digest by properly using URLs.

  Furthermore, using a widely known HTML technology, a moving image file is not simply placed on a Web server and directly accessed, but an HTML file corresponding to each moving image file is embedded in the inside, Preparing and accumulating HTML with links to “previous digest scene”, “next digest scene”, “overall digest scene”, etc. is possible with navigation via the mobile phone or web browser software that has been accessed. Is preferable in that it becomes possible.

  In this case, the required storage capacity will increase, but in general, when creating a digest, both the moving image file for each scene of the digest and the connected file are stored in order to make it less than half of the original video. Even so, it can be made smaller than the original moving image file. Furthermore, since an HTML file is generally much smaller than a moving image file, it is not a problem in capacity to create such an HTML file.

  The digest accumulating unit 16 uses the URL for accessing the digest moving image data (or the HTML file corresponding to the digest moving image data) accumulated in this way so that the print unit 18 described later can acquire the video image. It is assumed that data is written in a memory area on the printing apparatus 1 or a temporary file.

  The representative image creation unit 15 can be implemented by a module that executes the processing flow of FIG. 5 or 6 (internally calls a subroutine for executing the processing of FIGS. 7, 8, and 9).

First, the processing performed by the representative image creation unit 15 will be described with reference to FIG. The representative image creating unit 15 acquires a plurality of pieces of section information used by the digest creating unit 14 for creating the digest (step S10). Start sec t _s of the acquired segment, the end seconds and t _e, each section [t _s, t _e] for each repeat the following steps S12 to S15.

First, the photographing time calculation process described later (see FIG. 7), start second t _s of the section, the ends of seconds t _e, respectively photographing time T _s, converted to T _e (step S12). Subsequently, still image file name acquisition processing (see FIGS. 8 and 9) is performed using the shooting times T _s and _Te as arguments, and the still image file name F is acquired as the return value of these subroutines (step S13). ). Next, it is determined whether or not the still image file name F has been obtained (whether or not F is null) (step S14). If not, the process returns to the caller as it is, and the file name F is obtained. The file name F is notified to the print unit 18 as still image information selected in the section [t _s , t _e ] (step S15). Then return to the caller.

  In the case of the processing flow of FIG. 5, when a still image cannot be selected in the processing of step S13, the digest scene is not printed by the printing unit 18 described later. Therefore, the processing of the representative image creating unit 15 improved in this respect is a processing flow shown in FIG. In FIG. 6, the process from step S10 to S15 is the same as the flow shown in FIG. In the case of the processing flow of FIG. 6, when the file name F is null in the determination in step S14, the still image data to be replaced in step S16 is cut out from the moving image data, and the file name of the still image is F. , F are notified to the print unit 18. By doing so, the replaced image is printed by the printing unit 18 described later.

For the selection of still image data to be replaced, a method using the image of the first frame (T _s second frame) of the digest scene, a method using the image of the last frame (T _e second frame), an intermediate frame ((T _s + T _e ) / 2 second frame). Alternatively, as a more preferable example, the technique of Patent Document 4 is used, and the moving object is greatly captured from the moving image data of the section [T _s , _Te ] (synonymously described as video in Patent Document 4). It is also possible to carry out the method by extracting a moving object up frame and cutting out a still image of the extracted frame.

FIG. 7 is a process called in step S12 of FIGS. 5 and 6, and shows a flow of a process for calculating the photographing time of the number of seconds specified by the argument: t (t _s or t _e ). When the number of seconds t is designated, first, a record satisfying “scene start second” ≦ t <“scene end second” is searched from the moving image additional information 121 shown in FIG. 3A (step S20). The search scene start of result record s t _s a scene start photographing time T _shot, to calculate the photographing time T by the following equation (step S21).

Shooting time T = T _shot + (t−t _s )
For the still image file name acquisition process of FIGS. 8 and 9 called from the processing flow of FIG. 5 or FIG. 6, the threshold value process and the minimum value process for the time difference are performed based on the temporal additional information in FIG. An example is described, and FIG. 9 shows an example in which a threshold value process and a minimum value process regarding a time difference are performed based on additional positional information. With respect to these two examples, it is generally possible to set D2> D1 for the thresholds D1 and D2 relating to time, and by doing so, for the user of the present invention, “when using location information, Even a still image having a large time difference is guaranteed to be a still image taken at a position close to each other, so that it can be viewed as a representative image without a sense of incongruity.

The still image file name acquisition process (part 1) shown in FIG. 8 is as follows. First, as the threshold value D1, from the still image additional information 122 shown in FIG.
T _s −D1 ≦ “shooting time” ≦ T _e + D1
A record satisfying the condition is obtained (step S30). Next, the following processing is performed according to the number of search records (step S31).

When the number of search records is 0, null is set as the return value and the process returns to the caller. If the number of search records is 1, the “file name” of the record is set to F (step S32). Thereafter, F is returned to the caller with a return value. If the number of search records is 2 or more, "photographing time" and T _s in the search record, the magnitude relation between T _e, first calculates the diff as follows.
・ When “shooting time” <T _s ...... diff = T _s − “shooting time”
When T _s ≦ “shooting time” ≦ T _e diff = 0
・ When T _e <“shooting time” ....... diff = “shooting time” −T _e
The record having the minimum diff value thus obtained is selected (step S33).

  The “file name” of the selected record is set to F (step S34). Thereafter, F is returned to the caller with a return value.

The still image file name acquisition process (part 2) shown in FIG. 9 is as follows. First, find the record with the "Frame Seconds" closest to T _s from 3 moving picture additional information shown in (B) (position information) 121 'acquires the "latitude", "longitude", respectively S _lat , S _lon (step S40).

Next, as time threshold value D2, latitude / longitude threshold values D _lat and D _lon , from still image additional information (with position information) 122 ′ in FIG.
T _s −D2 ≦ “shooting time” ≦ T _e + D2
S _lat −D _lat ≦ “latitude” ≦ S _lat + D _lat
S _lon −D _lon ≦ “longitude” ≦ S _lon + D _lon
A record satisfying the condition is obtained (step S41).

Next, the following processing is performed according to the number of search records (step S42). When the number of search records is 0, null is set as the return value and the process returns to the caller. If the number of search records is 1, the “file name” of the record is set to F (step S43). Thereafter, F is returned to the caller with a return value. When the number of search records is 2 or more, the distance between the (S _lat , S _lon ) point and the “latitude / longitude” point of the still image additional information (with position information) 122 ′ in FIG. Is selected (step S44). The “file name” of the selected record is set to F (step S45). Thereafter, F is returned to the caller with a return value.

  Note that the processing method for selecting a still image using additional information is not necessarily limited to the processing shown in FIGS. 8 and 9, and application by other methods is also possible.

Regarding the print unit 18, the still image data representing the section [T _s , _Te ] notified from the representative image creating unit 15 and the digest video of the section written in the memory area or temporary file by the digest storage unit 16 The URL for accessing the image is acquired, and printing is performed as shown in FIG.

  In FIG. 10, 3 represents the output result on the paper surface by printing, 51 represents the printed representative still image, 52 represents the URL, 53 represents the barcode, and 54 represents the binder binding hole. As shown as bar code 53 in FIG. 10, URL 52 is printed not only by directly printing a character string but also by adding a bar code format that can be distinguished by many mobile phones with a camera (bar code reader function) at present. May be. Furthermore, the example of FIG. 10 shows an example in which a page is printed on a page for each section, and the page printed on the original moving image data is fastened with a binder or the like to form a booklet. Such an embodiment is suitable as a video print with high appreciation, which is an effect of the present invention, but the present technology is applicable without being limited to such a printing method.

  After the series of processing is completed, if necessary, the original moving image data and still image data read by the data search unit 13 and the table data created by the preprocessing of the data search unit 13 are deleted. Also good. This also has an effect on the problem of not increasing the storage media capacity of a disk device (HDD) or the like as much as possible.

  The video image printing process described above can be realized by a computer and a software program, and the program can be recorded on a computer-readable recording medium or provided through a network.

DESCRIPTION OF SYMBOLS 1 Video image printing apparatus 2 Camera 3 Output result on paper 4 Mobile phone with barcode reader 10 Moving image data storage unit 11 Still image data storage unit 12 Additional information data storage unit 13 Data search unit 14 Digest creation unit 15 Representative image creation Section 16 Digest accumulation section 17 Digest data storage section 18 Print section

Claims (7)

Moving image data storage means for storing moving image data;
A still image data storage means for storing a plurality of still image data in which a shooting target of the moving image data is shot;
Additional information data storage means for storing additional information including moving image data stored in the moving image data storage means and shooting time of still image data stored in the still image data storage means;
A feature amount of the moving image is calculated from the moving image data stored in the moving image data storage means, and one or more characteristic sections of the moving image data are extracted based on the calculated feature amount, and these are extracted. Digest creation means for creating digest video data by joining together;
Digest storage means for storing the digest moving image data at a position associated with data describing an access destination accessible through a communication line;
Based on the additional information data read from the additional information storage means and the section information used for moving picture digest creation obtained from the digest creating means, additional information of moving picture data relating to the section used for the digest creation; Representative image creation means for comparing the additional information of the still image data and selecting still image data representing the digest according to a predetermined comparison result condition as representative still image data;
Printing means for printing together the representative still image data created by the representative image creating means and the data describing the access destination of the digest moving image data accumulated by the digest accumulation means;
A video image printing apparatus comprising: The video image printing apparatus according to claim 1,
The representative image creating means includes means for selecting one frame image from the moving image data to be representative still image data when there is no still image data that satisfies the comparison result condition. A video image printing apparatus. The video image printing apparatus according to claim 1 or 2,
Means for converting the data describing the access destination into a bar code, and printing the character string of the data describing the access destination together with the bar code together with the representative still image data; A video image printing apparatus comprising: The video image printing apparatus according to claim 1, 2 or 3,
The condition of the comparison result used by the representative image creating means for selecting the still image data is that the difference between the shooting time of the specific scene of the moving image data and the shooting time of the still image data is the smallest or predetermined. A video image printing apparatus characterized by including a condition that the threshold value is less than a specified threshold value or both. The video image printing apparatus according to any one of claims 1 to 4, wherein:
The additional information stored in the additional information storage means includes shooting position information of moving image data stored in the moving image data storage means and shooting position information of still image data stored in the still image data storage means. Including
The condition of the comparison result used by the representative image creating means for selecting still image data is that the difference between the shooting position of a specific scene of the moving image data and the shooting position of the still image data is the smallest or predetermined. A video image printing apparatus characterized by including a condition that the threshold value is less than a specified threshold value or both. In the video image printing device according to any one of claims 1 to 5,
The digest storage means accesses the moving image file for each section and the moving image file obtained by connecting them separately for the characteristic sections extracted from the moving image data by the digest creation means. A video image printing apparatus characterized in that it is stored separately so as to be associated with the data described above.   7. The moving image data storage means, the still image data storage means, the additional information data storage means, and the digest creation means provided in the video image printing apparatus according to any one of claims 1 to 6. , A video image print program for functioning as the digest storage means, the representative image creation means, and the print means.

Images