JP2012249211A - Image file generating device, image file generating program and image file generating method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To extract the frame data suitable for displaying in slide show form from video data and to file the frame data thus extracted.SOLUTION: Frame data is extracted from video data every predetermined time ΔT, by repeating the loop from step S104 to step S108. If the discrepancy rate of the frame data thus extracted and the frame data extracted just before goes above a predetermined value, the frame data extracted in the (n+1)th is stored in a folder creation area. Consequently, plural pieces of frame data extracted from the video data every predetermined time ΔT and different from an image extracted just before are stored in the folder creation area of RAM. After completing the processing from step S104 to step S107 for the full length of the video data, an image file including plural pieces of frame data of comic pictures described later is generated through file generating processing (step S109).

Description

  The present invention relates to an image file generation device, an image file generation program, and an image file generation method, and more particularly to a technique for generating an image file from moving image data.

  In recent years, with the widespread use of digital cameras, it has become common to store photographs as digital image data. In the case of shooting with a conventional film camera, it was often accompanied by printing, but in the case of image data, it can be browsed with a digital camera that has shot, or it can be browsed with a personal computer that captures image data, etc. There are also changes in the way photos are enjoyed.

  In such a situation, so-called digital photo frames are realized and popularized so that image data can be enjoyed in the same way as a conventional print photograph (for example, Patent Document 1).

  Digital photo frames can realize various display modes such as displaying multiple photos in a slideshow format that could not be done with photo frames that display conventional print photos by taking advantage of the characteristics of digital methods, and image processing For example, an invention has been proposed in which an image having a different tone (for example, a painting tone) can be generated and displayed based on an original photograph (for example, a patent) Reference 2).

JP 2009-141678 A JP-A-8-44867

  However, the image displayed in the slide show format in the digital photo frame is an image obtained by taking a still image. Therefore, in order to display a larger number of different images in the slide show format, a large number of still images must be taken, and preparations are complicated.

  Therefore, it is conceivable to use moving image data composed of a plurality of temporally continuous frame data, and display individual frame data constituting the moving image data in a slide show format. In this way, for example, a large amount of frame data can be acquired by shooting a single moving image, so that many different images can be displayed in a slide show format with a simple advance preparation.

  However, in the case of a plurality of frame data captured by moving images, there is no difference between a plurality of temporally continuous frame data, and the same frame data is continuous or a slightly different frame data is continuous. There are many. Therefore, when the individual frame data constituting the moving image data is displayed in the slide show format, images that are the same or only slightly different are sequentially displayed. For this reason, there arises a problem that the user gets tired of viewing the image at an early stage.

  The present invention has been made in view of such conventional problems, and an image file generation apparatus, an image file generation program, and an image file generation program capable of extracting frame data suitable for display in a slide show format from moving image data and creating a file An object is to provide an image file generation method.

  In order to solve the above problems, in the image file generation device according to the present invention, among a plurality of frame data, a plurality of areas of the nth selected frame data and the n + 1th selected frame A plurality of areas of data are compared with each other, and when a predetermined ratio or more in the plurality of areas differs, the n + 1-th selected frame data is extracted.

  Further, in the image file generation program according to the present invention according to the present invention, the computer included in the image file generation device includes a plurality of areas of the frame data selected n-th among a plurality of frame data, and Compared with a plurality of areas of the n + 1-th selected frame data, and when a predetermined ratio or more in the plurality of areas is different, function as means for extracting the n + 1-th selected frame data It is characterized by.

  In the image file generation method according to the present invention according to the present invention, among a plurality of frame data, a plurality of areas of the nth selected frame data and the n + 1th selected frame data And the n + 1-th selected frame data is extracted when a predetermined ratio or more in the plurality of areas is different.

  According to the present invention, frame data having a difference suitable for display in a slide show format can be extracted from moving image data and filed by simple preparation for moving image shooting. Therefore, the interest property with respect to a user's slide show appreciation can be improved by simple prior preparation.

1 is a block diagram illustrating a circuit configuration and a system configuration of an image display apparatus according to an embodiment of the present invention. It is a figure which shows the memory | storage state of a memory card. It is a flowchart which shows the main routine of this Embodiment. It is a schematic diagram of moving image data. It is a figure which shows the comparison method of frame data. It is a flowchart which shows a comic picture conversion interrupt routine. It is a figure which shows an example of the comic picture with which the speech balloon was synthesize | combined.

  Hereinafter, embodiments of the present invention will be described. FIG. 1 is a block diagram showing an electrical configuration of an image display device 1 incorporating an image file generation device according to the present embodiment and an image tone conversion system including the image display device 1. The image display device 1 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12 connected to the CPU 11, a RAM (Random Access memory) 13, and an internal memory 14. The ROM 12 stores a program or the like for causing the CPU 11 to perform an operation shown in a flowchart described later.

  The internal memory 14 is a large-capacity nonvolatile memory such as a hard disk or a flash memory, and can store a new image file created by the processing shown in the flow described later. The display control unit 16 controls the CPU 11 to display the image and various menus on the display panel 3 by driving the display panel 3 made of liquid crystal or the like based on the image data stored in the buffer area of the RAM 13.

  The key input control unit 17 inputs a touch detection signal of the touch panel 5 and an operation signal of the key input unit 21 based on the control of the CPU 11. The touch panel 5 is stacked on the display panel 3, and detects a touch position and generates a touch position detection signal indicating the touch position. In the present embodiment, the key input unit 21 is provided with a power switch, a mode selection SW (switch), a moving image selection SW, and other switches.

  The memory card interface 18 is an input / output interface that controls data input / output between the CPU 11 and various memory cards 60 that are detachably mounted in the memory card slot. For example, moving image data is recorded on the memory card 60 by shooting with a digital camera having a moving image shooting function with sound. Therefore, the moving image data recorded on the memory card 60 is composed of a plurality of frame data having a predetermined frame rate that is temporally continuous and audio data representing audio. The moving image data can express a smooth motion at about 24 fps to 30 fps (Frame Per Second). As described above, for example, 24 to 30 still images are continuously displayed per second to express a moving image. Each still image is called a frame. It is the smallest unit to compose. Therefore, a moving image is expressed by displaying the continuously arranged still images one after another.

  The GPS control unit 20 acquires position information based on information received by the GPS antenna 7. Thereby, the current position of the image display apparatus 1 can also be known. The human sensor 19 is connected to the CPU 11 and detects whether a person is nearby. Therefore, when a person is not in the vicinity for a predetermined time or longer, the power is automatically turned off to save power (auto power off).

  The communication control unit 30 performs communication control including image and mail transmission / reception via the telephone line 31 or the wireless LAN 32. The address book 33 is used for mail transmission / reception, and is actually provided in the internal memory 14.

  The backup server 40 is connected via the network 90 and backs up data recorded in the internal memory 14 automatically or by a manual instruction. The content server 50 has a large number of contents and images, and can distribute data to the image display device 1 via the network 90.

  The imaging device 70 is a so-called digital camera, and includes an imaging device, an imaging control unit that controls the imaging device, an image transmission unit, and the like. In addition, the image capturing device includes a transmission unit that drives the image sensor to capture an image of a color subject at a predetermined period (frame rate) and transmits the captured live view image to the outside. The imaging device 70 is connected to the communication control unit 30 of the image display device 1 via a network 90 via a telephone line 31 or a wireless LAN 32. Therefore, the CPU 11 of the image display device 1 can sequentially capture the live view images captured by the imaging device 70 and transferred by the transmission unit.

  At this time, since the imaging device 70 is arranged at a remote location different from the location of the image display device 1 that the user has, the user can view the scenery of the remote location on the display panel 3 of the image display device 1. can do.

  Further, the power control unit 80 takes in AC power through the power plug 81, converts it into direct current, and supplies power to each unit. The auto power-off control described above is also performed. The RAM 13 is a working memory in which the CPU 11 temporarily stores various data as necessary, and includes a buffer area shown in a flow to be described later.

  FIG. 2 is a diagram showing moving image data stored in the memory card 60. The memory card 60 stores a plurality of moving image data (1) to (M) in advance. The moving image data (1) to (M) are those recorded and recorded by the digital camera as described above, or downloaded from the content server 50 via the network 90.

  Each moving image data (1) to (M) is composed of a plurality of frame data (1) to (N). For example, the moving image data (1) includes frame data (1) (1) having a predetermined frame rate. ) To (1) (N). Audio data 601 is also recorded in association with the frame data (1) (1) to (1) (N).

  Next, the operation of the present embodiment according to the above configuration will be described. When the power switch provided in the key input unit 21 is turned on and the image file generation mode is selected from the menu screen, the CPU 11 uses the RAM 13 as a work area according to the program stored in the ROM 12, and Start control and processing. FIG. 3 is a main flow showing the processing procedure of the CPU 11. First, the CPU 11 displays a list of moving image data stored in the memory card 60 on the display panel 3 (step S101).

  Next, based on the touch position detection by the touch panel 5, it is determined whether or not any one of the moving image data displayed on the display panel 3 has been selected (step S102). If any moving image data is selected, the first frame data of the selected moving image data is selected by the selection unit and stored in the folder creation area of the RAM 13 (step S103).

  That is, as shown in FIG. 4, the moving image data MD is composed of a plurality of frame data F that are continuous according to time t at a predetermined frame rate. The moving image data MD is a moving image shot by the digital camera as described above, and is composed of a plurality of frame data F from the shooting start time point TS to the shooting end time point TE of moving image shooting. In step S103, the selection unit first selects the frame data F1 at the shooting start time TS from the moving image data MD.

  Subsequently, in the moving image data MD, with respect to the frame data selected by the selection unit in step S103, frame data after a predetermined time ΔT has been selected by the selection unit and stored in an area other than the folder creation area of the RAM 13 ( Step S104). As a result, as shown in FIG. 4, the frame data F2 after the elapse of the predetermined time ΔT is selected for the frame data F1.

  Next, the selected frame data F1 and F2 are divided into a plurality of areas, and the colors are compared for each of the divided areas (step S105). That is, as shown in FIGS. 5A and 5B, the frame data F1 and F2 are divided into areas e1 and e2 having the same position and the same size made up of a plurality of independent circles, and the corresponding areas e1 and e2 are divided. Compare colors.

  In the present embodiment, the frame data F1 and F2 are divided into areas e1 and e2 made up of a plurality of independent circles, but the areas e1 and e2 e2 may have any shape.

  Further, although the same areas in the frame data F1 and F2 are compared with each other, the angle of view may be slightly shifted due to camera shake or other causes at the time of moving image shooting. In this case, even if there is almost no change in the content of the image, if the area of the area to be compared is small, there is a high possibility that the rate of change of the color of the image in the area will increase. Therefore, when a person is captured in the frame data, the function of recognizing the person's face is used, and the position of the face is used as a reference and the X-axis direction and the Y-axis direction are separated by the same distance. What is necessary is just to set an area.

  In addition, it is considered that the brightness (not color) is different in all or a large number of areas e1 and e2 due to a sudden change in weather (sunny → cloudy, cloudy → sunny). It is preferable to judge inconsistency by the combination of colors.

  Then, as a result of the comparison in step S105, the comparison unit determines whether or not the colors of the areas of a plurality of areas e1 and e2 are equal to or greater than a predetermined ratio, for example, 20% or more (step S106). If the mismatch rate is 20% or more (step S106; NO), it is determined that the frame data F1 and F2 are different images, and the frame data F2 selected by the extraction unit is stored in the folder creation area of the RAM 13. (Step S107).

  As a method for determining whether or not the colors of these 20% or more areas do not match, the corresponding areas of the plurality of areas e1 and the plurality of areas e2 are respectively compared, and the number of areas whose colors do not match There is a method for determining whether or not there is 20% or more. This compares each area e1 of the frame data F1 with each area e2 of the frame data F2 in a region corresponding to each area e1 of the frame data F1, and for example, as shown in FIG. 4, e1 and e2 When there are 15 each, a comparison is made for each of the 15 and 20% of the 15 are 3, so if the number of areas whose colors do not match is 3 or more, it is determined that they do not match, When the number of colors that do not match is less than 3, it can be determined that there is no mismatch.

  As another method, there is a method of comparing the plurality of areas e1 and the plurality of areas e2 and determining whether or not the colors of 20% or more of the areas are inconsistent. This method compares all the areas e1 of the frame data F1 with all the areas e2 of the frame data F2. For example, as shown in FIG. 4, when there are 15 e1 and e2 respectively, If the total color of the 15 areas of e2 does not match the total color of the 15 areas of e1 is 20% or more, it is determined that there is a mismatch. When the number of colors that do not match is less than 3, it can be determined that there is no mismatch.

  As described above, in the present embodiment, since the determination of the difference between the frame data F1 and F2 is performed based on the colors in the areas e1 and e2, it is possible to determine the difference between the frame data through simple processing without complicated processing. can do.

  Next, it is determined whether or not the processing of step S104 to step S107 described above has been completed for the entire length of the moving image data MD (step S108), and the loop of step S104 to step S108 is repeated until completion.

  Therefore, by repeating the loop of step S104 to step S108, frame data F3, F4... Are selected from the moving image data MD every predetermined time ΔT as shown in FIG. If the discrepancy rate between the selected frame data F3, F4... And the frame data selected immediately before is not less than a predetermined value, the n + 1th selected frame data is stored in the folder creation area of the RAM 13. Stored. Therefore, the folder creation area of the RAM 13 stores a plurality of frame data that is extracted from the moving image data MD every predetermined time ΔT and is different from the image extracted immediately before that.

  Then, when the processing of step S104 to step S107 is completed for the entire length of the moving image data MD, file generation processing is performed to generate an image file including a plurality of later-described comic picture frame data (step S109). . Further, the generated image file is stored in the internal memory 14 (step S110), and the process is terminated.

  FIG. 6 is a flowchart showing a comic picture conversion interrupt routine in the present embodiment. The CPU 11 interrupts the main routine shown in FIG. 3 at regular time intervals, and executes processing according to the comic picture conversion interrupt routine.

  That is, it is determined whether there is frame data newly extracted and stored in the folder creation area of the RAM 13 (step S201). If there is frame data newly extracted and stored in the folder creation area of the RAM 13, the main subject is recognized in the frame data (step S202).

  When recognizing the main subject in step S202, a method of recognizing the main subject on the condition that the object is a certain size or larger in the frame data, or a human subject that is a certain size or larger object Any method such as a method of recognizing a main subject on the condition of the above may be used.

  In the present embodiment, for example, the main subject is recognized on the condition that it is an object of a certain size or larger and is a human subject.

  Subsequently, the face portion in the recognized human subject is recognized (step S203). Further, only the face portion of the recognized human subject is enlarged (step S204), and only the facial portion of the enlarged human subject is converted into a cartoon picture (step S205). For the comic picture image conversion in step S205, for example, a technique described in JP 2010-146264 A can be used. In this technique, a comic picture is more easily and effectively converted by converting it into a comic picture using a contour image that is an image obtained by performing an expansion / contraction process on a binarized image. It can be displayed.

  In addition, since only the face portion is converted, the conversion to the comic picture image can be completed quickly without taking time.

  Note that not only the face portion but also the entire human subject or the entire frame data may be converted into a cartoon picture.

  In the present embodiment, since the face portion is enlarged and converted to a comic picture, the face is similar to an actual comic picture, which is an expression technique for exaggerating and emphasizing the features of the motif. You can get a cartoon picture with deformed parts.

  In addition, you may make it convert into a cartoon picture, without enlarging a face part.

  Next, it is determined whether or not audio data is recorded between the nth selected frame data and the (n + 1) th selected frame data in the moving image data MD (step S206). If recorded, the audio data recorded between the nth selected frame data and the (n + 1) th selected frame data is extracted (step S207). Further, the voice represented by the extracted voice data is converted into text data (step S208). Then, a speech balloon is created, and the character string of the text data converted in step S208 is arranged in the speech balloon, and the speech balloon in which this character string is arranged is synthesized in the n + 1th selected frame data (step S209). ). After that, the process returns to the main routine.

  Therefore, if this timer interrupt routine interrupts and executes the main routine, all the frame data stored in the folder creation area of the RAM 13 in step S103 and step S107 in the flowchart of FIG. . Therefore, the image file generated in step S109 includes a plurality of comic picture frame data, and is composed of sequentially different images.

  Therefore, when the frame data in the image file stored in the internal memory 14 is sequentially read out and displayed in the slide show format, the sequentially displayed images are different, and the user gets bored of viewing the images at an early stage. The problem can be solved.

  In addition, with simple advance preparation of moving image shooting, it is possible to extract frame data having different differences suitable for display in the slide show format from the moving image data and create a file. Therefore, the interest property with respect to a user's slide show appreciation can be improved by simple prior preparation.

  Moreover, in the present embodiment, as shown in FIG. 7, the face portion in the extracted frame data F is converted into a cartoon picture MP, and further, the voice is displayed as a character string C with a speech bubble B. It is possible to improve the interest in viewing slideshows.

  In this embodiment, each frame data is converted into a comic picture and the sound is displayed by a balloon. However, an image file composed of different frame data is generated without performing a comic picture conversion or a balloon display. You may make it do. Even in this case, the frame data having a difference suitable for display in the slide show format can be extracted from the moving image data and made into a file by simple preparation for moving image shooting.

  Hereinafter, the invention described in the scope of claims of the present application will be appended.

<Claim 1>
Among a plurality of frame data, a plurality of areas of the frame data selected nth and a plurality of areas of the frame data selected n + 1 are compared, and a predetermined ratio or more in the plurality of areas If the two are different from each other, the image data generating apparatus is characterized in that the n + 1th selected frame data is extracted.

<Claim 2>
The plurality of frame data is selected by a selection unit that selects the frame data for each predetermined time from moving image data composed of the plurality of frame data continuous in time,
The extraction unit extracts the n + 1-th selected frame data,
The comparison of the plurality of areas is performed for each corresponding area by the comparison unit,
The comparison unit compares colors of a plurality of areas of the nth selected frame data with a plurality of areas of the n + 1th selected frame data. The image file generation device described.

<Claim 3>
The comparison unit compares the corresponding areas of the nth plurality of areas and the (n + 1) th plurality of areas, respectively, and the number of areas whose colors do not match is equal to or greater than the predetermined ratio. Or the whole of the nth plurality of areas and the whole of the (n + 1) th plurality of areas are compared, and whether or not the color of the predetermined percentage or more of the whole of the plurality of areas does not match. 3. The image file generation apparatus according to claim 1, wherein one of the image file generation apparatus and the image file generation apparatus is determined.

<Claim 4>
4. The image file generation device according to claim 1, wherein the predetermined ratio is 20%.

<Claim 5>
A file generation unit that generates an image file including the plurality of frame data extracted by the extraction unit;
A cartoon picture conversion unit that converts the frame data extracted by the extraction unit into a cartoon picture image;
5. The image file generation device according to claim 2, wherein the file generation unit generates an image file including a plurality of comic picture images converted by the comic picture conversion unit.

<Claim 6>
The cartoon picture conversion unit includes a main subject recognition unit that recognizes a main subject in the frame data,
6. The image file generation apparatus according to claim 5, wherein an outline of the main subject recognized by the main subject recognition unit is extracted and only the main subject is converted into a comic picture image.

<Claim 7>
The main subject recognizing unit includes a face recognizing unit that recognizes only a face portion of a human subject as the main subject,
The image file generation device according to claim 6, wherein the comic picture conversion unit converts only a face portion of a human subject recognized by the face recognition unit into a comic picture.

<Claim 8>
8. The image file generating apparatus according to claim 7, wherein the comic picture conversion unit enlarges the face portion of the person subject and converts it into a comic picture.

<Claim 9>
In the case where the moving image data is recorded with audio data together with the plurality of frame data,
An audio data extraction unit that extracts the audio data recorded between the nth frame data selected by the selection unit and the n + 1th frame data;
A character string converter that converts the voice represented by the voice data extracted by the voice data extractor into a character string;
With
9. The comic picture conversion unit synthesizes a balloon including the character string converted by the character string conversion unit into the n + 1-th extracted frame data. The image file generation device described in the item.

<Claim 10>
A computer included in the image file generation device,
Among a plurality of frame data, a plurality of areas of the frame data selected nth and a plurality of areas of the frame data selected n + 1 are compared, and a predetermined ratio or more in the plurality of areas When the two are different, the image file generating program is caused to function as a means for extracting the n + 1-th selected frame data.

<Claim 11>
Among a plurality of frame data, a plurality of areas of the frame data selected nth and a plurality of areas of the frame data selected n + 1 are compared, and a predetermined ratio or more in the plurality of areas If the two are different, the n + 1th selected frame data is extracted.

DESCRIPTION OF SYMBOLS 1 Image display apparatus 3 Display panel 5 Touch panel 11 CPU
12 ROM
13 RAM
14 Internal Memory 16 Display Control Unit 17 Key Input Control Unit 18 Memory Card Interface 19 Human Sensor 20 GPS Control Unit 21 Key Input Unit 30 Communication Control Unit 60 Memory Card 601 Audio Data F Frame Data MD Video Data TE Shooting End Time TS Shooting Start time t

Claims (11)

  Among a plurality of frame data, a plurality of areas of the frame data selected nth and a plurality of areas of the frame data selected n + 1 are compared, and a predetermined ratio or more in the plurality of areas If the two are different from each other, the image data generating apparatus is characterized in that the n + 1th selected frame data is extracted. The plurality of frame data is selected by a selection unit that selects the frame data for each predetermined time from moving image data composed of the plurality of frame data continuous in time,
The extraction unit extracts the n + 1-th selected frame data,
The comparison of the plurality of areas is performed for each corresponding area by the comparison unit,
The comparison unit compares colors of a plurality of areas of the nth selected frame data with a plurality of areas of the n + 1th selected frame data. The image file generation device described.   The comparison unit compares the corresponding areas of the nth plurality of areas and the (n + 1) th plurality of areas, respectively, and the number of areas whose colors do not match is equal to or greater than the predetermined ratio. Or the whole of the nth plurality of areas and the whole of the (n + 1) th plurality of areas are compared, and whether or not the color of the predetermined percentage or more of the whole of the plurality of areas does not match. 3. The image file generation apparatus according to claim 1, wherein the image file generation apparatus determines whether one of   4. The image file generation device according to claim 1, wherein the predetermined ratio is 20%. A file generation unit that generates an image file including the plurality of frame data extracted by the extraction unit;
A cartoon picture conversion unit that converts the frame data extracted by the extraction unit into a cartoon picture image;
5. The image file generation device according to claim 2, wherein the file generation unit generates an image file including a plurality of comic picture images converted by the comic picture conversion unit. The cartoon picture conversion unit includes a main subject recognition unit that recognizes a main subject in the frame data,
6. The image file generation apparatus according to claim 5, wherein an outline of the main subject recognized by the main subject recognition unit is extracted and only the main subject is converted into a comic picture image. The main subject recognizing unit includes a face recognizing unit that recognizes only a face portion of a human subject as the main subject,
The image file generation device according to claim 6, wherein the comic picture conversion unit converts only a face portion of a human subject recognized by the face recognition unit into a comic picture.   8. The image file generating apparatus according to claim 7, wherein the comic picture conversion unit enlarges the face portion of the person subject and converts it into a comic picture. In the case where the moving image data is recorded with audio data together with the plurality of frame data,
An audio data extraction unit that extracts the audio data recorded between the nth frame data selected by the selection unit and the n + 1th frame data;
A character string converter that converts the voice represented by the voice data extracted by the voice data extractor into a character string;
With
9. The comic picture conversion unit synthesizes a balloon including the character string converted by the character string conversion unit into the n + 1-th extracted frame data. The image file generation device described in the item. A computer included in the image file generation device,
Among a plurality of frame data, a plurality of areas of the frame data selected nth and a plurality of areas of the frame data selected n + 1 are compared, and a predetermined ratio or more in the plurality of areas When the two are different, the image file generating program is caused to function as a means for extracting the n + 1-th selected frame data.   Among a plurality of frame data, a plurality of areas of the frame data selected nth and a plurality of areas of the frame data selected n + 1 are compared, and a predetermined ratio or more in the plurality of areas If the two are different, the n + 1th selected frame data is extracted.

Images