JP2010068347A - Image forming apparatus, image forming method and image forming program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image forming apparatus capable of extracting static image data desired by a user and outputting the extracted data as a photo album without performing complicated operation. <P>SOLUTION: The image forming apparatus includes: a creation means for creating static image data from moving image data at a predetermined frame rate; a first receiving means for receiving a detection instruction of a predetermined image from a user; a retrieval means for retrieving still image data coincident with the detection instruction from the still image data; a display means for displaying the still image data coincident with the detection instruction detected by the retrieval means; a selection receiving means for receiving the selection of desired still image data out of the still image data displayed by the display means from the user; and an output means for outputting the still image data selected by the user through the selection receiving means as an album for moving image data. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image forming apparatus, an image forming method, and an image forming program that output a still image cut out from a moving image.

  Conventionally, when creating a photo album from video data shot with a digital camera or the like, you must play the video data, specify the scene the user wants, output this scene as a still image, and edit it as an album. I had to. As a technique for supporting creation of a photo album, for example, Patent Document 1 discloses a technique for displaying a plurality of frame images constituting moving image data and allowing a user to select a desired frame image.

  As a technique for automatically generating a photo album, for example, Patent Document 2 discloses a photo album creation method in which a frame having a target audio is specified from moving image data with audio and extracted as a print candidate.

JP 2006-60648 A JP 2006-333065 A

  In creating an album, there is a desire to extract a scene including a specific person or a characteristic expression such as a smile. However, the number of frames included in the moving image data is enormous, and it takes time for the user to select a desired frame while viewing each frame as described above, and a complicated operation is required. Although Patent Literature 2 discloses a technique for specifying a frame by paying attention to voice, it is difficult to extract a characteristic expression or a scene of a specific person.

  The present invention has been made in view of the above, and an image forming apparatus, an image forming method, and an image forming apparatus that can extract still image data desired by a user and output it as a photo album without performing complicated operations. The purpose is to provide a program.

  In order to solve the above-described problems and achieve the object, the present invention is an image forming apparatus that creates still image data from moving image data at a predetermined frame rate, and created by the creating unit. Storage means for storing the recorded still image data, first reception means for receiving a predetermined image detection instruction from a user, and the first reception means from the still image data stored in the storage means. A search unit that searches for the still image data that matches the received detection instruction; a display unit that displays the one or more still image data that matches the detection instruction detected by the search unit; A selection receiving unit that receives a selection of desired still image data from the one or a plurality of the still image data displayed by the display unit; Characterized by comprising an output means for outputting as an album of the still image data selected by the user with respect to the video data.

  The present invention is also an image forming method executed by an image forming apparatus, the image forming apparatus including a storage unit, and the creation unit creates still image data from the moving image data at a predetermined frame rate. A creation step stored in the storage unit, a first reception unit receiving a first image detection instruction from a user, and a search unit from the still image data stored in the storage unit A search step for searching for the still image data that matches the detection instruction received in the first reception step; and one or a plurality of the still images whose display means matches the detection instruction detected in the search step A display step for displaying data, and a selection receiving means from the user, wherein the one or a plurality of the still image data displayed in the display step A selection receiving step for receiving selection of desired still image data from the output, and an output unit for outputting the still image data selected by the user in the selection receiving step as an album for the moving image data. Features.

  The present invention is also an image forming program to be executed by a computer, the computer comprising storage means, creating still image data at a predetermined frame rate from moving image data, and storing the still image data in the storage means. A creation step, a first reception step for receiving a predetermined image detection instruction from the user, and the detection instruction received in the first reception step from the still image data stored in the storage means. Or a search step for searching a plurality of the still image data, a display step for displaying the one or a plurality of the still image data that matches the detection instruction detected in the search step, and from the user in the display step. Selection acceptance for accepting selection of desired still image data from the displayed still image data And steps, characterized in that to execute an output step of outputting as an album on a computer the still image data selected by the user with respect to the moving picture data in the selection acceptance step.

  According to the present invention, a detection instruction for a predetermined image is received from a user, and still image data that matches the detection instruction is searched. Therefore, a still image having a feature that the user wants to extract can be automatically extracted from moving image data, and easily It is possible to create a photo album.

  Exemplary embodiments of an image forming apparatus, an image forming method, and an image forming program according to the present invention are explained in detail below with reference to the accompanying drawings.

  FIG. 1 is a block diagram illustrating a hardware configuration of an image forming apparatus 100 according to the embodiment. The image forming apparatus 100 includes a CPU 101 for controlling the apparatus, an LCD 102 that is a display device for an operation screen, an LCD controller (LCDC) 103 for controlling the display device, and a ROM 104 that stores a control program and the like. A RAM 105 used as a work memory; a plotter 106 that outputs print data; a plotter controller 107 that controls the plotter; a GPIO 110 that controls the keys 108 and LEDs 109; and a mass storage device that stores the print data 111, a network interface (NW / IF) 112 for connection to a LAN line, and an external storage media I / F 113 such as a USB host and an SD slot.

  The RAM 105 is also used as a temporary storage area for execution of software read from the ROM 104 and work associated therewith. The ROM 104 is an electrically rewritable storage medium that stores software for realizing the functions of the image forming apparatus 100.

  FIG. 2 is a block diagram illustrating a software configuration of the image forming apparatus 100. The software 20 constituting the image forming apparatus 100 is configured to provide a user interface layer that provides a user interface to a user, an application layer that implements each application function, and hardware to control the function. Service layer.

  The user interface layer includes a device UI 201 that provides a user interface to an operation panel of the apparatus main body, and a Web UI 202 such as a Web browser used as a user interface of application software.

  The application layer includes a copy application 211 for realizing the copy function, a fax application 212 for realizing the fax function, a printer application 213 for realizing the printer function, and a scanner application for realizing the scanner function. 214.

  The service layer manages the print service 221 that controls the plotter device, the operation panel service 222 that controls the LCDC 103, the key 108, and the LED 109, the network service 223 that controls the NW / IF 112, the RAM 105, and the mass storage device 111. And a memory service 224.

  FIG. 3 is a block diagram illustrating a functional configuration of an album creation processing unit 300 that performs album creation processing by the image forming apparatus 100. The album creation processing unit 300 includes a moving image data acquisition unit 302, a moving image data storage unit 304, a still image data creation unit 306, a still image data storage unit 308, an instruction reception unit 310, a display unit 312, and an image analysis. A unit 320, a time stamp assigning unit 322, a still image data selecting unit 324, a layout editing unit 326, a comment editing unit 328, an album creating unit 330, and an output unit 332. The instruction receiving unit 310 corresponds to a first receiving unit, a second receiving unit, and a third receiving unit. The image analysis unit 320 corresponds to search means. The time stamp assigning unit 322 corresponds to a date specifying unit and a drawing unit. The layout editing unit 326 corresponds to a size adjusting unit. The comment editing unit 328 corresponds to comment adding means.

  The moving image data acquisition unit 302 acquires moving image data from the outside. The moving image data storage unit 304 stores the moving image data acquired by the moving image data acquisition unit 302. The still image data creation unit 306 creates still image data from the moving image data stored in the moving image data storage unit 304 at a predetermined frame rate. The frame rate can be arbitrarily set by the user. The still image data storage unit 308 stores the still image data created by the still image data creation unit 306. The instruction receiving unit 310 receives various instructions from the user. The display unit 312 displays various information on the LCD 102.

  The image analysis unit 320 performs image analysis on still image data stored in the still image data storage unit 308 in accordance with an instruction from the user acquired via the instruction reception unit 310. Specifically, still image data desired by the user is extracted from the still image data by the feature detection module. Note that the still image data extracted here is displayed on the display unit 312.

  The time stamp adding unit 322 adds a time stamp to the still image data stored in the still image data storage unit 308. Specifically, the number of frames from the first frame in the moving image data corresponding to the still image data to be processed, that is, the number of frames is specified. Then, based on the shooting start date / time of the moving image data, the number of frames of still image data to be processed, and the frame rate, the shooting date / time of still image data to be processed is specified, and this is given as a time stamp. .

  The still image data selection unit 324 selects still image data to be included in the photo album according to the selection instruction from the user acquired via the instruction reception unit 310. The layout editing unit 326 edits the layout in the photo album according to the layout instruction from the user acquired via the instruction receiving unit 310. The comment editing unit 328 performs editing processing for comments to be added to each still image data in accordance with a comment instruction from the user acquired via the instruction receiving unit 310.

  The album creating unit 330 creates a photo album including still image data selected by the still image data selecting unit 324 in accordance with the processing of the layout editing unit 326 and the comment editing unit 328. The output unit 332 outputs the photo album created by the album creation unit 330.

  FIG. 4 is a flowchart showing album creation processing by the album creation processing unit 300. In the album creation process, first, the photo album setting screen 400 shown in FIG. 5 is displayed on the LCD 102 by the display unit 312 (step S100). The photo album setting screen 400 is provided with an edit box 401 for setting a read file and a read execution button 402 for instructing execution of reading.

  On the photo album setting screen 400, when a user inputs a desired movie data file name in the edit box 401 and the read execution button 402 is pressed, the still image data creation unit 306 reads the edit box from the movie data storage unit 304. The moving image data having the file name input in 401 is read to create still image data (step S102). When the still image data is created, the display unit 312 displays the still image data list screen 410 shown in FIG. 6 (step S104).

  The list screen 410 includes a still image display unit 411, a candidate search list box 412, a search button 413, an OK button 414, and a layout setting button 415. The still image display unit 411 displays a list of all still image data included in the moving image data specified on the photo album setting screen 400. In the present embodiment, a list is displayed by the scroll function.

  When the user wants to narrow down still image data to be displayed on the still image display unit 411 (step S106, Yes), the user sets a search condition in the candidate search list box 412, and presses the search button 413.

  When person detection is set as a search condition in the candidate search list box 412, when the search button 413 is pressed, the display unit 312 displays the template registration screen 420 shown in FIG. All the still image data can be displayed on the still image display unit 421 of the template registration screen 420.

  The user operates the image selection button 423 to select still image data on which the face of the person to be specified is displayed, and presses the decision button 424. As a result, the selected still image data is displayed on the still image display unit 421. Further, a frame line 422 is displayed on the still image data displayed on the still image display unit 421.

  When the user adjusts the frame size 425 and the frame position 426 so that the face of the person to be specified falls within the frame line 422 and presses the extraction button 427, the image analysis unit 320 creates a template of the selected person. . Then, the image analysis unit 320 uses the template as a standard pattern, performs pattern matching to determine whether there is a pattern that is the same as or close to the pattern to be extracted in all the still image data, and extracts still image data that matches the search target. .

  The same applies when smile detection is set as a search condition in the candidate search list box 412, and the user searches the template registration screen 420 for a smile scene of a person to be searched, and smiles using the frame line 422. Select a person. Thereby, the image analysis unit 320 creates a smile template for the selected person and performs pattern matching based on the created smile template. Note that, on the template registration screen 420, a characteristic expression other than a smile may be selected and used as a search target. As a result, it is possible to search for still image data of facial expressions according to the user's wishes.

  When range specification is set as a search condition in the candidate search list box 412, when the search button 413 is pressed, the display unit 312 displays a range specification screen 430 shown in FIG. On the range designation screen 430, the user can set search conditions by inputting the date and time of the range to be extracted in the date and time input fields 431 and 432. When the date and time are input and the extraction button 433 is pressed, the time stamp assigning unit 322 adds a time stamp to each still image data, and extracts still image data to which a time stamp matching the search condition is added.

  As described above, when the search is performed according to each search condition and the still image data is narrowed down (step S108), the screen returns to the list screen 410 again, and still image data matching the search condition is displayed. If the still image data is not narrowed down (No at Step S106), the process proceeds to Step S110 without narrowing down.

  Next, the user selects still image data to be included in the album and presses an OK button 414. Thus, the still image data selected by the user is read from the still image data storage unit 308 by the still image data selection unit 324 as data to be included in the photo album (step S110).

  Next, when the user presses a layout setting button 415 on the list screen 410, a layout setting screen 440 shown in FIG. 9 is displayed. The layout editing unit 326 edits the layout according to the input content from the user on the layout setting screen 440 (step S112).

  The user inputs the number of images to be displayed in one page in the input field 441 of the layout setting screen 440. FIG. 10 is a diagram illustrating a layout example. Thus, the number of images is arranged according to the number of images specified on the layout setting screen 440.

  Further, the user sets the presence / absence of a header, a footer, and a page number as additional information for each page in the additional information check fields 442, 443, and 444. Further, input contents to be input to the header and the footer are input to the input fields 448 and 449.

  FIG. 11 is a diagram illustrating a display example of a header and a footer. When at least one of the header and the footer is set, the layout editing unit 326 automatically adjusts the size of the still image data so that the header and the footer do not overlap with an area where the still image data is displayed. Fonts such as colors and character sizes displayed in the header and footer can be set separately. Furthermore, an image may be displayed as a header and a footer.

  Further, the user checks the presence / absence of a time stamp and a comment as additional information for each image in the additional information check fields 445 and 446. When the time stamp is selected, the time stamp giving unit 322 gives the time stamp to the still image data selected by the still image data selecting unit 324. FIG. 12 is a diagram illustrating still image data to which a time stamp is added. In this way, a time stamp is drawn in still image data. The display position of the time stamp may be settable.

  When a comment is selected, when the enter button 450 is pressed on the layout setting screen 440, a comment setting screen 460 shown in FIG. 13 is displayed. The comment editing unit 328 edits the comment according to the input content from the user on the comment setting screen 460 (step S114).

  On the comment setting screen 460, the still image data selected on the list screen 410 is displayed on the image display unit 461. The user operates the image selection button 462 to select still image data to be processed. Furthermore, the user inputs the details of the comment in the check box 463 for presence / absence of comment, the comment position input field 464, and the comment content input field 465, and presses the registration button 466. The comment editing unit 328 registers comments according to these inputs.

  FIG. 14 is a diagram illustrating still image data in which a comment is drawn. When the comment position is outside the still image data such as below or above the still image data, the comment editing unit 328 arranges the comment below or above the still image data as specified. When the display is selected so as to be superimposed on the still image data as the comment position, the comment editing unit 328 arranges the comment on the still image data as shown in FIG. Furthermore, when a person is detected in the target still image data, a comment lead line is drawn from the target person. Furthermore, the face image is recognized, and the position of the comment frame is automatically adjusted so that the comment display does not overlap the face.

  In face recognition, for example, several standard face patterns related to a face are defined. Then, a plurality of standard face patterns defined for one input video are matched. That is, the input image and each standard face pattern are matched one-on-one, and the most accurately matched portion of the input image is determined as a face. A face recognition technique may be used to identify the position of the face and place the comment as a “speech balloon”.

  When registration of comments for all still image data is completed and the user presses the enter button 467, the album creating unit 330 creates a photo album according to the contents designated by the user (step S116). Specifically, the photo album is edited according to the processing of the time stamp assigning unit 322, the layout editing unit 326, and the comment editing unit 328, including the still image data selected by the still image data selection unit 324.

  Next, the display unit 312 displays the print preview screen 470 shown in FIG. 15 (step S118). When the user confirms the photo album 471 displayed on the print preview screen 470 and presses the print button 472, the output unit 332 outputs the photo album created by the album creation unit 330 (step S120). Thus, the album creation process is completed.

  The album creating program executed by the album creating unit of the image forming apparatus according to the embodiment may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. good. Further, the album creating program executed by the image forming apparatus may be provided or distributed via a network such as the Internet. Further, the album creating program according to the present embodiment may be provided by being incorporated in advance in a ROM or the like.

  The album creation program executed by the image forming apparatus 100 according to the present embodiment includes the above-described units (video data acquisition unit, video data storage unit, still image data creation unit, still image data storage unit, instruction reception unit, display) Unit, image analysis unit, time stamp assigning unit, still image data selection unit, layout editing unit, comment editing unit, album creation unit, output unit), and the actual hardware is a CPU (processor ) Reads out the album creation program from the storage medium and executes it to load each of the above units onto the main storage device, and a moving image data acquisition unit, a moving image data storage unit, a still image data generation unit, a still image data storage unit, Instruction receiving unit, display unit, image analyzing unit, time stamp adding unit, still image data selecting unit, layout editing unit, comment editing unit, Album creating section, the output section is adapted to be generated on the main memory.

2 is a block diagram illustrating a hardware configuration of the image forming apparatus 100. FIG. 2 is a block diagram illustrating a software configuration of the image forming apparatus 100. FIG. 3 is a block diagram showing a functional configuration of an album creation processing unit 300. FIG. 5 is a flowchart showing album creation processing by the album creation processing unit 300. It is a figure which shows the album setting screen. It is a figure which shows the list screen. It is a figure which shows the template registration screen. It is a figure which shows the range designation | designated screen 430. It is a figure which shows the layout setting screen 440. FIG. It is a figure which shows the example of a layout. It is a figure which shows the example of a display of a header and a footer. It is a figure which shows the still image data to which the time stamp was provided. It is a figure which shows the comment setting screen. It is a figure which shows the still image data by which the comment was drawn. FIG. 6 is a diagram showing a print preview screen 460.

Explanation of symbols

100 Image forming apparatus 101 CPU
102 LCD
103 LCDC
104 ROM
105 RAM
106 Plotter 107 Plotter Controller 108 Key 109 LED
110 GPIO
111 Mass storage device 112 NW / IF
113 External storage medium 201 Device UI
202 Web UI
211 Copy application 212 Fax application 213 Printer application 214 Scanner application 221 Printing service 222 Operation panel service 223 Network service 224 Memory service 300 Album creation processing unit 302 Movie data acquisition unit 304 Movie data storage unit 306 Still image data creation unit 308 Still image data Storage unit 310 Instruction receiving unit 312 Display unit 320 Image analysis unit 322 Time stamp adding unit 324 Still image data selecting unit 326 Layout editing unit 328 Comment editing unit 330 Album creating unit 332 Output unit

Claims (9)

Creation means for creating still image data from the video data at a predetermined frame rate;
Storage means for storing a plurality of the still image data created by the creation means;
First receiving means for receiving a predetermined image detection instruction from a user;
Search means for searching for the still image data that matches the detection instruction received by the first receiving means from the still image data stored in the storage means;
Display means for displaying one or a plurality of the still image data matching the detection instruction detected by the search means;
Selection accepting means for accepting selection of desired still image data from the one or more still image data displayed by the display means from a user;
An image forming apparatus comprising: output means for outputting the still image data selected by a user through the selection accepting means as an album for the moving image data. The first receiving means receives the detection instruction for instructing to detect a desired person;
The image forming apparatus according to claim 1, wherein the retrieval unit retrieves still image data including the person by pattern matching. The first receiving means receives the detection instruction that instructs to detect a specific facial expression of the person,
The image forming apparatus according to claim 1, wherein the retrieval unit retrieves still image data including the specific facial expression by pattern matching. A second receiving means for receiving an instruction to give a predetermined comment to the still image data from the user;
When the second accepting unit accepts the giving instruction, the comment is added to an area other than the area where the predetermined image is drawn out of the still image data selected by the user via the selection accepting unit. The image forming apparatus according to claim 2, further comprising comment giving means for drawing. The first receiving unit receives the detection instruction that specifies a time range in the moving image data,
When the first receiving unit receives the detection instruction specifying the time range, the moving image creation date and time included in the moving image data and the number of frames from the first frame of each still image data in the moving image data Based on the date and time specifying means for specifying the shooting date and time of each still image data,
The search means searches for still image data included in the time range indicated by the detection instruction based on the shooting date and time of each still image data specified by the date and time specifying means. The image forming apparatus described in 1. Date and time specifying means for specifying the shooting date and time of each still image data based on the movie creation date and time included in the movie data and the number of frames from the first frame of each still image data in the movie data;
Drawing means for drawing the shooting date and time on the still image data;
The image forming apparatus according to claim 1, wherein the output unit outputs the still image data on which the photographing date and time are drawn by the drawing unit as the album. Third receiving means for receiving a layout instruction for assigning characters and images to a print page of still image data;
Size adjusting means for adjusting the image size of the still image data so that the still image data in the print page does not overlap the character or image when the third receiving means receives the layout instruction; The image forming apparatus according to claim 1, further comprising: An image forming method executed by an image forming apparatus,
The image forming apparatus includes a storage unit,
A creating step for creating still image data from the moving image data at a predetermined frame rate and storing the still image data in the storage unit;
A first receiving means for receiving a predetermined image detection instruction from a user;
A search step for searching for the still image data that matches the detection instruction received in the first receiving step from the still image data stored in the storage unit;
A display step for displaying one or a plurality of the still image data matching the detection instruction detected in the search step;
A selection receiving step for receiving a selection of desired still image data from the one or a plurality of the still image data displayed in the display step from a user;
An image forming method comprising: an output unit that outputs the still image data selected by the user in the selection receiving step as an album for the moving image data. An image forming program for causing a computer to execute,
The computer includes storage means,
A creation step of creating still image data from moving image data at a predetermined frame rate and storing the data in a storage means;
A first receiving step for receiving a predetermined image detection instruction from a user;
A search step of searching for the still image data that matches the detection instruction received in the first receiving step from the still image data stored in the storage means;
A display step of displaying one or a plurality of the still image data that matches the detection instruction detected in the search step;
A selection receiving step for receiving a selection of desired still image data from the one or more of the still image data displayed in the display step;
An image forming program for causing a computer to execute an output step of outputting the still image data selected by the user in the selection receiving step as an album for the moving image data.

Images