JP2017108326A - Information processing device, control method thereof, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a mechanism for causing a user to easily confirm a frame by displaying the frame in which sound is reproduced during photographing a moving image in an identifiable manner.SOLUTION: Sound is reproduced during photographing a moving image. Frames of the photographed moving image are arranged in time series, and displayed in a time line format. A section in which the sound is reproduced on a time line is specified by specifying frames corresponding to a time when the sound reproduction is started and a time when the sound reproduction is ended. Specified sound reproduction sections and sections with no sound reproduced are displayed in an identifiable manner so that a user can easily confirm frames of the sections in which the sound is reproduced in the photographed moving image.SELECTED DRAWING: Figure 11

Description

  The present invention relates to an information processing apparatus capable of discriminatingly displaying a frame in which sound is reproduced during moving image shooting and allowing a user to easily confirm the frame, a control method thereof, and a program.

  Conventionally, there are many photographing applications for photographing and editing pictures and moving images. The user uses this shooting application to generate image data by performing shooting while using an effect that is not found in a normal camera or by processing a shot image.

  Among the captured images, there are images that cannot be corrected by later processing unless they are devised at the time of shooting. For example, when the subject to be photographed is an animal such as a dog or a cat or a baby, since the consciousness that the subject is photographed is not possible, the direction of the photographing camera may not be pointed. If the subject's figure or facial expression cannot be photographed well at the time of shooting, the subsequent processing cannot be done and the image must be taken again.

  Therefore, a method is conceivable in which a moving image is taken for a certain period of time, a user selects an arbitrary moment from the moving image, and cut out as a still image. Capturing a still image from a series of video images allows you to shoot a more natural appearance of the subject that is the subject than when shooting a single still image, or you can choose the moment you like I can do it.

  Moreover, if the frames which comprise a moving image are arranged in time series and displayed like a film roll in that case, the user can grasp | ascertain the content, without reproducing | regenerating a moving image.

  Patent Document 1 discloses a mechanism for recording various point information while shooting a moving image, extracting an image of a point determined to be an important scene from the point information, displaying it on a film roll, and reproducing the moving image. Thereby, the user can preferentially confirm the image at the moment determined to be important in advance from the captured moving images. The various types of point information indicate points at which the user has performed a zoom operation during moving image shooting, points at which the sound level input from the microphone has peaked, points at which the current time is well cut, and the like.

JP 2007-134771 A

  However, there is no guarantee that an image extracted in advance from specific information is an image requested by the user. For example, as an important scene determination method, an important scene is a timing at which the user moves the photographing device using an acceleration sensor provided in the photographing device, or a moment when a date changes using time lapse information. There is a way. However, if the subject cannot capture this intention, such as animals or babies, even if the user moves the photographic device during shooting, it often does not move as the user thinks, and the subject points in the desired direction. Not necessarily. For humans, the moment from New Year's Eve to the New Year is an important scene. However, since babies and animals cannot understand the time-lapse situation, they cannot affect the subject and cannot become an important scene. As described above, the user's shooting operation and external factors do not affect some subjects and may be meaningless.

  Further, as in Patent Document 1, by arranging the frames constituting the frame moving image in time series and displaying like a film roll, the user can grasp the contents of the rough moving image without playing. However, if you want to check any scene or frame that you could shoot accidentally while shooting, it is difficult to specify the detailed frame in the film roll format display, and eventually play a video or check consecutive frames There is no choice but to search and it is inconvenient.

  SUMMARY OF THE INVENTION An object of the present invention is to provide a mechanism that allows a user to easily identify a frame in which sound is reproduced during moving image shooting so that the user can easily confirm the frame.

  In order to achieve the above object, an information processing apparatus of the present invention is an information processing apparatus including a photographing device capable of photographing a moving image, and a sound reproducing unit that reproduces sound during moving image photographing by the photographing device; A display means for displaying a frame of a moving picture taken by the photographing apparatus in a timeline format; and a frame specifying means for specifying the frame corresponding to a time when the sound is reproduced by the sound reproducing means. The means displays the frame specified by the frame specifying means so as to be distinguishable from the frame not specified by the frame specifying means.

  According to the present invention, it is possible to display a frame in which a sound is reproduced during shooting of a moving image so as to be identifiable and allow the user to easily confirm the frame.

2 is a diagram illustrating a hardware configuration of the information processing apparatus 100. FIG. 2 is a diagram illustrating a functional configuration of the information processing apparatus 100. FIG. It is a flowchart which shows the flow of a series of processes in this embodiment. It is a figure which shows the example of a screen structure before video recording. It is a figure which shows the table structure of 500 of a sound reproduction time recording table. FIG. 4 is a continuation of the process of FIG. 3 and is a flowchart showing a flow of a series of processes in the present embodiment. It is a figure which shows the example of a screen structure after video recording. It is a figure which shows the table structure of the frame imaging | photography time recording table. FIG. 7 is a continuation of the process of FIG. 6 and is a flowchart showing a flow of a series of processes in the present embodiment. It is a figure which shows the table structure of the sound button table. It is a figure which shows an example which identified and displayed the frame of the moving image displayed on the timeline. It is a figure which shows an example which cuts out a still image from the moving image displayed on the timeline. It is a flowchart which shows the flow of a process of a sound button setting. It is a figure which shows the example of a screen structure of a setting screen. It is a figure which shows the example of a screen which determines the sound corresponding to a sound button. It is a figure which shows the example of a screen which determines the color corresponding to a sound. 5 is a diagram showing a table configuration of a sound button table 1700. FIG. 5 is a diagram showing a table configuration of a sound color table 1800. FIG. FIG. 7 is a continuation of the process of FIG. 6 and is a flowchart showing a flow of a series of processes in the present embodiment.

  Hereinafter, a first embodiment of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a configuration diagram illustrating a hardware configuration of the information processing apparatus 100 according to the first embodiment of the present invention. Note that the hardware configuration shown in FIG. 1 is merely an example.

  The information processing apparatus 100 is an apparatus that includes a touch panel. In the present embodiment, description will be made assuming a device such as a so-called smartphone or tablet terminal, but the present invention is not limited to this. A personal computer may be used as long as the device includes a touch panel.

  The CPU 101 comprehensively controls the memory and the controller via the system bus. The ROM 102 or the flash memory 114 stores a basic input / output system (BIOS) that is a control program of the CPU 101 and an operating system. Furthermore, various programs described below that are necessary for realizing the functions executed by the information processing apparatus 100 are stored.

  The RAM 103 functions as a main memory, work area, and the like for the CPU 101. The CPU 101 implements various operations by loading a program or the like necessary for execution of processing into the RAM 103 and executing the program.

  The display controller 104 controls display on a display device such as the display 110 (display unit). The display 110 is a liquid crystal display, for example. A touch panel 111 is provided on the surface of the display 110. The touch panel controller 105 controls detection of a touch operation on the touch panel 111. The touch panel controller 105 can also detect touch operations (hereinafter referred to as multi-touch) on a plurality of locations on the touch panel 111.

  The camera controller 106 controls photographing with the camera 112. The camera 112 is a digital camera, and an image photographed under the control of the camera controller 106 is converted into digital data by a photographing element. The camera 112 can capture still images and moving images.

  The sensor controller 107 controls input from various sensors 113 included in the information processing apparatus 100. Various sensors exist in the sensor 113 of the information processing apparatus 100, such as an orientation sensor and an acceleration sensor.

  The network controller 108 is connected to and communicates with an external device via a network, and executes communication control processing on the network. For example, Internet communication using TCP / IP is possible.

  The flash memory controller 109 controls access to the flash memory 114 that stores a boot program, browser software, various applications, font data, user files, editing files, various data, and the like. In this embodiment, the flash memory is described, but a storage medium such as a hard disk, a flexible disk, or a card-type memory connected to a PCMCIA card slot via an adapter may be used.

  The CPU 101, each memory, and each controller described above are integrated into one chip 115. The information processing apparatus 100 is provided in the form of a so-called SoC (System on Chip).

  For example, the CPU 101 can perform display on the display 110 by executing outline font rasterization processing on a display information area in the RAM 103.

  Various programs and the like used by the information processing apparatus 100 of the present invention to execute various processes described later are recorded in the flash memory 114, and are executed by the CPU 101 by being loaded into the RAM 103 as necessary. is there. Further, definition files and various information tables used by the program according to the present invention are stored in the flash memory 114. The above is the hardware configuration of the information processing apparatus 100 in the present embodiment.

  FIG. 2 is a diagram illustrating a functional configuration of the information processing apparatus 100. Note that the functional configuration shown in FIG. 2 is merely an example.

  As functional units of the information processing apparatus 100, a storage unit 201, a display control unit 202, an operation detection unit 203, a shooting control unit 204, and a sound reproduction unit 205 are provided. These are functional units that operate in the operating system 200.

  The storage unit 201 is a functional unit for storing, in the RAM 103 and the flash memory 114, information constituting various tables described later, various screens, information constituting image data generated by photographing with the camera 112, and the like. Add, update, and delete information as necessary. The same operation is performed when there is a request from the photographing application 210 described later.

  The display control unit 202 is a functional unit for controlling display on the display 110. When a display instruction such as a screen is issued from the display control unit 202, the display instruction is transmitted to the display 110 via the display controller 104. Upon receiving this instruction, the display 110 displays a screen or the like according to the instruction (display means).

  The operation detection unit 203 is a functional unit for detecting a touch operation on the touch panel 111. The position of the touch operation on the touch panel 111, the number of touch operations, and the like can be detected. In addition, the type of touch operation such as whether the touch operation has been performed once on the touch panel 111 or whether the user has slid while performing the touch operation (hereinafter referred to as a slide operation) can also be detected.

  The photographing control unit 204 is a functional unit for controlling photographing with the camera 112. When a shooting instruction is issued from the shooting control unit 204, the shooting instruction is transmitted to the camera 112 via the camera controller 106. When the camera 112 receives this instruction, it starts shooting according to the instruction.

  The sound reproduction unit 205 is a functional unit for reproducing sound in response to an instruction from a sound control unit 215 described later. When a sound reproduction instruction is issued from the sound control unit 215, the sound reproduction unit 205 receives the sound reproduction instruction. Then, the sound reproduction unit 205 reproduces the sound according to the instruction (sound reproduction means).

  The information processing apparatus 100 is installed with a shooting application 210 that can operate in the operating system 200. The shooting application 210 includes a moving image data control unit 211, a time storage unit 212, an operation acquisition unit 213, a sound storage unit 214, a sound control unit 215, and a still image acquisition unit 216.

  The moving image data control unit 211 is a functional unit for executing processing on moving image data stored in the storage unit 201. The moving image data control unit 211 can acquire and save moving image data.

  The time storage unit 212 is a functional unit for storing the length of time of a moving image shot by the shooting application 210, the time of sound playback controlled by the sound playback unit 205, and the like. The time storage unit 212 passes the time stored in the time storage unit 212 to the storage unit 201 for storage.

  The operation acquisition unit 213 is a functional unit for acquiring the content of the touch operation detected by the operation detection unit 203. In the photographing application 210, each functional unit executes photographing control, screen display control, and the like according to the content of the touch operation passed from the operation detection unit 203 to the operation acquisition unit 213.

  The sound storage unit 214 is a functional unit for storing sounds controlled by the sound control unit 215 and information corresponding to each sound. The sound storage unit 214 stores the sound acquired from the storage unit 201 and identification form information corresponding to each sound, and performs addition / change / deletion of the sound as necessary (storage unit).

  The sound control unit 215 is a functional unit for controlling the sound stored in the sound storage unit 214. A reproduction instruction is issued to the sound reproduction unit 205 to reproduce the sound selected by the user.

  The still image acquisition unit 216 is a functional unit for acquiring a still image from the moving image controlled by the moving image data control unit 211. In the present embodiment, the moving image data control unit 211 displays moving image data frames in a timeline format, and based on the user's selection detected by the operation detection unit 203, the still image acquisition unit 216 sets any frame as a still image. get. The functional configuration in the present embodiment has been described above.

  Next, a flow of a series of processing in the present embodiment will be described with reference to the flowcharts of FIGS.

  In step S301, the CPU 101 of the information processing apparatus 100 reads out the program of the photographing application 210 from the flash memory 114 according to an instruction from the user, and activates it.

  In step S <b> 302, the CPU 101 of the information processing apparatus 100 detects a touch operation from the user by the function of the operation acquisition unit 213, and determines whether the moving image shooting button is pressed. If it is determined that the shooting button has been pressed, the process proceeds to step S303. If it is not determined that the shooting button has been pressed, the processing in step S302 is repeated until the pressing of the shooting button is detected. The shooting button will be described later.

  In step S <b> 303, the CPU 101 of the information processing apparatus 100 starts moving image shooting by the shooting control unit 204 in response to the pressing of the moving image shooting button from the user acquired in step S <b> 302.

  In step S304, the CPU 101 of the information processing apparatus 100 detects a touch operation from the user by the function of the operation acquisition unit 213, and determines whether or not the sound button is pressed. If it is determined that the sound button has been pressed, the process proceeds to step S304. If it is not determined that the sound button has been pressed, the process proceeds to step S309. The sound button will be described later.

  FIG. 4 is a diagram showing an example of the screen configuration of the shooting screen of the shooting application 210. The shooting screen includes a shooting button 401, a sound button 402, a sound button 403, and a sound button 404. The shooting button 401 is a button for starting shooting. When the shooting button 401 is pressed again during shooting, shooting is finished. That is, it functions as a shooting end button during shooting. The sound button 402, the sound button 403, and the sound button 404 are buttons for reproducing sound. Different sounds are set for the sound buttons 402 to 404, respectively, and the sound set for the sound button that has been pressed is reproduced in response to a touch operation from the user. Playing a sound while shooting a movie produces an effect that attracts the interest of the subject. Note that the number of sound buttons is not limited to three, and may be one or more.

  In step S305, the CPU 101 of the information processing apparatus 100 plays a sound in response to pressing of the sound button detected in step S304. More specifically, the record corresponding to the sound button whose touch operation is detected by the operation acquisition unit 213 is specified from the sound button table 1000. Then, the sound corresponding to the pressed sound button is reproduced by the sound reproduction unit 205.

  In step S <b> 306, the CPU 101 of the information processing apparatus 100 records the time when the sound reproduction unit 205 starts sound reproduction by the function of the time storage unit 212. More specifically, the operation acquisition unit 213 acquires which sound button the user has pressed. The acquired sound button ID and the sound playback start time calculated after recording the moving image are written in the sound playback time recording table 500 (see FIG. 5). The sound playback time recording table 500 will be described later.

  The sound playback time recording table 500 is a data table for recording the playback start time and playback end time of a sound when the sound is played back during moving image shooting. The sound reproduction time recording table 500 includes a button ID 501, a sound reproduction start time 502, and a sound reproduction end time 503. The button ID 501 indicates unique identification information assigned to each sound button stored in the storage unit 201, and the ID of the sound button pressed by the user is written in the table every time the button is pressed. The sound reproduction start time 502 indicates the time when the sound starts to be reproduced. The sound reproduction end time 503 indicates the time when the sound has been reproduced. When the sound starts or ends, the time is written in this data table. The recording time is calculated from the time when the moving image starts to be recorded. In other words, the elapsed time from when the record button is pressed until the sound button is pressed is recorded as the sound playback start time, and the sound playback end time is also elapsed from when the record button is pressed until the sound finishes playing Record the time.

  In step S <b> 307, the CPU 101 of the information processing apparatus 100 ends the reproduction of the sound reproduced by the sound reproduction unit 205 by the sound control unit 215. The sound reproduction may be terminated after all sound data has been reproduced, or may be terminated in the middle of reproduction upon detection of a user's sound button press.

  In step S308, the CPU 101 of the information processing apparatus 100 records the playback end time of the sound played back by the sound playback unit 205 by the function of the time storage unit 212. More specifically, the reproduction end time when the sound reproduction ends in step S307 is written in the sound reproduction time recording table 500.

  In step S309, the CPU 101 of the information processing apparatus 100 acquires a touch operation from the user by the function of the operation acquisition unit 213, and determines whether or not the shooting end button has been pressed. If it is determined that the shooting end button has been pressed, the process proceeds to step S309. If it is not determined that the shooting end button has been pressed, the process returns to step S304.

  In step S310, when the CPU 101 of the information processing apparatus 100 determines that the shooting end button has been pressed in step S309, the shooting of the moving image ends.

  The description will be moved to FIG. In step S601, the CPU 101 of the information processing apparatus 100 displays the captured moving image in a timeline format by using the functions of the moving image data control unit 211 and the display control unit 202. More specifically, the frames of the captured moving images are displayed in time series.

  FIG. 7 is an example of a screen configuration before reproduction of a moving image including the moving image display in the timeline format displayed in step S601. A moving image playback button 701 and a timeline display 702 are provided. When the movie playback button 701 is pressed, the movie shot in step S303 can be played. At that time, by displaying the timeline display 702 at the same time, the moving image can be reproduced while checking the frames constituting the moving image. The screen is merely an example of a moving image display method, and the moving image may be displayed on the timeline without reproducing the moving image.

  In step S <b> 602, the CPU 101 of the information processing apparatus 100 acquires the sound reproduction start time by the function of the time storage unit 212. More specifically, the sound reproduction start time 502 of the sound reproduction time recording table 500 is used to specify the time when the sound has started reproduction.

  In step S603, the CPU 101 of the information processing apparatus 100 identifies the frame shot at the sound playback start time by comparing the sound playback start time recorded in step S306 with the frame shooting time. More specifically, the time is collated using the sound reproduction start time 502 of the sound reproduction time recording table 500 and the frame shooting time recording table 800 (see FIG. 8). Frame information taken at the same time as the sound reproduction start time is acquired, and a frame ID is specified (frame specifying means).

  The frame shooting time record table 800 is a data table that records the time at which a frame was shot. The frame shooting time recording table 800 includes a frame ID 801 and a frame shooting time 802. The frame ID 801 indicates unique identification information assigned to each frame stored in the moving image data control unit 211. The frame shooting time 802 indicates the time when the frame was shot. When a moving image is shot, the times when the frames constituting the moving image are shot are written in the frame shooting time recording table 800 one after another.

  In step S <b> 604, the CPU 101 of the information processing apparatus 100 determines whether or not the frame at the start of sound reproduction specified in step S <b> 603 is displayed on the screen in a timeline display by the function of the display control unit 202. The number of frames constituting the moving image data is very large. When frames are arranged in time series and displayed in a timeline format, it may not be possible to arrange all the frames on the screen. In that case, the timeline is displayed with some frames removed. Therefore, it is determined whether or not the display control unit 202 controls display of the frame at the start of sound reproduction. If it is determined that the frame at the start of sound reproduction is displayed on the timeline, the process proceeds to step S606. If it is not determined that the frame at the start of sound reproduction is displayed on the timeline on the screen, the process proceeds to step S605.

  In step S605, the CPU 101 of the information processing apparatus 100 is photographed at a time closest to the photographing time of the specified frame among the frames whose time series is later than the frame specified in step S603, and is displayed on the timeline. The frame is specified using the frame shooting time recording table 800.

  In step S606, the CPU 101 of the information processing apparatus 100 uses the frame at the start of sound playback specified in step S603 or the timeline display frame taken immediately after the start of sound playback specified in step S605 as the identification display start frame. To decide.

  In step S <b> 607, the CPU 101 of the information processing apparatus 100 acquires the sound reproduction end time by the function of the time storage unit 212. More specifically, the sound reproduction end time 503 in the sound reproduction time recording table 500 is used to specify the time when the sound has finished reproduction.

  In step S608, the CPU 101 of the information processing apparatus 100 identifies the frame shot at the sound playback end time by comparing the sound playback end time recorded in step S308 with the frame shooting time. More specifically, the time is collated using the sound reproduction end time 503 of the sound reproduction time recording table 500 and the frame shooting time recording table 800. Frame information taken at the same time as the sound reproduction end time is acquired, and a frame ID is specified (frame specifying means).

  In step S609, the CPU 101 of the information processing apparatus 100 determines whether or not the display control unit 202 controls display of a frame at the end of sound reproduction. If it is determined that the frame at the end of sound reproduction is displayed on the timeline, the process proceeds to step S611. If it is not determined that the frame at the end of sound playback is displayed on the screen as a timeline, the process proceeds to step S610.

  In step S610, the CPU 101 of the information processing apparatus 100 is photographed at a time closest to the photographing time of the identified frame among the frames whose time series is later than the frame identified in step S608, and is displayed on the timeline. The frame is specified using the frame shooting time recording table 800.

  In step S611, the CPU 101 of the information processing apparatus 100 uses the frame at the end of sound playback specified in step S608 or the timeline display frame taken immediately after the end of sound playback specified in step S605 as the identification display end frame. To decide.

  The description is moved to FIG. In step S901, the CPU 101 of the information processing apparatus 100 specifies the button ID 1001 corresponding to the sound button pressed in step S304 using the sound button table 1000 (see FIG. 10).

  The sound button table 1000 is a data table in which data of sound buttons arranged on the shooting screen of the shooting application 210 is recorded. The sound button table 1000 includes a button ID 1001, a color 1002, a sound 1003, and a file 1004. A button ID 1001 indicates unique identification information assigned to each button stored in the storage unit 201. A color 1002 indicates color information (hereinafter referred to as identification color information) corresponding to each button of the button ID 1001. A sound 1003 indicates sound information corresponding to each button of the button ID 1001. A file 1004 indicates a storage location of the sound file indicated by the sound 1003.

  In step S902, the CPU 101 of the information processing apparatus 100 acquires identification color information from the color 1002 corresponding to the button ID 1001 specified in step S901.

  In step S903, the CPU 101 of the information processing apparatus 100 identifies and displays the frames arranged in the section between the identification start frame determined in step S606 and the identification end frame determined in step S611 in the color acquired in step S902. To do. As a result, the section in which the sound is played back (hereinafter referred to as the sound playback section) can be identified as the section in which the sound is not played back, and by identifying the color associated with the button, which button is pressed This frame can also be displayed in an identifiable manner.

  In step S904, the CPU 101 of the information processing apparatus 100 determines whether there is a sound playback section that is not identified and displayed in addition to the section identified and displayed in step S903. If it is determined that there is a sound playback section that is not identified and displayed, the process returns to step S602, and the process proceeds to identify and display the sound playback section that is not identified and displayed. If it is not determined that there is a sound playback section that is not identified and displayed, the process proceeds to step S905.

  In step S <b> 905, the CPU 101 of the information processing apparatus 100 determines whether a frame designation is received by a touch operation or a slide operation from the user on the timeline display of the moving image, using the function of the operation acquisition unit 213. If it is determined that the designation of a frame has been received from the user, the process proceeds to step S906. If it is not determined that the frame designation has been received, the series of processing ends (frame selection means).

  In step S <b> 906, the CPU 101 of the information processing apparatus 100 acquires a touch operation from the user using the function of the operation acquisition unit 213, and displays the frame that has received the designation on the display screen as a still image using the function of the display control unit 202.

  In step S907, the CPU 101 of the information processing apparatus 100 determines whether the frame displayed in step S906 has been accepted from the user by pressing a save button on the screen. There is a save button on the same screen, and it may be pressed, or the image may be saved by an operation of the user pressing and holding the image for a long time. If it is determined that a frame save instruction has been received, the process advances to step S908. If it is not determined that a frame save instruction has been received, the process returns to step S905 (save means).

  In step S908, the CPU 101 of the information processing apparatus 100 stores the frame for which the storage instruction has been received in step S907 as a still image by the still image acquisition unit 216. Thus, by displaying the sound reproduction section as the section where the sound is not reproduced, it is possible to confirm at which point in time the sound is flowing without reproducing the moving image. In addition, it makes it easier to find an image of the moment when the subject is affected by the sound, such as when the subject is interested in the played sound and turns to the camera, and there are multiple sound buttons. In this case, it is possible to confirm which button was pressed and the subject's reaction was strong. In addition, by extracting a certain amount of important scenes from the video called “sound playback section”, it is possible to select a user's favorite frame as a still image from the identification section by slide operation or touch operation on the timeline. Have.

  FIG. 11 is a diagram of a timeline identification display 1100 in which three different sound buttons are pressed during video recording, and then video recording is terminated and video frames are displayed in a timeline format. In this figure, each sound playback section is identified and displayed in a color corresponding to the pressed button. The identification display 1101 indicates a sound playback section in which the sound button 402 in FIG. 4 is pressed. The identification display 1102 indicates a sound playback section in which the sound button 403 in FIG. 4 is pressed. The identification display 1103 indicates a sound playback section in which the sound button 404 in FIG. 4 is pressed. In the present embodiment, the upper and lower sides of the frame are identified by being colored, but the photograph itself may be colored or may be identified by a mark. As long as it corresponds to a button or sound, the identification method may color any part of the frame or may not be a color.

  FIG. 12 is a diagram illustrating an example when an arbitrary frame is selected from a moving image displayed in a timeline and the sound reproduction section is identified and stored as a still image. An arbitrary frame is selected from the identified and displayed frames by a touch operation or a slide operation. The selected figure is displayed on the screen as a still image and can be confirmed. If the user finds an arbitrary still image, the user presses the save button 1202 to save the still image separately from the moving image.

  Next, a second embodiment will be described. In the first embodiment, the process of identifying and displaying the sound playback section by the identification color corresponding to the sound button pressed by the user is performed. However, in the second embodiment, it corresponds to an arbitrary sound set by the user. A process for identifying and displaying the sound reproduction section by the identified identification color will be described.

  Hereinafter, the second embodiment will be described. However, the hardware configuration and the functional configuration in the second embodiment are the same as those in the first embodiment, and the description thereof will be omitted.

  Next, the flow of processing in which the user sets an arbitrary sound and identification color for the sound button will be described with reference to the flowchart of FIG.

  First, in step S <b> 1301, the CPU 101 of the information processing apparatus 100 activates a setting program for the photographing application 210 in accordance with an instruction from the user.

  The setting screen 1400 (see FIG. 14) is a diagram illustrating an example of a screen for setting a volume and sound selection at the time of sound reproduction of the photographing application 210, a timer setting time for automatically reproducing the sound, and the like. The setting screen 1400 includes a sound button 1401, a sound button 1402, and a sound button 1403. When the user presses a sound button arranged on the setting screen 1400, each setting of the sound button can be changed. The flow of setting sound buttons will be described below in steps S1302 to S1306.

  In step S <b> 1302, the CPU 101 of the information processing apparatus 100 accepts pressing of a sound button from the user on the setting screen 1400.

  In step S1303, the CPU 101 of the information processing apparatus 100 accepts selection of a sound corresponding to the sound button pressed by the user. More specifically, selection of an arbitrary sound is accepted from the sounds stored in the sound storage unit 214.

  FIG. 15 is a diagram in which the user selects a sound corresponding to the sound button 1401. When the user presses the sound button 1401 on the setting screen 1400, the display control unit 202 displays a list of sounds stored in the sound storage unit 214 on the sound selection screen 1500. The user selects an arbitrary sound, that is, a cat sound as shown by 1501 in FIG.

  In step S1304, the CPU 101 of the information processing apparatus 100 accepts selection of an identification color selected by the user.

  In step S <b> 1305, the CPU 101 of the information processing apparatus 100 determines whether the setting registration button has been pressed by the function of the operation acquisition unit 213. If it is determined that the setting registration button has been pressed, the process proceeds to step S1306. If it is not determined that the setting registration button has been pressed, the setting is not saved and the setting is not changed, and the series of sound button setting processing ends.

  In step S <b> 1306, when the CPU 101 of the information processing apparatus 100 accepts pressing of the setting registration button from the user in step S <b> 1305, the CPU 101 saves the setting change for the sound button in the storage unit 201. More specifically, the sound that has been selected in step S1302, the file information that is the storage location of the sound, and the sound button information are written in the sound button table 1700 (see FIG. 17). The sound button table 1700 will be described later. Also, the identification color information that has been selected in step S1304 is written into the sound color table 1800 (see FIG. 18). The sound color table 1800 will be described later.

  FIG. 16 is a diagram illustrating an example of selecting an identification color corresponding to the sound that has been selected in FIG. When the color selection button 1601 is pressed, a list of identification colors that can be selected in a pull-down format is displayed. Note that the identification color selection method of this embodiment is an example, and the display method at the time of color selection is not limited to the pull-down format. When the color selection is completed and the user presses an OK button 1602 that is a setting registration button, the settings for the sound are stored in the storage unit 201.

  The sound button table 1700 is a table for recording a sound corresponding to the sound button received on the setting screen 1400. The sound button table 1700 includes a button ID 1701, a sound 1702, and a file 1703. The button ID 1701 indicates unique identification information assigned to each button stored in the storage unit 201. A sound 1702 indicates the file name and file type of the sound set in the button ID 1701. A file 1703 shows information on the storage location of the sound described in the sound 1702.

  The sound color table 1800 is a table for recording identification color information set for a sound. The sound color table 1800 includes a sound 1801, a color 1802, and a sound file 1803. The sound 1801 indicates the sound name of the sound stored in the sound storage unit 214. A color 1802 indicates an identification color associated with the sound 1801. A sound file 1803 indicates the file name and file type of the sound 1801. The color selected by the user on the identification color selection screen in FIG.

  Next, the flow of the identification display processing of the sound reproduction section of the moving image displayed on the timeline in the present embodiment will be described with reference to the flowchart of FIG. FIG. 19 is a modification of FIG. 9 in the first embodiment. Therefore, the flowchart of FIG. 19 is a continuation from FIG. 3 and FIG. 6 of the first embodiment.

  In step S1901, the CPU 101 of the information processing apparatus 100 specifies the button ID 1701 corresponding to the sound button pressed in step S304 using the sound button table 1700.

  In step S1902, the CPU 101 of the information processing apparatus 100 specifies the sound 1702 corresponding to the button ID 1701 specified in step S1901 using the sound button table 1700.

  In step S1903, the CPU 101 of the information processing apparatus 100 acquires the identification color corresponding to the sound acquired in step S1902 from the color 1802 of the sound color table 1800.

  In step S1904, the CPU 101 of the information processing apparatus 100 identifies and displays the frames arranged in the section between the identification start frame determined in step S606 and the identification end frame determined in step S611 in the color acquired in step S1903. To do. This makes it possible to identify the section where the sound is played as the section where the sound is not played, and by identifying the color associated with the sound, it is also possible to identify which sound is the frame when it was played. Can be displayed.

  The processing from step S1905 to step S1909 is the same as the processing from step 904 to step 908 in FIG.

  In this way, by setting the identification color for each sound and identifying and displaying the sound playback section with the identification color selected by the user, it is possible to check which sound the subject is most interested in and responded to. Can be made easier.

  As described above, according to the present embodiment, it is possible to display the frame in which the sound is reproduced during the shooting of the moving image in an identifiable manner so that the user can easily confirm the frame.

  The present invention can be implemented as a system, apparatus, method, program, storage medium, or the like, and can be applied to a system including a plurality of devices. You may apply to the apparatus which consists of one apparatus.

  Note that the present invention includes a software program that implements the functions of the above-described embodiments directly or remotely from a system or apparatus. The present invention also includes a case where the object is achieved by the computer of the system or apparatus reading out and executing the supplied program code.

  Accordingly, since the functions of the present invention are implemented by computer, the program code installed in the computer also implements the present invention. That is, the present invention also includes a computer program for realizing the functional processing of the present invention.

  In that case, as long as it has the function of a program, it may be in the form of object code, a program executed by an interpreter, script data supplied to the OS, and the like.

  Examples of the recording medium for supplying the program include a flexible disk, hard disk, optical disk, magneto-optical disk, MO, CD-ROM, CD-R, and CD-RW. In addition, there are magnetic tape, nonvolatile memory card, ROM, DVD (DVD-ROM, DVD-R), and the like.

  As another program supply method, a browser on a client computer is used to connect to an Internet home page. The computer program itself of the present invention or a compressed file including an automatic installation function can be downloaded from the homepage by downloading it to a recording medium such as a hard disk.

  It can also be realized by dividing the program code constituting the program of the present invention into a plurality of files and downloading each file from a different homepage. That is, a WWW server that allows a plurality of users to download a program file for realizing the functional processing of the present invention on a computer is also included in the present invention.

  In addition, the program of the present invention is encrypted, stored in a storage medium such as a CD-ROM, distributed to users, and key information for decryption is downloaded from a homepage via the Internet to users who have cleared predetermined conditions. Let me. It is also possible to execute the encrypted program by using the downloaded key information and install the program on a computer.

  Further, the functions of the above-described embodiments are realized by the computer executing the read program. In addition, based on the instructions of the program, an OS or the like running on the computer performs part or all of the actual processing, and the functions of the above-described embodiments can also be realized by the processing.

  Further, the program read from the recording medium is written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer. Thereafter, the CPU of the function expansion board or function expansion unit performs part or all of the actual processing based on the instructions of the program, and the functions of the above-described embodiments are realized by the processing.

  The above-described embodiments are merely examples of implementation in carrying out the present invention, and the technical scope of the present invention should not be construed as being limited thereto. That is, the present invention can be implemented in various forms without departing from the technical idea or the main features thereof.

101 CPU
102 ROM
103 RAM
104 Display controller 105 Touch panel controller 106 Camera controller 107 Sensor controller 108 Network controller 109 Flash memory controller 110 Display 111 Touch panel 112 Camera 113 Sensor 114 Flash memory

Claims (7)

An information processing apparatus including a photographing device capable of photographing a movie,
Sound playback means for playing back sound during moving image shooting by the shooting device;
Display means for displaying a frame of a moving image shot by the shooting device in a timeline format;
Frame specifying means for specifying the frame corresponding to the time when the sound is played by the sound playing means,
The information processing apparatus, wherein the display means displays the frame specified by the frame specifying means so as to be distinguishable from the frame not specified by the frame specifying means. The display means displays one of the frames of the moving image shot by the shooting device in a timeline format,
When the frame specified by the frame specifying means is not displayed on the display means, the frame specifying means is shot at a time closest to the shooting time of the specified frame and displayed on the display means The information processing apparatus according to claim 1, wherein a specified frame is specified. The information processing apparatus includes:
Frame selecting means for selecting the frame instructed by the user from among the frames displayed by the display means;
The information processing apparatus according to claim 1, further comprising: a storage unit that stores the frame selected by the frame selection unit as a still image.   The information processing apparatus according to claim 3, wherein the display unit displays the frame in a timeline format, and further displays a still image of the frame selected by the frame selection unit. The information processing apparatus includes:
Storage means for storing an identification form corresponding to the sound reproduced by the sound reproduction means;
5. The display unit according to claim 1, wherein the display unit displays the frame specified by the frame specifying unit so that the frame can be identified by the identification form corresponding to the sound reproduced by the sound reproduction unit. The information processing apparatus described. A method for controlling an information processing apparatus including a photographing device capable of photographing a movie,
A sound reproduction step in which the sound reproduction means of the information processing apparatus reproduces a sound during moving image shooting by the imaging apparatus;
A display step in which the timeline display means of the information processing device displays a frame of a moving image shot by the shooting device in a timeline format;
A frame specifying unit for specifying the frame corresponding to the time at which the sound was played in the sound playing step;
The display step of the information processing apparatus displays the frame specified in the frame specifying step so as to be distinguishable from the frame not specified in the frame specifying step. Method. A program capable of executing a control method of an information processing device including a photographing device capable of photographing a movie,
The information processing apparatus;
Sound playback means for playing back sound during moving image shooting by the shooting device;
Display means for displaying a frame of a moving image shot by the shooting device in a timeline format;
Function as frame specifying means for specifying the frame corresponding to the time when the sound was played by the sound playback means;
The display means displays the frame specified by the frame specifying means so as to be distinguishable from the frame not specified by the frame specifying means.

Images