JP2019102850A - Information processing apparatus, control method of the same, and program - Google Patents

Abstract

To provide a mechanism for efficiently setting an in point or an out point for editing video data on the basis of a sound included in video data of a person being taken.SOLUTION: The information processing apparatus includes: detection means for detecting sound included in video data in which a person is photographed; and setting means for setting an in point or an out point for editing the video data to a position specified on the basis of the sound detected by the detection means.SELECTED DRAWING: Figure 3

Description

  The present invention relates to an information processing apparatus, a control method thereof, and a program, and more particularly to a technique for efficiently setting an in point or an out point for editing moving image data.

  Conventionally, in order to cut out a part of a moving image from moving images, the user manually inputs and specifies the cutout position of the moving image while checking the image of the moving image, and performs the extraction processing, and the operation Was complicated.

  Patent Document 1 describes that a user manually inputs and designates a cutting start time and a cutting end time from an operation key while confirming a series of moving pictures, and performs a cutting process from moving picture data.

JP, 2010-178010, A

  For example, a service may be considered that generates animation data for each student as a student's graduation album. For example, in such a service, in order to shoot a plurality of students efficiently, one camera starts shooting, student A enters the shooting range of the camera, says a single comment, and then the students As A goes out of the shooting range and the next student B enters the shooting range of the camera and says a single comment without stopping shooting, one moving image file including moving images of each student is generated Cases are conceivable.

  That is, it is conceivable that one camera continuously shoots each student who turns into a shooting range and a plurality of students alternately enter a shooting range and says a single comment, and generates one moving image file including moving images of each student. .

  Conventionally, in order to cut out a moving image file in which the student is shown from each moving image file obtained by photographing in this manner, the user checks the video of the one moving image file, It was necessary to manually enter and specify the start time and end time of the cut, and the work was complicated. In addition, at that time, the user also checks a video (a video in which a student is not shown) when a plurality of students are replaced, and it has been difficult to perform the cutting work efficiently.

  Also, for example, there may be a case where a student forgets a comment given while shooting and wants to temporarily interrupt it, but the user confirms such video, and manually inputs the start time and end time of the cut. It had to be specified, and the work was complicated.

  As described above, conventionally, it has been difficult to efficiently generate, for each person, a moving image file including the person from a single moving image file including moving images of each person in which a plurality of persons are alternately photographed.

  Therefore, the present invention is to provide a mechanism for efficiently setting an in point or an out point for editing moving image data based on the sound included in the moving image data in which a person is photographed.

  The present invention specifies a detection unit that detects a voice included in moving image data obtained by photographing a person, and an in point or an out point for editing the moving image data based on the voice detected by the detection unit. And setting means for setting the position to be set.

  Further, the present invention is a control method of an information processing apparatus, which comprises a detection step of detecting a sound included in moving image data obtained by photographing a person, and an in point or an out point for editing the moving image data. And a setting step of setting the position specified based on the voice detected by the detection step.

  Further, the present invention is characterized by a program for executing the control method.

  According to the present invention, it is possible to efficiently set an in point or an out point for editing moving image data based on the sound included in the moving image data in which a person is photographed.

1 shows an example of a system configuration of an information processing system of the present invention. It is a block diagram which shows an example of the hardware constitutions of the information processing apparatus applicable to PC101 in embodiment of this invention. FIG. 6 is an example of a diagram showing the relationship between the position (time) of each frame of a moving image, the file start point 301, each IN point, each OUT point, and the file end point 306. It is a figure which shows an example of a moving image edit screen. 7 is an example of a display screen displayed in a display area 409. It is an example of the flowchart which shows the process which concerns on the moving image edit of this embodiment. FIG. 7 is a diagram showing an example of a flowchart showing processing of performing preview reproduction of a moving image. It is a figure showing an example of a person list of a subject. It is a figure which shows an example of the conceptual diagram explaining each frame of moving image data, the audio | voice detected by the said flame | frame (frame group), and the position (time) of IN point and OUT point. It is a figure which shows an example of the conceptual diagram of one moving image file 1001 containing the moving image of a plurality of students by which a plurality of users (for example, students) were taken alternately and obtained.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 shows an example of the system configuration of the information processing system of the present invention.

  The PC 101 is an application example of the information processing apparatus of the present invention, and is a PC such as a desktop personal computer, a notebook personal computer, and a tablet personal computer.

  Also, the camera 104 is a digital camera or digital video camera capable of capturing still images including moving images and photographs. The camera 104 has a function of transmitting and receiving data to and from the PC 101 by wired or wireless communication. Therefore, the camera 104 can transmit the moving image file of the moving image captured by the camera 104 to the PC 101 by the wired or wireless communication, and record the moving image file in the storage unit such as the memory of the PC 101.

  In addition, the camera 104 can be mounted with a portable memory (portable recording medium) such as an SD card, and has a function of recording a moving image file of a moving image captured by the camera 104 in the portable memory.

  The imaging range (imaging range) of the camera 104 is an imaging range in which an object (person 103) sitting on the chair 105 can be imaged.

  When the camera 104 receives a shooting instruction from the user, the camera 104 starts shooting a moving image (video). In addition, the camera 104 stores the captured moving image and still image in the storage unit of the PC 101 through communication, or records the captured moving image and the still image in the portable memory. When stored in the portable memory, the portable memory is taken out from the camera 104, and the portable memory taken out is attached to the PC 101, and the moving picture and still picture stored in the portable memory are transferred to the PC 101. Copy (expand) to the memory of.

  Thus, the PC 101 can acquire a moving image and a still image captured by the camera 104.

  After the camera 104 starts shooting a moving image, a person (user) enters the imaging range of the camera 104, and the person sits on the chair 105. Then, when the person is ready to shoot a moving image of the person, he / she speaks the word "start". And the said person says a word comment. In addition, when the person in question, for example, forgets a comment and wants to temporarily suspend in the middle of a single comment, he / she hears the word “interrupt”.

  In addition, when the person is ready for shooting again, he / she speaks the word "Start again". Then, when the person wants to end the shooting of the moving image of the person, he / she speaks the word “end”.

  Then, a person (user) sitting on the chair 105 stands up, moves out of the imaging range of the camera 104, and performs a frame out.

  Then, without stopping shooting, the next different person comes into the imaging range of the camera 104, sits on the chair 105, and performs the same operation. Thus, the camera 104 generates one moving image file including moving images of a plurality of students by the plurality of students alternately entering the shooting range and saying a single comment, and the generated moving image file is used as the PC 101. The storage means of will be stored.

  FIG. 10 is a view showing an example of a conceptual diagram of one moving image file 1001 (moving image data) including moving images of a plurality of students, obtained by alternately photographing a plurality of users (for example, students).

  As shown in FIG. 10, one moving image file 1001 including videos of a plurality of students includes a moving image 1002 with a single comment by Taro Kiyano, and then one Jiro Kiyano Video 1003 with comment is included, followed by video 1004 with comment by Saburo Kiyano, and then video 1005 with comment by Shiro Kiyano Indicates that it is included.

  Further, the person list (FIG. 8) of the subject is stored in advance in the storage means (memory) of the PC 101, and shooting is performed in the order shown in the person list (FIG. 8).

  FIG. 8 is a diagram showing an example of the person list of the subject.

  As shown in FIG. 8, Taro Kiyano is shown at the first, Shiro Kiyano is shown at the second, Saburo Kiyano is shown at the third In the fourth place, Shiro Koyano is shown.

  FIG. 8 shows an application example of the person list according to the present invention, in which identification information for individually identifying each person included in the moving image data is determined corresponding to the order in which the moving image data is generated by the generating means. The person list is stored in storage means (memory) of the PC 101.

  The storage means of the PC 101 stores a program, various lists, and a table, which will be described later. By executing the program, the operation and processing of the function according to the present invention are executed.

  FIG. 2 is a block diagram showing an example of a hardware configuration of an information processing apparatus applicable to the PC 101 in the embodiment of the present invention. The respective devices have the same configuration, and therefore will be described using the same reference numerals.

  As shown in FIG. 2, the information processing apparatus includes a central processing unit (CPU) 201, a read only memory (ROM) 202, a random access memory (RAM) 203, an input controller 205, a video controller 206, and a system bus 204. The memory controller 207 and the communication I / F controller 208 are connected.

  The CPU 201 centrally controls the devices and controllers connected to the system bus 204.

  A storage device such as the ROM 202 or the external memory 211 is a control program executed by the CPU 201 such as a BIOS (Basic Input / Output System), an OS (Operating System), or a computer readable and executable program for realizing the information processing method. And holds necessary data (including data tables).

  The RAM 203 functions as a main memory, a work area, and the like of the CPU 201. The CPU 201 loads programs necessary for execution of processing from the ROM 202 or the external memory 211 to the RAM 203 and executes the loaded programs to realize various operations.

  The input controller 205 controls an input from the input device 209. Examples of the input device 209 include a keyboard, a touch panel, and a pointing device such as a mouse.

  When the input device 209 is a touch panel, various instructions can be given by the user pressing (touching with a finger or the like) according to an icon, a cursor, or a button displayed on the touch panel.

  Further, the touch panel may be a touch panel capable of detecting a position touched by a plurality of fingers, such as a multi-touch screen.

  Video controller 206 controls display on an external output device such as display 210. The display shall also include the display of a notebook computer integrated with the main unit. The external output device is not limited to the display, and may be, for example, a projector. In addition, an input device 209 is provided for an apparatus that can receive the touch operation described above.

  The video controller 206 can control a video memory (VRAM) for display control, and can use part of the RAM 203 as a video memory area, or can separately provide a dedicated video memory. It is possible.

  The memory controller 207 controls access to the external memory 211. The external memory is connected via an adapter to an external storage device (hard disk) that stores a boot program, various applications, font data, user files, editing files, various data, etc., a flexible disk (FD), or a PCMCIA card slot. Compact flash (registered trademark) memory etc. can be used.

  The communication I / F controller 208 connects and communicates with an external device (camera 104) via a network, and executes communication control processing on the network. For example, communication using TCP / IP, communication using Wi-Fi, and 3G communication are possible.

  The storage device such as the external memory 211 is a medium for permanently storing information, and the form thereof is not limited to the storage device such as a hard disk. For example, a medium such as a solid state drive (SSD) may be used.

  Moreover, it can also be used as a temporary memory area at the time of various processes performed by the communication terminal in the present embodiment.

  The CPU 201 enables display on the display 210 by executing, for example, outline font rasterization processing on a display information area in the RAM 203. In addition, the CPU 201 enables user instruction with a mouse cursor (not shown) or the like on the display 210.

  Next, processing executed by the PC 101 in the present embodiment will be described using the flowchart in FIG.

  FIG. 6 is an example of a flowchart showing processing relating to moving image editing of the present embodiment.

  FIG. 6 shows a process in which the CPU 201 of the PC 101 reads and executes a predetermined control program.

  First, when the PC 101 starts a predetermined control program, the PC 101 displays a moving image editing screen on a display unit such as the display 210.

  Then, the PC 101 receives and reads the selection of the person list of the subject shown in FIG. 8 to be processed by the user (S601). The PC 101 specifies the name (information having a low number) in the order of the person list read in S601 (information identifying a subject) as the processing target.

  Then, the PC 101 receives an instruction to read one moving image file to be processed by the user, corresponding to the person list selected in S601, and starts reading the moving image file (S602).

  For example, a moving image file for each class is generated, and for example, a person list of subjects as shown in FIG. 8 is stored for each class of student, and it is selected which moving image of the student of the class is to be processed. You can load a movie file of that class as a processing target.

  The start of the moving image file reading process in step S602 means that reading starts from the first frame of the moving image in order.

  First, the PC 101 reads a moving image (frame), analyzes the read moving image (frame), and determines whether a person is included in the frame, that is, whether a person in the frame can be detected. It determines (S603).

  Then, when it is determined that the person is not included in the frame, that is, it is determined that the person in the frame can not be detected (S603: NO), the PC 101 reads the next frame, and the reading is performed. The process of S603 is performed on the frame.

  When it is determined that the person is included in the frame, that is, the person in the frame is detected (S603: YES), the PC 101 detects the frame in which the person is not detected and the person. The file start point 301 is set at a position (time) between the frame determined to be successful (step S604).

  S603 is an application example of the specifying unit of the present invention, and specifies a time or a frame in which a person is framed in for each person from among moving image data captured in turn.

  FIG. 3 is an example of a diagram showing the relationship between positions (time) of each frame of the moving image, the file start point 301, each IN point, each OUT point, and the file end point 306.

  FIG. 3A is an example of a diagram showing the relationship when the “end” voice is detected before the person is no longer detected.

  FIG. 3B is an example of a diagram showing the relationship in the case where the person can not be detected in a state where the “end” voice can not be detected.

  Referring to FIG. 3A as an example, in S604, the file start point 301 is set at a position (time) between the frame 307 in which the person is not detected and the frame 308 in which the person is determined to be detected. .

  Then, when the file start point 301 is set, the PC 101 determines whether or not a predetermined voice "start" has been detected (S605). When it is determined that the predetermined voice "start" has not been detected (S605: NO), the next frame is also to be processed, and the frame (group) to be processed immediately before is selected. It is determined whether a predetermined voice of "start" is detected, including voice (S605).

  S605, S608, S611, and S615 are application examples of the detection means of the present invention, and detect the sound included in the moving image data in which a person is photographed.

  The detection means of S605 detects the first voice (start) for setting the in point (302), and the setting means of S607 detects the first voice by the detection means of S605 in the moving image data. Set the In point at a position specified based on time or frame. That is, as shown in FIG. 9, for example, as shown in FIG. 9, the setting unit of S607 is a frame (308) in which the first sound is detected by the detection unit in the moving image data, and a frame subsequent to the frame. An in point is set at a position between the frame (309) in which the voice was not detected.

  Then, if it is determined that the predetermined voice "start" has been detected (S605: YES), whether or not the talk continues continuously before and after the detected predetermined voice "start". Is determined (S606). Specifically, the PC 101 reads adjacent frames (frame groups of a predetermined time width) before and after a frame (frame group) in which a predetermined speech of “start” has been detected, and the detected “start” speech It is determined whether the talk continues continuously before and after (S606).

  Then, if it is determined that the speech continues continuously before and after the detected "start" voice (S606: YES), the PC 101 sets the processing to S605 with the next frame as the processing target. return.

  Further, when it is determined that the speech does not continue continuously before and after the detected “start” speech (S606: NO), the PC 101 detects a frame in which the speech “start” is detected, and The IN point 302 is set at a position (time) between the immediately following frame (a frame in which the voice "start" is not detected) (S607).

  S607, S610, S613, S617, and S627 are application examples of the setting unit of the present invention, and the in point or the out point of the moving image data is set at the position specified based on the sound detected by the detection unit.

  Referring to FIG. 3A as an example, in S607, the position between the frame 308 in which the voice of "start" is detected and the frame 309 immediately after that (frame in which the voice of "start" is not detected). The IN point 302 is set to (time).

  Although only one frame 308 between the file start point 301 and the IN point 302 shown in FIGS. 3 and 5 is shown to simplify the description, a plurality of frames may be included. Similarly, not only the frame 309 but also a plurality of frames may be included between the IN point 302 and the OUT point 303. Similarly, not only the frame 310 but a plurality of frames may be included between the OUT point 303 and the IN point 304. Similarly, not only the frames 311 and 312 but also three or more frames may be included between the IN point 304 and the OUT point 305. Similarly, not only the frame 313 but also a plurality of frames may be included between the OUT point 305 and the file end point 306.

  FIG. 9 is a conceptual diagram illustrating each frame of moving image data, audio detected in the frame (frame group), and positions (time) of IN point and OUT point.

  FIG. 9 is a view showing an example of a conceptual diagram for explaining each frame of moving image data, audio detected in the frame (frame group), and positions (time) of IN point and OUT point.

  Next, when the IN point 302 is set, the PC 101 determines the next frame as a processing target, and determines whether or not a predetermined voice of “suspend” is detected (S 607). Then, if it is determined that the predetermined sound of “suspend” is not detected (S607: NO), the PC 101 shifts the processing to S614. In S614, the PC 101 determines whether a person is not included in the current processing target frame, that is, whether or not a person in the frame can not be detected (S614).

  If the PC 101 determines that the person includes the person in the frame, that is, it is determined that the person in the frame has been detected (S614: NO), the process proceeds to S615, and the predetermined “end” is given. It is determined whether or not the voice of B. has been detected (S615). Then, when it is determined that the predetermined voice "END" has not been detected (S615: NO), the PC 101 also sets the next frame as a processing target, and the frame (group) processed immediately before is the processing target. It is determined whether or not to detect a predetermined sound of "suspension", including the sound of (S608).

  S 608 is an application example of the detection means of the present invention, and in the moving image data between the in point (302) and the out point (305), a first in point (304) different from the in point (302) To detect the third voice (disruption) to set.

  The detection means of S615 detects the second voice for setting the out point (305), and the setting means of S617 is based on the time or frame at which the second voice is detected by the detection means in the moving image data. The out point (305) is set at the identified position. That is, the setting unit of S617 is between the frame in which the second sound is detected by the detection unit in the moving image data and the frame which is a frame before the frame and in which the second sound is not detected. Set the out point (305) to the position.

  Then, when the PC 101 detects a predetermined voice of “suspension” (S 608: YES), the PC 101 determines whether the speech continues continuously before and after the detected predetermined voice of “suspension”. (S609). Specifically, the PC 101 reads adjacent frames (frame groups of a predetermined time width) before and after a frame (frame group) in which a predetermined speech of “suspension” has been detected, and the detected “suspension” speech It is determined whether the talk continues continuously before and after (S609).

  Then, when it is determined that the speech continues continuously before and after the detected “suspended” voice (S 609: YES), the PC 101 sets the processing to S 608 with the next frame as the processing target. return.

  In addition, when it is determined that the speech does not continue continuously before and after the detected “suspended” voice (S 609: NO), for example, as shown in FIG. The OUT point 303 is set at the position (time) between the frame in which the frame is detected and the frame immediately before it (the frame in which the voice "interrupt" is not detected) (S610).

  Referring to FIG. 3A as an example, in S610, the position between the frame 310 in which the voice of "suspension" is detected and the frame 309 immediately before that (frame in which the voice of "suspension" is not detected). The OUT point 303 is set to (time).

  S610 is an application example of the setting means of the present invention, and in the moving image data between the in point (302) and the out point (305), the time when the fourth voice (interruption) is detected by the detection means or A first out point (303) is set at a time specified based on a frame.

  Then, when the OUT point 303 is set, the PC 101 determines the next frame as a processing target, and determines whether or not a predetermined voice "restart" is detected (S611). Then, when it is determined that the PC 101 does not detect the predetermined voice "restart" (S611: NO), the next frame is also to be processed, and the frame (group) to be processed immediately before is It is determined whether or not a predetermined voice of "restart" is to be detected, including the voice of (S611).

  S 608 is an application example of the detection means of the present invention, and in the moving image data between the in point (302) and the out point (305), a first out point (303) different from the out point (305). To detect the fourth voice (start again) for setting.

  Then, when the PC 101 detects a predetermined voice “restart” (S 611: YES), whether or not the talk continues continuously before and after the detected predetermined voice “restart” Is determined (S612). Specifically, the PC 101 reads adjacent frames (frame groups of a predetermined time width) before and after a frame (frame group) in which the predetermined voice "restart" has been detected, and the detected "restart" It is determined whether the speech continues continuously before and after the voice (S612).

  Then, if it is determined that the speech continues continuously before and after the detected "restart" voice (S612: YES), the PC 101 sets the processing to be performed on the next frame as the processing target S611. Back to.

  If the PC 101 determines that the speech does not continue continuously before and after the detected "restart" voice (S612: YES), for example, as shown in FIG. The IN point 304 is set at the position (time) between the frame in which the frame is detected and the frame immediately after that (the frame in which the voice "restart" is not detected) (S613).

  Referring to FIG. 3A as an example, in S613, a period between a frame 310 in which the voice "restart" is detected and a frame 311 immediately after that (a frame in which the voice "restart" is not detected). The IN point 304 is set to the position (time) of

  S613 is an application example of the setting means of the present invention, and in the moving image data between the in point (302) and the out point (305), the time when the third voice (restart) is detected by the detection means Alternatively, the first in point (304) is set to a time specified based on a frame.

  In this way, by executing the processing from S608 to S613, for example, when a student forgets a comment given while shooting and wants to temporarily suspend, the user confirms the interrupted video, and the clipping start time, clipping By manually entering the end time, it is possible to reduce the number of designated operations.

  After setting the IN point 304, the PC 101 sets the next frame as a processing target, and shifts the processing to S614.

  In S614, the PC 101 determines whether a person is not included in the current processing target frame, that is, whether or not a person in the frame can not be detected (S614).

  If the PC 101 determines that the person includes the person in the frame, that is, it is determined that the person in the frame has been detected (S614: NO), the process proceeds to S615, and the predetermined “end” is given. It is determined whether or not the voice of B. has been detected (S615). Then, when it is determined that the predetermined voice "END" has not been detected (S615: NO), the PC 101 also sets the next frame as a processing target, and the frame (group) processed immediately before is the processing target. It is determined whether or not to detect a predetermined sound of "suspension", including the sound of (S608).

  When the PC 101 detects a predetermined voice of "end" (S615: YES), the PC 101 determines whether the speech continues continuously before and after the detected predetermined voice of "end". (S616). Specifically, the PC 101 reads adjacent frames (frame groups of a predetermined time width) before and after a frame (frame group) in which a predetermined voice of “end” has been detected, and the detected “end” voice It is determined whether the talk continues continuously before and after (S616).

  Then, if it is determined that the speech continues continuously before and after the detected “end” voice (S 616: YES), the PC 101 sets the processing to S 615 with the next frame as the processing target. return.

  If the PC 101 determines that the speech does not continue continuously before and after the detected "end" voice (S616: YES), for example, as shown in FIG. 9, the voice "end" is detected. The OUT point 305 is set at the position (time) between the received frame and the frame immediately preceding it (the frame in which the voice of "end" is not detected) (S617).

  Thus, in S 607, between the frame in which the first sound (for example, the start) is detected in the moving image data and the frame subsequent to the frame and in which the first sound is not detected. The in point (302) is set at the position, and in S617, the frame in which the second speech (for example, end) is detected and the frame before the frame but in which the second speech is not detected. To set the out point (305) at a position between them, the first voice and the second voice are not included between the in point (302) and the out point (305), and the preferred in point ( It becomes possible to efficiently set the 302) and the out point (305).

  Taking FIG. 3A as an example, in S617, the position between the frame 313 in which the voice of "end" is detected and the frame 312 (frame in which the voice of "end" is not detected) immediately before that. The OUT point 305 is set to (time).

  After setting the OUT point 305, the PC 101 sets the next frame as a processing target, and shifts the processing to S618.

  Next, in S618, the PC 101 determines whether a person is not included in the current processing target frame, that is, whether or not a person in the frame can not be detected (S618).

  When it is determined that the person contained in the frame, that is, the person in the frame has been detected (S 618: NO), the PC 101 sets the next frame as the processing target, and performs the processing in S 618 again. Do the processing.

  In addition, when it is determined that the person is not included in the frame, that is, it is determined that the person in the frame can not be detected (S618: YES), the PC 101 determines that the person is not included. The file end point 306 is set at the position (time) between the frame and the frame immediately preceding it (the frame in which the person is detected) (S619).

  If the PC 101 determines in S614 that no person is included in the frame, that is, if it is determined that the person in the frame could not be detected (S614: YES), it is assumed that no person is included. The OUT point 305 is set at the position (time) between the determined frame and the frame immediately preceding it (the frame in which a person is detected) (S627).

  Taking FIG. 3B as an example, in S627, at a position (time) between a frame 314 determined to contain no person and a frame 313 (a frame in which a person is detected) immediately before that. The OUT point 305 is set.

  The PC 101 displays identification information for identifiably displaying (highlighting) a frame to be displayed (a frame in which a person is detected) immediately before the position (time) at which the OUT point 305 is set in S627. And add (S628). Then, as illustrated in FIG. 3B, the PC 101 sets the file end point 306 at the same position (time) as the OUT point 305 set in S627 (S619).

  S 618 and S 614 are application examples of the specifying unit of the present invention, and specify the time or frame in which the frame is out for each person from the moving image data in which the person is alternately photographed.

  S627 is an application example of the setting means of the present invention, and when the second voice (end) for setting the out point (305) can not be detected by the detection means, the frame is output from the moving image data, the frame The out point is set at the division position specified based on the time or frame that was out.

  The PC 101 cuts out the moving image file read in S602 at the position (time) where the file start point 301 is set in S604 and the position (time) where the file end point 306 is set in S619, and the cut out moving image file And the current processing target name (information for identifying the subject) are registered (set) in association with each other (S620).

  S620 is an application example of the generation means of the present invention, and the moving picture data is divided at divided positions specified based on the time or frame specified by the specifying means of S603, S618, S614 to individually set moving picture data for each person. Generate to

  S620 is an application example of the registration means of the present invention, and the moving image data divided and generated sequentially from the top of the moving image data is defined in the person list (FIG. 8) corresponding to the order in which the moving image data is generated. Associate with the registered identification information and register.

  Then, the PC 101 determines whether all moving image files read in S602 have been subjected to the processing of the processing of S603 to S619, or all names of the person list read in S601 as the processing target, from S603 to S619. It is determined whether the process of cutting out the moving image file read in S602 and setting of the IN point and the OUT point ends (S621).

  In step S621, the PC 101 processes all the moving image files read in step S602 as processing targets in steps S603 to S619, or sets all names in the person list read in step S601 as processing targets in steps S603 to S619. If it is determined that S has been executed, it is determined that the process of cutting out the moving image file read in S602 and setting of the IN point and the OUT point is ended (S621: YES), and the process proceeds to S622.

  Also, in S621, the PC 101 does not process all the moving image files read in S602 as the processing targets of S603 to S619, or processes all names in the person list read in S601 as S603 to S619. If it is determined that the process up to is not executed, it is determined that the process of cutting out the moving image file read in S602 and the setting of the IN point and the OUT point is not ended (S621: NO) and read in S601 While processing the processing target to the next frame of the moving image file read in S602 as the processing target, while processing the processing target as the processing target, while making the processing target the name with the earliest (low number) name among unprocessed names in the person list .

  S621 is an application example of the control means of the present invention, and it is specified based on the frame position-in time or frame-specific divided position specified by the identification means and the frame-out time or frame specified by the identification means Control is performed so that detection processing by the detection unit and setting processing by the setting unit are performed on the moving image data between the divided positions.

  The PC 101 displays a list (list) of the moving image file cut out in S602 and the names registered in association with the moving image file on the moving image editing screen (FIG. 4) (S622).

  S 622 is an application example of the display means of the present invention, and as shown in the moving picture editing screen (FIG. 4), images of a plurality of frames of moving picture data are displayed in time series and The control relating to the in point set by the (501, 506) and the control relating to the out point (505, 502) are displayed.

  FIG. 4 is a diagram showing an example of a moving image editing screen.

  Reference numeral 412 denotes the value of the order of the person list shown in FIG. 8, and reference numeral 401 denotes the name. The clipped moving image file registered in S620 in association with the information identifying the subject is shown as a moving image timeline in 403 and 404. 403 indicates the animation limeline between the IN point 302 and the OUT point 305, and 404 indicates the animation timeline between the OUT point 305 and the file end point 306. Reference numeral 405 denotes a moving image after the file end point 306.

  Reference numeral 402 denotes a reduced image (for example, a thumbnail image) of the first frame of the moving image between the IN point 302 and the OUT point 305 or the moving image between the IN point 302 and the OUT point 303.

  Also, reference numeral 408 is shown as a timeline of the moving image between the file start point 301 and the IN point 302.

  Reference numeral 407 denotes an area where the waveform of the sound included in the clipped moving image file is displayed.

  In 406, the clipped moving image file and the name of the current processing target (information for identifying a subject) are associated (registered) (set) in S620, and an IN point is arbitrarily selected in S626 in accordance with a user instruction. It is a button (instruction accepting unit) for selecting the processing target of the moving image file which is set based on the IN point and the OUT point after editing (modification) of the OUT point is made. That is, it is a button for selecting which moving image is to be processed.

  As shown in FIG. 4, the above-mentioned 401, 402, 403, 404, 405, 406, 407, 408, and 412 are listed and displayed for each name (for each moving image file cut out in S620).

  Reference numeral 411 denotes a file generation button.

  As shown in FIG. 4, the above-mentioned 401, 402, 403, 404, 405, 406, 407, 408, and 412 are listed and listed for each name (for each moving image file cut out in S620). When a moving image or a name (information identifying a subject) is selected, the moving image editing screen (FIG. 4) has a display area 410 for displaying a preview of the moving image. Reference numeral 413 denotes a play button, which displays a preview of the moving image in the display area 410 when any one of the moving images or names (information for identifying a subject) is selected and the reproduction button 413 is pressed by the user. .

  Reference numeral 409 denotes a display area in which either the moving image or the name displayed in a list is selected, and the user receives an instruction to edit the IN point and OUT point of the selected moving image. Specifically, the display area 409 is, for example, a display area where FIG. 5 (A) or FIG. 5 (B) is displayed.

  In S 622, the PC 101 displays the selected moving image and the selected moving image of the names in the display area 409 in the moving image editing screen (FIG. 4), as shown in FIG. 5 at this time. , A frame 308 of the moving image between the file start point 301 and the IN point 302, a frame 310 of the moving picture between the OUT point 303 and the IN point 304 (interruption area: a frame of the interruption video time zone 408), and a file end point from the OUT point 305 With regard to the frame 313 of the moving image between 306, blackout or the color of the frame of the frame is identifiably displayed, for example, by changing the frame of another frame.

  In S628, identification information for identifiably displaying (highlighting) a frame to be displayed immediately before the position (time) where the OUT point 305 is set in S627 (a frame in which a person is detected) is relevant in S628. Since the frame is added, the frame to which the identification information is added is identified and displayed.

  This identification display is an application example of the notification means of the present invention, and when the PC 101 can not detect the second voice for setting the out point by the detection means, the frame is out of the moving image data. (2) A notification is given to notify the user that a frame out of moving image data has not been detected because the audio can not be detected.

  That is, the notification means does not detect the second voice for setting the out point by the detection means, and when the moving picture data is out of frame, the frame displayed immediately before the frame which is out of the frame is The display is made distinguishably in a form different from the display form of the frame.

  As a result, the user can not detect the second voice for setting the out point, and can grasp that the frame is out of the moving image data, and adjusts the position of the control 502 of the out point (305). It will be appreciated that it is necessary and it will be difficult to forget to adjust the position of control 502.

  FIG. 5 is an example of a display screen displayed in the display area 409.

  FIG. 5 is an example of a diagram showing the relationship between positions (time) of each frame of the moving image cut out in the range from the file start point 301 to the file end point 306 shown in FIG. 3 and each IN point and each OUT point.

  FIG. 5A is an example of a diagram showing the relationship when “end” sound is detected before a person is no longer detected.

  FIG. 5B is an example of a diagram showing the relationship when the person can not be detected in a state where the “end” voice can not be detected.

  As shown in FIG. 5, in the display area 409, controls 501, 502, 505 which allow the user to accept editing (modification) of each frame of the moving image selected by the user and each IN point and each OUT point. , 506 are displayed.

  A frame 308 is between the file start point 301 set in S604 and the IN point 302 set in S607.

  A frame 309 is the frame between the IN point 302 set in step S607 and the OUT point 303 set in step S610. A frame 310 is between the OUT point 303 set in step S610 and the IN point 304 set in step S613. Then, the time zone 408 of the interruption video is displayed in a distinguishable manner.

  Also, the frames between the IN point 304 set in S613 and the OUT point 305 set in S617 are 311 and 312, respectively.

  Further, the frame between the OUT point 305 set in S617 and the file end point 306 set in S619 is 313 (FIG. 5A).

  As shown in FIG. 5, on the initial screen before accepting the editing instruction from the user, the control 501 capable of accepting the editing (modification) of the IN point 302 by the user at the position of the IN point 302 set in S607. Is displayed.

  Further, at the position of the OUT point 303 set in S610, a control 505 which allows the user to receive an edit (change) of the OUT point 303 is displayed.

  Further, at the position of the IN point 304 set in step S613, a control 506 that allows the user to accept editing (modification) of the IN point 304 is displayed.

  Further, at the position of the OUT point 305 set in step S617, a control 502 which allows the user to accept editing (modification) of the OUT point 305 is displayed.

  The PC 101 receives an operation instruction from the user via the moving image editing screen (FIG. 4) (S623).

  For example, the PC 101 moves (changes) the control 501, 505, 506, 502 to a position between arbitrary frames according to the operation of the user. This makes it possible to edit a moving image by changing the positions of each IN point and each OUT point to an arbitrary position.

  Further, in S 622, the PC 101 displays the selected moving image and name moving image of the moving image and name listed in the moving image editing screen (FIG. 4) in the display area 409, and performs the same operation for each moving image. It can be done.

  Then, the PC 101 receives the selection of each of the moving images and each name button 406 (instruction receiving unit) listed and displayed on the moving image editing screen (FIG. 4) by the user, and receives the pressing of the file generation button 411 (S623).

  If the PC 101 determines in S623 that the operation content accepted by the user is an editing instruction for editing a moving image by changing the position of the IN point or the OUT point to an arbitrary position (S624) : Editing instruction), editing processing for registering the position at which the IN point or the OUT point is arbitrarily changed by the user operation (S626). Then, the process returns to S623.

  S623 is an application example of the receiving unit of the present invention, and the user receives an instruction to change the position at which the control is displayed on the image (frame) by the display unit.

  Further, S626 is an application example of the changing means of the present invention, and according to the change instruction accepted by the accepting means of S623, the change of the display position of the control with respect to the image (frame) and the in point set by the setting means Or change the position of the out point and display the result.

  Further, the PC 101 receives the selection of the button 406 (instruction accepting unit) of each moving image and each name displayed in the list at step S623, and the operation content accepted by the user is an instruction (generation If it is determined that it is an instruction (S 624: generation instruction), the file of the selected moving image is generated (S 625). Specifically, the moving image is cut at the positions of each IN point and each OUT point currently registered, and a moving image file for each name (user) is generated. That is, when the editing process of S626 is performed, the moving image is cut at the position of the edited IN point and / or OUT point, and a moving image file for each name (user) is generated.

  For example, as shown in FIG. 5A, the control 501 of the IN point 302 is set at a position between the frame 308 and the frame 309, and the OUT point 303 is set at a position between the frame 309 and the frame 310. The control 505 is set, the control 506 of the IN point 304 is set at a position between the frame 310 and the frame 311, and the control 502 at the OUT point 305 is set at a position between the frame 312 and the frame 313. If so, the PC 101 cuts out the frame (group) between the control 501 and the control 505 and the frame (group) between the control 506 and the control 502 in S625, Frame (s) and controls 506 and controls And it connects the frame (s) between the roll 502 to generate a single video file.

  Here, if the controls 505 and 506 do not exist, the frame (group) between the control 501 and the control 502 is cut out to generate one moving image file.

  This processing is executed for each moving image (for each name (user)) selected by the user in S623, and a moving image file for each name (user) is generated (S625). Then, the process ends.

  Next, processing for performing preview reproduction of a moving image will be described with reference to FIG.

  FIG. 7 is a diagram showing an example of a flowchart showing processing for performing preview reproduction of a moving image.

  The flowchart illustrated in FIG. 7 is a process in which the CPU 201 of the PC 101 reads and executes a control program.

  The process shown in FIG. 7 is a process that can be executed in S623 of FIG.

  The PC 101 determines whether one moving image is selected from the moving images of the respective names listed on the moving image editing screen of FIG. 4 by the user, and the user determines whether the pressing of the play button 413 has been received or not. It is determined whether an instruction to play back the moving image has been received (S701).

  Here, whether a playback instruction for a moving image before performing the editing process of S626 or a moving image after the editing process of S626 (moving image in which the position of the IN point or the OUT point is changed to an arbitrary position in S626) has been received It is determined whether or not.

  Then, when it is determined that the reproduction instruction of the moving image is accepted, the PC 101 reads the moving image file of the moving image (S702), and reproduces the moving image (S703). Here, the frame (s) between the control 505 and the control 506 are not reproduced skipping. Further, the frame (group) before the control 501 and the frame (group) after the control 502 are not reproduced similarly.

  The moving image reproduced here is a moving image in which the frame (group) between the control 501 and the control 505 and the frame (group) between the control 506 and the control 502 are linked.

  When the reproduction of the moving image ends (S704: YES), the PC 101 returns the process to S701. In addition, when the reproduction of the moving image is not finished, the moving image is continuously reproduced (S703).

  In the PC 101, as described in S606, S609, S612, and S616, when the voice is continuously detected within a predetermined time immediately before or after the voice for setting the in point or the out point by the detection unit. (S606: NO, S609: NO, S612: NO, S616: NO) The in point or out point is not set for setting the in point or out point of the moving image data based on the voice.

  As described in S606, S609, S612, and S616, the PC 101 does not detect voice continuously within a predetermined time immediately before or after the voice for setting the in point or the out point (S606: YES, step S609: YES, step S612: YES, step S616: YES), setting the in point or out point of the moving image data based on the voice to set the in point or out point (S607, S610, S613, S617).

  As described above, according to the present invention, it is possible to efficiently set an in point or an out point for editing moving image data based on the sound included in the moving image data in which a person is photographed.

  Further, according to the present invention, it is possible to efficiently generate, for each person, a moving image file including the person from a single moving image file containing moving images of each person in which a plurality of persons are alternately photographed. Become.

  The present invention can be embodied as, for example, a system, an apparatus, a method, a program, or a recording medium. Specifically, the present invention may be applied to a system constituted by a plurality of devices, or may be applied to an apparatus comprising a single device.

  Further, the program in the present invention is a program that can execute the processing method of the flowchart illustrated in the computer, and the storage medium of the present invention stores a program that can execute the processing method. The program in the present invention may be a program for each processing method of each device.

  As described above, the recording medium recording the program for realizing the functions of the above-described embodiments is supplied to the system or apparatus, and the computer (or CPU or MPU) of the system or apparatus stores the program stored in the recording medium. It goes without saying that the object of the present invention can also be achieved by reading and executing.

  In this case, the program itself read out from the recording medium realizes the novel function of the present invention, and the recording medium recording the program constitutes the present invention.

  As a recording medium for supplying the program, for example, a flexible disk, hard disk, optical disk, magneto-optical disk, CD-ROM, CD-R, DVD-ROM, magnetic tape, non-volatile memory card, ROM, EEPROM, silicon A disk etc. can be used.

  Further, by executing the program read by the computer, not only the functions of the above-described embodiment are realized, but also an operating system (OS) or the like running on the computer is actually executed based on the instructions of the program. It goes without saying that the processing is partially or entirely performed, and the processing realizes the functions of the above-described embodiments.

  Furthermore, after the program read from the recording medium is written to the memory provided to the function expansion board inserted into the computer or the function expansion unit connected to the computer, the function expansion board is read based on the instruction of the program code. It goes without saying that the case where the CPU or the like provided in the function expansion unit performs part or all of the actual processing and the functions of the above-described embodiment are realized by the processing.

  Further, the present invention may be applied to a system constituted by a plurality of devices or to an apparatus comprising a single device. It goes without saying that the present invention can also be applied to the case where it is achieved by supplying a program to a system or apparatus. In this case, by reading a recording medium storing a program for achieving the present invention into the system or apparatus, the system or apparatus can receive the effects of the present invention.

  Further, by downloading and reading out a program for achieving the present invention from a server on a network, a database or the like by a communication program, the system or apparatus can receive the effects of the present invention. In addition, the structure which combined each embodiment mentioned above and its modification is also contained in this invention altogether.

101 PC
102 Photographer 103 Person 104 Camera 105 Chair

Claims (16)

Detection means for detecting sound included in moving image data in which a person is photographed;
Setting means for setting an in point or an out point for editing the moving image data to a position specified based on the sound detected by the detection means;
An information processing apparatus comprising: The detection means detects a first voice for setting the in-point;
2. The apparatus according to claim 1, wherein the setting unit sets the in point at a position specified based on a time or a frame at which the first sound is detected by the detection unit in the moving image data. Information processing equipment.   The setting means is a frame between the frame in which the first sound is detected by the detection means and the frame which is a subsequent frame of the frame and in which the first sound is not detected in the moving image data. The information processing apparatus according to claim 2, wherein the in-point is set at a position. The detection means detects a second voice for setting the out point;
4. The apparatus according to claim 1, wherein the setting unit sets the out point at a position specified based on a time or a frame at which the second sound is detected by the detection unit in the moving image data. The information processing apparatus according to any one of the above.   The setting means is a frame between the frame in which the second sound is detected by the detection means and the frame which is a frame before the frame and in which the second sound is not detected in the moving image data. 5. The information processing apparatus according to claim 4, wherein the out point is set at a position of. Display means for displaying images of a plurality of frames of the moving image data in time series and displaying a control on an in point set by the setting means at a position between the plurality of images and a control on an out point;
Accepting means for accepting, by the user, an instruction to change the position at which the control is displayed on the image by the display means;
Changing means for changing the display position of the control with respect to the image, and changing the position of the in point or the out point set by the setting means, in accordance with the change instruction received by the reception means;
The information processing apparatus according to any one of claims 1 to 5, further comprising: A specifying means for specifying a frame-in and a frame-out time or frame for each person from moving image data in which a person is alternately photographed;
Generation means for dividing the moving image data at division positions specified based on the time or frame specified by the specifying means to individually generate moving image data for each person;
The information processing apparatus according to any one of claims 1 to 6, further comprising:   For moving image data between a divided position specified based on the frame in time or frame specified by the specifying means and a divided position specified based on the frame out time or frame specified by the specifying means 8. The information processing apparatus according to claim 7, further comprising: control means for controlling to perform detection processing by the detection means and setting processing by the setting means. Storage means for storing a person list in which identification information for individually identifying each person included in the moving image data is determined corresponding to the order of generation of the moving image data by the generating means;
Registration means for registering moving image data sequentially divided and generated from the head of the moving image data in association with identification information defined in the person list in correspondence with the order in which the moving image data is generated;
The information processing apparatus according to claim 7, further comprising:   When the setting means is not able to detect the second voice for setting the out point by the detection means, and the frame is out from the moving image data, the division specified based on the time or frame when the frame is out The information processing apparatus according to any one of claims 1 to 9, wherein the out point is set at a position.   If the second audio for setting the out point can not be detected by the detection means and the frame is out of the moving image data, the second audio can not be detected and the frame is out of the moving image data 11. The information processing apparatus according to claim 10, further comprising notification means for notifying a user of notification.   When the notification means does not detect the second voice for setting the out point by the detection means, and the frame is out of the moving image data, the frame displayed immediately before the frame which has been out of the frame is selected. The information processing apparatus according to claim 11, wherein the information processing apparatus displays the information in a distinguishable manner in a form different from the display form of the other frame. The detection means is a third voice for setting a first in point different from the in point in the moving image data between the in point and the out point, and a different from the out point Detect the fourth voice to set the first out point,
The setting means sets the first out point at a time specified based on a time or a frame at which the fourth sound is detected by the detection means in the moving image data between the in point and the out point. 13. The method according to claim 1, wherein the first in-point is set to a time set based on a time or a frame at which the third voice is detected by the detection means. The information processing apparatus described in the item.   The setting unit is configured to, when the voice is continuously detected within a predetermined time immediately before or after the voice for setting the in point or the out point by the detection means, the in point or the in point. The in point or the out point of the moving image data based on the voice for setting the out point is not set, and the predetermined time immediately before or after the voice for setting the in point or the out point When a voice is not continuously detected in the inside, the setting of the in point or the out point of the moving image data based on the voice for setting the in point or the out point is performed. The information processing apparatus according to any one of claims 1 to 13. A control method of the information processing apparatus,
A detection step of detecting sound included in moving image data in which a person is photographed;
A setting step of setting an in point or an out point for editing the moving image data to a position specified based on the sound detected by the detection step;
A control method comprising: A program for executing the control method according to claim 15.

Images