JP2021114071A - Apparatus and method for specifying object to be edited, program, and recording medium - Google Patents

Abstract

To provide an editing object specifying apparatus which can easily edit an image automatically captured by a camera, etc., in accordance with the purpose.SOLUTION: An editing object specifying apparatus 1 includes image acquisition means 11, storage means 12, editing object specifying means 13, and scoring means 14. The image acquisition means 11 acquires images. The storage means 12 stores the images and attribute information of an editing object. The editing object specifying means 13 specifies an image including the editing object from among the images by collating the images with the attribute information of the editing object. The scoring means 14 combines a plurality of images including the editing object to generate a score, and selects an image of which the score exceeds a threshold.SELECTED DRAWING: Figure 1

Description

The present invention relates to an edit target identification device, an edit target identification method, a program, and a recording medium.

Various video editing devices have been proposed that create a digest video based on a video shot by a camera (see, for example, Patent Document 1).

JP-A-2019-110422

However, for example, there is a demand for a new technique that can easily edit an image automatically taken by a camera or the like according to a purpose.

Therefore, an object of the present invention is to provide, for example, an edit target identification device and an edit target identification method that can easily edit an image automatically taken by a camera or the like according to a purpose.

In order to achieve the above object, the editing target specifying device of the present invention is used.
Includes image acquisition means, storage means, edit target identification means, and scoring means.
The image acquisition means acquires an image and
The storage means stores the image and the attribute information to be edited.
The editing target specifying means collates the image with the attribute information of the editing target, identifies an image including the editing target from the images, and identifies the image.
The scoring means is a device that combines a plurality of images including the editing target and scores them, and selects an image whose score exceeds a threshold value.

The method for specifying an edit target of the present invention is
Includes an image acquisition process, a storage process, an edit target identification process, and a scoring process.
The image acquisition step acquires an image and
The storage step stores the image and the attribute information to be edited.
In the editing target specifying step, the image is collated with the attribute information of the editing target, and an image including the editing target is specified from the images.
The scoring step is a method in which a plurality of images including the editing target are combined and scored, and an image whose score exceeds a threshold value is selected.

According to the present invention, for example, an image automatically taken by a camera or the like can be easily edited according to a purpose.

FIG. 1 is a block diagram showing an example of the configuration of the edit target specifying device of the first embodiment. FIG. 2 is a block diagram showing an example of the hardware configuration of the edit target specifying device of the first embodiment. FIG. 3 is a flowchart showing an example of processing in the editing target specifying device of the first embodiment. FIG. 4 is a block diagram showing an example of the configuration of the edit target specifying device according to the second embodiment. FIG. 5 is a block diagram showing an example of the hardware configuration of the edit target specifying device according to the second embodiment. FIG. 6 is a flowchart showing an example of processing in the editing target specifying device of the second embodiment. FIG. 7 is a schematic view showing an example of a mode in which the editing target specifying means identifies an image including the editing target from the images in the first embodiment. FIG. 8 is a schematic diagram showing an example of an embodiment in which the editing target identifying means divides a moving image at predetermined time intervals and identifies an image including the editing target from the moving image in the first embodiment. FIG. 9 is a schematic diagram showing an example of a mode in which, in the first embodiment, a plurality of images including the editing target are combined and scored by a scoring means, and an image whose score exceeds a threshold value is selected. FIG. 10 is a schematic diagram showing an example of a mode in which, in the first embodiment, a plurality of images including the editing target are combined and scored by a scoring means, and an image whose score exceeds a threshold value is selected. FIG. 11 is a schematic diagram showing an image including an edit target and an example of a label.

In the edit target identification device of the present invention, for example, the image may be a moving image, and the edit target identification means may specify an image including the edit target by dividing the image at predetermined time intervals.

The editing target specifying device of the present invention further includes, for example, labeling means, and the labeling means attaches a label containing the attribute information of the editing target and information about the image to the image including the editing target, and the above-mentioned. The storage means may be in the form of associating the image including the editing target with the label and storing the image.

In the editing target specifying device of the present invention, for example, even if the information regarding the image is at least one selected from the group consisting of the shooting date and time, the shooting location, the weather, and the shooting state of the editing target. good.

In the editing target specifying device of the present invention, for example, the editing target is a person, and the scoring means is composed of a group consisting of a person around the editing target, a shooting location of the image, and a shooting time of the image. The scoring may be performed using at least one selected.

In the editing target specifying device of the present invention, for example, the editing target is a person, and the attribute information of the editing target consists of a group consisting of a face image, an ID (identification, identification), a name, an age, and a gender. It may be at least one selected.

In the method for specifying an editing target of the present invention, for example, the image may be a moving image, and the editing target specifying step may be divided into predetermined time intervals to specify an image including the editing target.

In the method for specifying an editing target of the present invention, for example, a labeling step is further included, and the labeling step attaches a label containing the attribute information of the editing target and information about the image to the image including the editing target. The storage step may be an embodiment in which the image including the editing target and the label are associated and stored.

In the method for specifying an editing target of the present invention, for example, even if the information regarding the image is at least one selected from the group consisting of the shooting date and time, the shooting location, the weather, and the shooting state of the editing target. good.

In the method for specifying an editing target of the present invention, for example, the editing target is a person, and the scoring step consists of a group consisting of a person around the editing target, a shooting location of the image, and a shooting time of the image. The scoring may be performed using at least one selected.

In the method for specifying an editing target of the present invention, for example, the editing target is a person, and the attribute information of the editing target is at least one selected from a group consisting of a face image, an ID, a name, an age, and a gender. It may be in the form of.

The program of the present invention is a program capable of executing the method of specifying the editing target of the present invention on a computer.

The recording medium of the present invention is a computer-readable recording medium on which the program of the present invention is recorded.

Next, an embodiment of the present invention will be described with reference to FIGS. 1 to 11. The present invention is not limited or limited by the following embodiments. In addition, in FIGS. 1 to 11, the same parts are designated by the same reference numerals. The descriptions in each embodiment can be incorporated into each other.

[Embodiment 1]
FIG. 1 is a block diagram showing an example of the configuration of the edit target specifying device 1 of the present embodiment. As shown in FIG. 1, the editing target specifying device 1 includes an image acquisition means 11, a storage means 12, an editing target specifying means 13, and a scoring means 14. The editing target identification device of the present invention may be incorporated in, for example, a server. Further, the editing target specifying device of the present invention may be, for example, a personal computer (PC) in which the program of the present invention is installed.

FIG. 2 illustrates a block diagram of the hardware configuration of the editing target specifying device 1. As shown in FIG. 2, the editing target specifying device 1 includes, for example, a CPU (central processing unit) 101, a memory 102, a bus 103, a storage means 12, an input device 105, a display device 106, a communication device 107, and the like. Each part of the edit target specifying device 1 is connected via the bus 103 by, for example, each interface (I / F).

The CPU 101 is responsible for controlling the entire editing target specifying device 1. In the editing target detection device 1, for example, the program of the present invention and other programs are executed by the CPU 101, and various information is read and written. Specifically, for example, the CPU 101 functions as an image acquisition means 11, an editing target specifying means 13, and a scoring means 14. In the present invention, another arithmetic unit such as a GPU may be used instead of the CPU.

The bus 103 can also be connected to, for example, an external device. Examples of the external device include an external storage device (external database, etc.), a printer, and the like. The editing target specifying device 1 can be connected to the communication network by, for example, the communication device 107 connected to the bus 103, and can also be connected to the external device via the communication network. The communication network is not particularly limited, and a known network can be used. For example, it may be wired or wireless. Examples of the communication line network include an Internet line, WWW (World Wide Web), telephone line, LAN (Local Area Network), WiFi (Wireless Fidelity) and the like.

The memory 102 includes, for example, a main memory, and the main memory is also referred to as a main storage device. When the CPU 101 performs processing, for example, the memory 102 reads various operation programs 104 such as the program of the present invention stored in the storage means 12 described later, and the CPU 101 receives data from the memory 102. Program 104 is executed. The main memory is, for example, a RAM (random access memory). The memory 102 further includes, for example, a ROM (read-only memory).

The storage means 12 is also referred to as a so-called auxiliary storage device with respect to the main memory (main storage device), for example. As described above, the storage means 12 stores an operation program 104 including the program of the present invention. Further, the storage means 12 stores the image and the attribute information to be edited. The storage means 12 includes, for example, a storage medium and a drive for reading and writing to the storage medium. The storage medium is not particularly limited, and may be an internal type or an external type, for example, HD (hard disk), FD (floppy (registered trademark) disk), CD-ROM, CD-R, CD-RW, MO, etc. Examples thereof include a DVD, a flash memory, a memory card, and the like, and the drive is not particularly limited. The storage means 12 may be, for example, a hard disk drive (HDD) in which a storage medium and a drive are integrated.

The editing target specifying device 1 further includes, for example, an input device 105 and a display device 106. The input device 105 is, for example, a touch panel, a keyboard, a mouse, or the like. Examples of the display device 106 include an LED display and a liquid crystal display.

In the editing target specifying device 1, the memory 102 and the storage means 12 can also store access information and log information from the user, and information acquired from an external database (not shown).

Next, an example of the method for specifying the editing target of the present embodiment will be described with reference to the flowchart of FIG. The editing target specifying method of the present embodiment is carried out as follows, for example, using the editing target specifying device 1 of FIG. The editing target specifying method of the present embodiment is not limited to the use of the editing target specifying device 1 of FIG.

First, the image is acquired by the image acquisition means 11 (S11). The image acquisition means 11 may acquire the image by, for example, taking a picture with an image pickup means such as a camera mounted on the editing target specifying device 1. Further, the image is mounted on a moving body such as a camera (for example, a fixed camera capable of controlling the shooting direction, a fixed camera whose shooting direction cannot be controlled, a drone, etc.) provided separately from the editing target specifying device 1, for example. It may be an image taken by an image pickup means such as a mobile camera or the like, and acquired by an image acquisition means 11 of the edit target identification device 1 via the communication network. The image may be, for example, a still image or a moving image.

Next, the storage means 12 stores the image and the attribute information to be edited (S12). The editing target is not particularly limited, and for example, participants of parties such as athletic meet, school festival, wedding, etc. (for example, children, grandchildren, children, students, students, grooms, brides, attendees, etc.), performers of concerts, etc. , People such as spectators, animals such as race horses in horse racing, automobiles such as race cars in car races, and gods in festivals. The editing target may be one (for example, one person, one animal, one car, etc.) or a plurality (for example, a plurality of people, a plurality of animals, a plurality of automobiles, etc.). Etc.). Examples of the attribute information of the editing target include a face image, ID, name, age, gender, and the like if the editing target is a person. Note that FIG. 3 shows an example in which the attribute information to be edited is stored in the storage step (S12) after the image acquisition step (S11), but the present invention is not limited to this. In the present invention, the attribute information to be edited in the storage step (S12) may be stored before the image acquisition step (S11), or in the image acquisition step (S11) and the storage step (S12). The attribute information to be edited may be stored at the same time.

Next, the editing target specifying means 13 collates the image with the attribute information of the editing target, and identifies an image including the editing target from the images (S13).

FIG. 7 shows an example of a mode in which the editing target specifying means 13 identifies an image including the editing target from the images. As shown in FIG. 7, in this example, the editing target is a child who participates in an athletic meet. In this example, the editing target specifying means 13 collates the image stored in the storage means 12 with the attribute information of the editing target (in this example, facial images of a plurality of children), and sets the editing target. The image to be included (in this example, the image of the child who finishes in the race) is specified. For the identification, for example, a known face recognition technique or the like can be used. As illustrated in FIG. 7, the editing target identifying means 13 further detects a plurality of face images (four of # 1 to # 4 in this example), and sets the ID of a child having a high degree of matching as a label. You may.

When the image is a moving image, the editing target specifying means 13 may specify the image including the editing target by dividing it at predetermined time intervals, for example, as shown in FIG. The predetermined time is not particularly limited, and is, for example, 1 second, 1 minute, or the like.

Next, the scoring means 14 combines a plurality of images including the editing target into a score, and selects an image whose score exceeds the threshold value (S14).

9 and 10 show an example of a mode in which the scoring means 14 combines a plurality of images including the editing target and scores them, and selects an image whose score exceeds the threshold value. As shown in FIG. 9, in this example, three edits are made: an image of a child scoring a goal in a race (image 1), an image of a child doing gymnastics (image 2), and an image of a child doing tug of war (image 3). The images including the target are combined. As illustrated in FIG. 9, in images 1 to 3, facial images of a plurality of children (in this example, six children a to f) are extracted by the editing target identifying means 13, and the degree of coincidence with the images is high. Children's IDs (IDs-a to f) are set as labels. The scoring means 14 carries out the scoring as follows, for example. That is, first, as illustrated in FIG. 9, the scoring means 14 summarizes the combinations of the children a to f included in the images 1 to 3 in a table, and the images including the child a include the children c and f. It is found that the probability of being included is high, and if there is a face image whose ID is unknown in image 2, it is presumed that it is highly likely that it is a face image of children c and f.

Next, as illustrated in FIG. 10, in the images 1 to 3, if the children on the vertical axis are included, the probability that the children on the horizontal axis are included is scored, and the score exceeds the threshold value. Select. In this way, by selecting an image having a high score, it is possible to select more images including the target editing target than to identify the editing target only by face authentication.

In addition to the face image, the editing target specifying means 13 can estimate the editing target based on, for example, the shooting location of the image, the shooting time of the image (for example, the shooting time based on the time of program progress, etc.), and the like. be.

The editing target specifying device 1 of the present embodiment may output, for example, an image including the specified editing target to the image editing device. The image editing device is not particularly limited. According to the image editing device, for example, an image including the editing target can be edited into various modes such as an album, a document, a story, and a diary. When the image is a moving image, for example, it is possible to switch the moving image for each person who is speaking. Also, in the case of an athletic meet, it is possible to make one video that combines the people participating in the competition and the supporters.

According to the present embodiment, since the editing target can be specified from the image automatically taken by the camera or the like, for example, the editing can be easily performed according to the purpose.

[Embodiment 2]
Next, an embodiment in which the editing target specifying device 1 further includes the labeling means 15 will be described. As shown in FIGS. 4 and 5, the editing target specifying device 1 of the present embodiment further includes a labeling means 15. Other configurations are the same as those in the first embodiment.

Next, an example of the processing in the editing target specifying device of the present embodiment will be described with reference to the block diagram of FIG. 4 and the flowchart of FIG.

First, in the same manner as in the first embodiment, the image acquisition means 11 acquires the image (S11), the storage means 12 stores the image and the attribute information of the edit target (S12a), and the edit target specifying means 13 acquires the edit target. Identification of the image including (S13) and scoring by the scoring means 14 (S14) are carried out.

Next, the labeling means 15 attaches a label including the attribute information of the editing target and the information related to the image to the image including the editing target (S15). That is, the label is paired with the image including the editing target. The information regarding the image is not particularly limited, and for example, the shooting date and time, the shooting location, the weather, and the shooting state of the editing target (for example, the presence or absence of out-of-focus, whether the editing target is included in whole or only partially included, etc. The composition is good or bad, and if the editing target is a person, the facial expression, etc.). An image including the editing target and an example of the label are shown in FIG. Then, the storage means 12 further stores the image including the editing target and the label in association with each other (S12b). For example, the storage means 12 may store the image and the label separately, or may include the label as extended information in the data of the image and store the label. When the image is a moving image, the label may be attached to each of the separated images.

The editing target specifying device 1 of the present embodiment may output, for example, an image including the editing target with the label (hereinafter, also referred to as “labeled image”) to the image editing device. The image editing device includes, for example, a storage means, a labeled image extraction means, and an editing means.

Editing an image using the image editing device will be described with an example.

First, the labeled image is stored by the storage means.

Next, the labeled image extracting means extracts an image that meets a predetermined condition from the labeled image. Examples of the predetermined condition in extracting the image include the existence of a specific person, a predetermined story, and the like if the editing target is a person. The specific person may be, for example, one person or a group of a plurality of people. Examples of the predetermined story include various modes such as an album style, a document style, a story style, and a diary style. The stories in the various modes can be automatically generated by using, for example, AI (Artificial Intelligence).

For example, if the editing target is a child participating in an athletic meet, the labeled image extraction means includes an image including a child a who finishes in a foot race, an image including a child a who performs gymnastics, and a child a who tug of war. Extract images, etc. The labeled image extraction means may extract an image including a plurality of people (for example, three children a, a child b, and a child c) instead of the image including the child a.

Further, the labeled image extraction means is, for example, an event such as a person other than the specific person (in the example of the above-mentioned athletic meet, a family supporting the child a), an athletic meet such as a landscape, etc., according to the predetermined story. The entire image may be extracted.

Next, the extracted image is edited by the editing means based on predetermined conditions. Examples of the predetermined condition in editing the image include a list format of extraction results, a book format centered on still images by arranging images in chronological order, and a movie format including moving images. , The ratio of the landscape may be increased. Further, when the image is a moving image, for example, it is possible to switch the moving image for each person who is speaking.

According to the present embodiment, since the editing target can be specified from the image automatically taken by the camera or the like, for example, the editing can be easily performed according to the purpose.

[Embodiment 3]
The program of this embodiment is a program that can execute the method of each of the above-described embodiments on a computer. Further, the program of the present embodiment may be recorded on a computer-readable recording medium, for example. The recording medium is not particularly limited, and examples thereof include a read-only memory (ROM), a hard disk (HD), an optical disk, a floppy (registered trademark) disk (FD), and the like.

<Additional notes>
Some or all of the above embodiments may be described as, but not limited to, the following appendices.
(Appendix 1)
Includes image acquisition means, storage means, edit target identification means, and scoring means.
The image acquisition means acquires an image and
The storage means stores the image and the attribute information to be edited.
The editing target specifying means collates the image with the attribute information of the editing target, identifies an image including the editing target from the images, and identifies the image.
The scoring means combines a plurality of images including the editing target into a score, and selects an image whose score exceeds a threshold value.
Editing target identification device.
(Appendix 2)
The image is a moving image.
The editing target specifying means identifies an image including the editing target by dividing it at predetermined time intervals.
The edit target identification device described in Appendix 1.
(Appendix 3)
In addition, including labeling means,
The labeling means attaches a label containing the attribute information of the editing target and information about the image to the image including the editing target.
The storage means stores the image including the editing target in association with the label.
The editing target specifying device according to Appendix 1 or 2.
(Appendix 4)
The information about the image is at least one selected from the group consisting of the shooting date and time, the shooting location, the weather, and the shooting state to be edited.
The editing target specifying device according to any one of Supplementary notes 1 to 3.
(Appendix 5)
The editing target is a person
The scoring means performs the scoring using at least one selected from a group consisting of a person around the editing target, a shooting location of the image, and a shooting time of the image.
The editing target specifying device according to any one of Supplementary notes 1 to 4.
(Appendix 6)
The editing target is a person
The attribute information to be edited is at least one selected from the group consisting of a face image, an ID, a name, an age, and a gender.
The editing target specifying device according to any one of Supplementary notes 1 to 5.
(Appendix 7)
Includes an image acquisition process, a storage process, an edit target identification process, and a scoring process.
The image acquisition step acquires an image and
The storage step stores the image and the attribute information to be edited.
In the editing target specifying step, the image is collated with the attribute information of the editing target, and an image including the editing target is specified from the images.
In the scoring step, a plurality of images including the editing target are combined and scored, and an image whose score exceeds the threshold value is selected.
How to specify the edit target.
(Appendix 8)
The image is a moving image.
The editing target specifying step identifies an image including the editing target by dividing it at predetermined time intervals.
The method for identifying the editing target described in Appendix 7.
(Appendix 9)
In addition, it includes a labeling process
In the labeling step, the image including the editing target is labeled with the attribute information of the editing target and the information related to the image.
In the storage step, the image including the editing target and the label are stored in association with each other.
The method for identifying an edit target according to Appendix 7 or 8.
(Appendix 10)
The information about the image is at least one selected from the group consisting of the shooting date and time, the shooting location, the weather, and the shooting state to be edited.
The method for identifying an edit target according to any one of Supplementary notes 7 to 9.
(Appendix 11)
The editing target is a person
The scoring step performs the scoring using at least one selected from the group consisting of a person around the editing target, a shooting location of the image, and a shooting time of the image.
The method for identifying an editing target according to any one of Supplementary Provisions 7 to 10.
(Appendix 12)
The editing target is a person
The attribute information to be edited is at least one selected from the group consisting of a face image, an ID, a name, an age, and a gender.
The method for identifying an edit target according to any one of Supplementary notes 7 to 11.
(Appendix 13)
A program capable of executing the editing target identification method described in any one of Appendix 7 to 12 on a computer.
(Appendix 14)
A computer-readable recording medium on which the program described in Appendix 13 is recorded.

Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above embodiments. Various changes that can be understood by those skilled in the art can be made to the structure and details of the present invention within the scope of the present invention.

According to the present invention, it is possible to specify an editing target from an image automatically taken by a camera or the like, which is particularly useful in an event such as an athletic meet.

1 Editing target specifying device 11 Image acquisition means 12 Storage means 13 Editing target specifying means 14 Scoring means 15 Labeling means

Claims (9)

Includes image acquisition means, storage means, edit target identification means, and scoring means.
The image acquisition means acquires an image and
The storage means stores the image and the attribute information to be edited.
The editing target specifying means collates the image with the attribute information of the editing target, identifies an image including the editing target from the images, and identifies the image.
The scoring means combines a plurality of images including the editing target into a score, and selects an image whose score exceeds a threshold value.
Editing target identification device. The image is a moving image.
The editing target specifying means identifies an image including the editing target by dividing it at predetermined time intervals.
The editing target specifying device according to claim 1. In addition, including labeling means,
The labeling means attaches a label containing the attribute information of the editing target and information about the image to the image including the editing target.
The storage means stores the image including the editing target in association with the label.
The editing target specifying device according to claim 1 or 2. The information about the image is at least one selected from the group consisting of the shooting date and time, the shooting location, the weather, and the shooting state to be edited.
The editing target specifying device according to any one of claims 1 to 3. The editing target is a person
The scoring means performs the scoring using at least one selected from a group consisting of a person around the editing target, a shooting location of the image, and a shooting time of the image.
The editing target specifying device according to any one of claims 1 to 4. The editing target is a person
The attribute information to be edited is at least one selected from the group consisting of a face image, an ID, a name, an age, and a gender.
The editing target specifying device according to any one of claims 1 to 5. Includes an image acquisition process, a storage process, an edit target identification process, and a scoring process.
The image acquisition step acquires an image and
The storage step stores the image and the attribute information to be edited.
In the editing target specifying step, the image is collated with the attribute information of the editing target, and an image including the editing target is specified from the images.
In the scoring step, a plurality of images including the editing target are combined and scored, and an image whose score exceeds the threshold value is selected.
How to specify the edit target. A program capable of executing the editing target identification method according to claim 7 on a computer. A computer-readable recording medium on which the program according to claim 8 is recorded.

Images