JP7122243B2 - Image identification device, classification system, production support system, methods and programs thereof - Google Patents

Description

The present invention relates to an image identification device, a classification system, a production support system, methods and programs thereof.

In recent years, image recognition technology has been developed, and a technology for identifying a player from a game video has been proposed (for example, Patent Literature 1). This technology uses characteristics such as the player's face, uniform color, jersey number, and program super (telop) to obtain information about who the player is in the video.

JP-A-2002-236913

By the way, when identifying a person in an image, the person to be identified does not always face the front, but usually faces in various directions.

Under such circumstances, there is a demand for a technology that can identify a specific person from a video with high accuracy.

Accordingly, it is an object of the present invention to provide an image identification device, a classification system, a production support system, methods and programs thereof that can identify a specific person from a video with high accuracy.

One aspect of the present invention is an image identification device that identifies a specific person from an image, the person is recognized from the image, the position of the recognized person on the image is referred to, and the identified person on the image is identified. an orientation estimation unit for estimating an orientation of a person; a plurality of specific person identification units trained to identify the specific person for each orientation of the specific person; and the specific person estimated by the orientation estimation unit. and a control unit that selects the specific person identification unit corresponding to the orientation of the person and causes the selected specific person identification unit to identify the specific person from the video.

One aspect of the present invention is a classification system for classifying teacher data for an image identification device that identifies a specific person from a video, wherein the person is recognized from the video, and the position of the recognized person on the video is referenced. and storing the image in which the orientation of the specific person is estimated in an orientation estimation unit for estimating the orientation of the specific person on the image and a storage unit corresponding to the orientation of the specific person estimated by the orientation estimation unit. A classification system having a storage for

One aspect of the present invention includes a relay van loaded with a camera, an image identification device that identifies a specific specific person from the image of the camera, and a reporting system that reports that the specific person has passed a predetermined point. In the production support system, the relay vehicle has a position information acquisition unit that acquires position information on the course of the relay vehicle, and the image identification device identifies a person from an image of a camera mounted on the relay vehicle. an orientation estimating unit that recognizes, refers to the position of the recognized person on the image, and estimates the orientation of the specific person on the image; A plurality of learned specific person identification units and the specific person identification unit corresponding to the orientation of the specific person estimated by the orientation estimation unit are selected, and the selected specific person identification unit identifies the a control unit that identifies a specific person, the specific person identification unit refers to the orientation of the specific person estimated by the orientation estimation unit, and identifies the positional relationship of the specific person; , a tallying unit for tallying the results of the report, and the production support system stores the position information of the relay vehicle and the position of the specific person identified by the image identification device from the image of the camera loaded on the relay vehicle. The production support system includes a screen generation unit that generates a screen that visually expresses the positional relationship between the broadcast van and the specific person using the relationship and the aggregated result of the reporting system.

One aspect of the present invention is an image identification method for identifying a specific person from a video, which includes recognizing the person from the video, referring to the position of the recognized person on the video, and performing the identification on the video. estimating the orientation of a person, selecting a specific person identification unit corresponding to the estimated orientation of the specific person, causing the selected specific person identification unit to identify the specific person from the video, and identifying the specific person from the image; is an image identification method in which learning for identifying the specific person is performed for each orientation of the specific person.

One aspect of the present invention is a classification method for classifying teacher data for an image identification device that identifies a specific person from a video, wherein the person is recognized from the video, and the position of the recognized person on the video is referenced. and estimating the orientation of the specific person on the image, and storing the image with the estimated orientation of the specific person in a storage unit corresponding to the estimated orientation of the specific person.

One aspect of the present invention is a program for an information processing device that identifies a specific person from a video, recognizing the person from the video, referring to the position of the recognized person on the video, an orientation estimation unit for estimating the orientation of the specific person; a plurality of specific person identification units trained to identify the specific person for each orientation of the specific person; A program that selects the specific person identification unit corresponding to the direction of the specific person and causes the information processing device to function as a control unit that identifies the specific person from the video by the selected specific person identification unit.

One aspect of the present invention is a program for an information processing device that classifies training data for an image identification device that identifies a specific person from a video, the program for recognizing the person from the video, and identifying the recognized person on the video. An orientation estimating unit for estimating the orientation of the specific person on the image by referring to the position, and a storage unit corresponding to the orientation of the specific person estimated by the orientation estimating unit. A program that causes an information processing device to function as a storage unit that stores video.

The present invention can accurately identify a specific person from an image.

FIG. 1 is a block diagram showing a configuration example of a video production system according to the first embodiment. FIG. 2 is a diagram for explaining the first embodiment. FIG. 3 is a diagram for explaining the first embodiment. FIG. 4 is a diagram for explaining the first embodiment. FIG. 5 is a diagram for explaining the first embodiment. FIG. 6 is a block diagram showing a configuration example of the image identification device 2 in the first embodiment. FIG. 7 is a conceptual diagram when detecting a person existing in an image of an image frame at time t. FIG. 8 is a conceptual diagram when detecting a person existing in the image of the image frame at time t+1. FIG. 9 is a conceptual diagram when detecting a person existing in an image of an image frame at time t. FIG. 10 is a conceptual diagram when detecting a person existing in the image of the image frame at time t+1. FIG. 11 is a diagram for explaining a region of interest. FIG. 12 is a diagram showing an example of the likelihood calculated for each identification target (class). FIG. 13 is a diagram for explaining the first embodiment. FIG. 14 is a block diagram showing a configuration example of an image identification device according to the second embodiment. FIG. 15 is a diagram for explaining the second embodiment. FIG. 16 is a diagram for explaining the second embodiment. FIG. 17 is a block diagram of a classification system according to the third embodiment. FIG. 18 is a block diagram of the fourth embodiment. FIG. 19 is a diagram showing an example of metadata. FIG. 20 is a diagram for explaining the fourth embodiment. FIG. 21 is a block diagram of the fifth embodiment. FIG. 22 is a diagram for explaining the fifth embodiment. FIG. 23 is a block diagram showing a configuration example of a video production system according to modification 2 of the first embodiment. FIG. 24 is a diagram for explaining Modification 2 of the first embodiment. FIG. 25 is a diagram for explaining another example of the fifth embodiment.

<First embodiment>
A video production system including an image identification device and a program according to a first embodiment of the present invention will be described below with reference to the drawings.

In the first embodiment of the present invention, as an example of identifying a specific person from a video, an example of identifying a player (specific person) during a long-distance relay race from a video in which many spectators (persons) are present will be described. do. However, this is only an example, and the present invention is not limited to this example.

FIG. 1 is a block diagram showing a configuration example of a video production system according to this embodiment. The video production system comprises a plurality of cameras 1a-1d and an image identification device 2.

The cameras 1a to 1d are cameras for capturing images of the athletes during the relay race. In FIG. 1, four cameras are shown.

Here, as shown in FIG. 2, the camera 1a is a camera that photographs the player from a photographing angle that captures the face of the player from the front. In other words, it is a camera that captures an image in which the direction of the player on the image is the front. The camera 1b, as shown in FIG. 3, is a camera for photographing the player from a photographing angle that captures the player from behind. In other words, it is a camera that captures an image in which the back of the player on the image is oriented. The camera 1c, as shown in FIG. 4, is a camera that takes an image of the player from a shooting angle that captures the player's face from the left side. In other words, it is a camera that captures an image in which the direction of the player on the image is to the left. The camera 1d, as shown in FIG. 5, is a camera that takes an image of the player from a shooting angle that captures the player's face from the right side. In other words, it is a camera that captures an image in which the orientation of the player on the image is to the right. However, one camera may be used to photograph the player from various shooting angles.

FIG. 6 is a block diagram showing a configuration example of the image identification device 2 in this embodiment. The image identification device 2 includes an orientation estimation section 21, a control section 22, and image identification sections 23a to 23d.

The orientation estimating unit 21 receives an image from one of the cameras 1a to 1d. The orientation estimation unit 21 recognizes a person from the input image, refers to the position of the recognized person on the image, and estimates the orientation of the player (specific person) on the image.

Here, as for the recognition of the person on the image, it is sufficient if the person can be recognized, and it is not necessary to be able to recognize the attributes of each person. In other words, it is sufficient to be able to recognize existing objects such as trees and cars in an image and persons to be distinguished from each other. In addition, it is not necessary to identify the attributes (name, team to which the person belongs, etc.) that identify the recognized person.

A specific example of orientation estimation by the orientation estimation unit 21 will be described. FIG. 7 is a conceptual diagram when detecting a person existing in an image of an image frame at time t. In FIGS. 7 and 8, those enclosed in squares are detected as persons, and all objects that can be identified as persons are detected regardless of whether they are athletes or spectators. Conventional image recognition technology can be used to recognize people in these images.

Next, the orientation of the player (specific person) on the image is estimated by the orientation estimation unit 21, and is identified from the position of the recognized person on the image. As one estimation method, the orientation of the player (specific person) on the video is estimated by referring to the temporal change in the position of the recognized person on the video. For example, in sports such as marathons and long-distance relay races, there are many cases in which the camera moves while maintaining a certain distance from the athlete so that the athlete is captured at a certain size. In this case, the athletes move with the camera, but the spectators tend to stay where they are.

For example, in a video shot from the front where a player's face can be seen, when focusing on a specific spectator, the video of the image frame at time t is as shown in FIG. 7, the video of the image frame at time t+1 is That particular spectator remains in place, resulting in an image such as that shown in FIG. In other words, although the positions of the athletes and the camera are maintained as time progresses, the specific spectator moves backward in the video because the distance between the specific spectator and the camera increases. In such an image, it can be assumed that the player is facing the front.

On the other hand, when focusing on a specific spectator in the video shot from behind the player, if the video of the image frame at time t is as shown in FIG. The spectator stays in place, and the image shown in FIG. 10 is displayed. In other words, although the positions of the players and the camera are maintained as time progresses, the distance between the specific spectator and the camera decreases, so the specific spectator moves forward in the image. In such an image, it can be estimated that the player is facing backward (back).

In this way, by performing tracking processing that associates the same person with the persons in the frame images obtained at different times, it is possible to estimate the movement vector of each person relative to the camera. By using this movement vector, the orientation of the player can be estimated. The movement vector is represented by a difference vector of the coordinate positions of the person in the associated images obtained at different times.

Specifically, in a given image, the amount of movement of the movement vectors of a small number of persons is small, the direction of the movement vectors of a large number of persons is backward in the image (the direction of moving away), and the amount of movement of the movement vectors of the large number of persons is is greater than a predetermined threshold value, it is estimated that the video was shot from the front where the player's face can be seen. That is, it is estimated that the player is facing forward.

On the other hand, in a certain image, the amount of movement of the movement vectors of a small number of persons is small, the direction of the movement vectors of many persons is forward (approaching direction) in the image, and the amount of movement of the movement vectors of the large number of persons is predetermined. If the amount of movement is larger than the threshold value, it is estimated that the image is shot from behind the player. That is, it is estimated that the orientation of the player is the back.

Using such a movement vector, the orientation of the player is estimated.

In the above example, the case where the player faces the front and the back has been described, but the same method can be used to estimate when the player faces left or right. For example, when the player faces left, the temporal movement vectors of spectators other than the player increase in the right direction. On the other hand, when the player faces right, the spectators other than the player have larger temporal movement vectors in the left direction. In this way, the player's left-right orientation can be estimated.

Also, the method of estimating the orientation of the player (specific person) described above is an example, and an appropriate method is selected according to the type of video. For example, in videos of field events such as volleyball and soccer, contrary to sports such as marathons and long-distance relay races, the athletes move while the spectators remain in place (for example, sitting in the spectator seats). There are many. In such a game, the orientation of the player can be estimated by using the direction of the movement vector with a large amount of movement.

Alternatively, the direction of the player may be estimated from the position of the identified person on the video without using the movement vector. For example, in a video shot from the front side where the net is reflected in a volleyball game, three players (from the left) front left (FL), front center (FC), front right (FR)) stands, and three players (from the left, back left (BL), back center (BC), back light (BR)) stand as the rear guard (the side farthest from the net). is detected. In such a case, it can be assumed that the player is facing forward. On the other hand, in the video with the net behind, three players (from the left, front left (FL), front center (FC), and front right (FR)) are standing in the vanguard (the side closest to the net). , and three players (back left (BL), back center (BC), and backlight (BR) from the left) standing as rear guards (far side from the net) are detected. In such a case, it can be assumed that the player is facing the back.

Thus, the method of estimating the orientation of the player (specific person) is an example, and an appropriate method is selected depending on the type of video.

The control unit 22 receives the estimation result of the orientation estimation unit 21, selects one of the image identification units 23a to 23d corresponding to the estimated orientation of the player, and outputs an image to the selected image identification unit 23a to 23d. .

The image identification units 23a to 23d are image identification units that have undergone learning for identifying a player for each orientation of the player.

In the first embodiment, the image identification unit 23a is an image identification unit that has undergone learning for identifying a player using a video image of the player taken from the front. In other words, the training data is a video in which the player is facing the front (the direction in which the player's face can be seen), and the player is trained as training data. This is an image identification unit used to identify the player from the video. It should be noted that the training data does not consist only of videos in which the player is facing straight ahead (the direction in which the player's face can be seen), but rather in which the player is facing in a left-right direction that is close to the front (for example, left and right). 45 degree) video may be added to the teacher data.

Further, the image identification unit 23b is an image identification unit that has undergone learning for identifying a player using a video image of the player taken from behind. In other words, a video in which the player is facing backwards (a direction in which the player's face cannot be seen) is learned as training data, and a video in which the player is facing backwards (a player's face cannot be seen) Direction) This is an image identification unit used to identify the player from the video. In addition, the training data does not consist only of images in which the player faces the back (a direction in which the player's face cannot be seen) in the video, but in which the player faces the left and right directions close to the back (for example, 45 degrees to the left and right) may be added to the teacher data.

Also, the image identification unit 23c is an image identification unit that has undergone learning for identifying a player using an image captured from the left side of the player. In other words, the training data is a video in which the player is facing left (the direction in which the player's left face can be seen). is visible) This is an image identification unit used to identify the player from the video. It should be noted that not only images in which the player is facing left on the video (the direction in which the player's left face is visible) are used as training data, but the player is facing in a horizontal direction close to the left (for example, left and right 45 degrees). degree) You may add an image|video to teacher data.

Also, the image identification unit 23d is an image identification unit that has undergone learning for identifying a player using an image taken from the right lateral side of the player. In other words, the training data is a video in which the player is facing right (the direction in which the player's right face can be seen). is visible) This is an image identification unit used to identify the player from the video. It should be noted that not only the video in which the player is facing right (the direction in which the player's right face is visible) is used as training data, but rather the player is facing in a left-right direction close to the right (for example, left and right 45 degrees). degree) You may add an image|video to teacher data.

As a learning method of the image identification units 23a to 23d, there are pattern matching, machine learning using techniques such as deep learning, and the like.

Here, the identification targets identified by the image identification units 23a to 23d are, for example, a player (specific person) in the video and attributes of the player. A player's attributes include not only the name and age of each player, but also the team or university to which the player belongs, role (pitcher, fielder, etc. in the case of baseball, offense or defense in the case of soccer), etc. . In the following description, for the sake of convenience, an object to be identified may be referred to as a class. A category is a type of feature used to identify a player and attributes of the player. Representative categories include, for example, the face of the player, the uniform worn by the player, the uniform number, the sash, and the letters written on the bib.

In order to correctly identify players and their attributes from a video in which many people are present, it is important which category should be focused on for identification. For example, in order to identify the position of a player in an image or the team to which the player belongs, the important categories are the color and pattern of the uniform, the characters written on the sash and the bib. On the other hand, if it is desired to specify even individual players (names, etc.), the category of each player's face is important. Furthermore, what categories are used and what weights are given to those categories differ from sport to sport. For example, if the competition is a marathon or a relay road race, there are many people such as spectators other than the athletes in the video. There is a spectator who wears , and it is not possible to identify it with high accuracy. On the other hand, when the game is baseball, soccer, etc., the positions of the spectators in the video are almost fixed, so it is relatively easy to exclude the spectators from the identification target before identification. Just look at the uniforms the players are wearing.

For the above reasons, the image identification units 23a-23d may be configured to store different weight parameters for each category for each sport or for each player to be identified.

The image identification device 2 described above includes, for example, a processor such as a CPU (Central Processing Unit), a memory as temporary storage, and a nonvolatile storage device (EEPROM or hard disk). A processor such as a CPU functions as the orientation estimation unit 21, the control unit 22, and the image identification units 23a to 23d by reading the program stored in the storage device into the memory and executing the program.

Next, the operation of the video production system including the image identification device and program according to the embodiment of the present invention will be described. In the following description, an example will be described in which the competition is an Ekiden, the specific person to be identified is a player, and the identification target is the position of the player and the name of the university to which the player belongs.

First, the cameras 1a to 1d shoot images including spectators and athletes. In this example, the camera 1a is a camera for photographing the player from a photographing angle that captures the face of the player from the front. The camera 1b is a camera for photographing the player from a photographing angle that captures the player from behind. The camera 1c is a camera that takes an image of the player from a shooting angle that captures the player's face from the left side. The camera 1d is a camera that takes an image of the player from a shooting angle that captures the player's face from the right side.

One of these cameras 1a to 1d is selected and used for broadcasting, for example. Here, it is assumed that the image of the camera 1a is used first. Therefore, the image input to the direction estimation unit 21 is the image of the camera 1a.

The direction estimation unit 21 inputs the image of the camera 1a and estimates the direction of the player on the image. In this example, the image of the image frame at the time t of the image of the camera 1a is shown in FIG. It is assumed that the image of the image frame at the time t+1 of the image of the camera 1a is shown in FIG. Then, as described above, focusing on a specific spectator, the positions of the players and the camera are maintained even if the time advances, but the distance between the specific spectator and the camera increases. moves backward on the image. In such a video, the orientation of the player is presumed to be frontal. The estimated result (the player is facing forward) is output to the control unit 22 .

The control unit 22 receives the estimation result of the orientation estimation unit 21 (the orientation of the player is the front), and selects an image identification unit for identifying the player and the attributes of the player from any of the image identification units 23a to 23d. In this example, the estimation result of the orientation estimating unit 21 indicates that the player is facing forward, so the image identification unit 23a for the player facing front is selected. Then, the image of the camera 1a is output to the image identifying section 23a.

The image identification unit 23a identifies the player and the attributes of the player from the video of the camera 1a. First, the image identification unit 23a detects a person from the image of the frame. A method for detecting a person is not limited. In FIG. 7, the objects enclosed in squares are detected as persons, and those that can be identified as persons are detected regardless of whether they are athletes or spectators. Since the orientation estimation unit 21 has already identified the persons in the video, the identification of these persons may take over the processing of the orientation estimation unit 21, and the identification of the persons itself may be omitted. can.

Next, as shown in FIG. 11, the image identification unit 23a sets the distance of a straight line connecting the top of the head to the midpoint of the neck of the detected person as L, and places a large image of L×2L at a position downward from the center of the neck of the person. set the region of interest for Then, the likelihood (reliability) that the detected person belongs to an identification target (class) learned in advance, that is, to a specific player is calculated. The image identifying unit 23a determines a person whose calculated likelihood exceeds a predetermined threshold and has the highest likelihood as a person belonging to the class (specific player).

FIG. 12 is a diagram showing an example of the likelihood calculated for each identification target (class). In the example of FIG. 12, in addition to the identified player's college name, there are also classes for police motorcycles and spectators in order to improve accuracy. In the example of FIG. 12, person A is most likely to be a spectator, person B is most likely to be a university Y athlete, person C is most likely to be a university X athlete, and person D is most likely to be an athlete. is most likely to be a player from University Z. Here, if the threshold value is 0.7, the image identification unit 23a determines that person A belongs to the audience class, person B belongs to the university Y class, person C belongs to the university X class, person D belongs to the university class, and person D belongs to the university class. It is determined that it belongs to the Z class. Then, the position information of the person B on the image (for example, coordinate information specifying the attention area) and the university name Y are output. Similarly, the position information of the person C on the image (for example, coordinate information specifying the attention area) and the university name X are output. Similarly, the position information of the person D on the image (for example, the coordinate information specifying the attention area) and the university name Z are output. Note that the person A is not a player, so he is excluded from the output targets, but this does not prevent him from being output.

In this example, the player (class) is identified from the video of the player captured at a shooting angle that captures the player's face from the front using the image identification unit 23a specialized for identification when the player faces the front. Therefore, the player (specific person) can be identified with higher accuracy than image recognition that does not consider the direction of the player.

In this way, players and their attributes are identified from the video. The identified player's position information and the player's attribute (university name in the above example) can be used for video processing. For example, as shown in FIG. 13, an image of the identified player's college name may be computer-graphically superimposed on each identified player. If such a form is adopted, video production does not need to be visually confirmed to identify players, and video production can be automated.

Furthermore, in the Ekiden, runners are determined for each section, so it is possible to limit the classes for each section to identify the runners who will run in that section. Furthermore, if the university name of the identified athlete can be identified, the name of the athlete who runs the section can also be identified, and the identified athlete name can be displayed above the athlete on the video.

Subsequently, it is assumed that time has passed and the image for broadcasting has been switched to the image of the camera 1b. In this case, the image input to the direction estimation unit 21 is the image of the camera 1b.

The orientation estimation unit 21 inputs the image from the camera 1b and estimates the orientation of the player on the image. In this example, the image of the image frame at the time t of the image of the camera 1b is shown in FIG. 10 shows the image of the image frame at time t+1 of the image of the camera 1b. Then, as described above, focusing on a specific spectator, the positions of the athletes and the camera are maintained even if the time advances, but the distance between the specific spectator and the camera is shortened. will move forward on the image. In such a video, it is assumed that the player is facing backwards (back). The estimated result (the player is facing the back) is output to the control unit 22 .

The control unit 22 receives the estimation result of the direction estimation unit 21 (the player is facing the back), and selects an image identification unit for identifying the player and the attributes of the player from any of the image identification units 23a to 23d. In this example, the estimation result of the orientation estimating unit 21 indicates that the player is facing the back, so the image identification unit 23b for the player facing the back is selected. Then, the image of the camera 1b is output to the image identifying section 23b.

The image identification unit 23b identifies the player and the attributes of the player from the image of the camera 1b. First, the image identification unit 23b detects a person from the image of the frame. A method for detecting a person is not limited. FIG. 9 is a conceptual diagram when detecting a person existing in an image of an image frame at time t. In FIG. 9, the objects enclosed in squares are detected as persons, and those that can be identified as persons are detected regardless of whether they are athletes or spectators.

In the same manner as described above, the image identifying unit 23b sets the distance of the straight line connecting the top of the head to the midpoint of the neck of the detected person as L, and the position downward from the center of the neck of the person as shown in FIG. A region of interest with a size of 2L is set. Then, the likelihood (reliability) that the detected person belongs to an identification target (class) learned in advance is calculated. The image identifying unit 23b determines a person whose calculated likelihood exceeds a predetermined threshold and has the highest likelihood as a person belonging to the class.

In this example, the player (class) is identified from the video of the player captured at a shooting angle that captures the player from the back using the image identification unit 23b that specializes in identifying when the player is facing the back. , the player (specific person) can be identified with high accuracy compared to image recognition that does not consider the direction of the player.

Depending on the determination of the orientation by the orientation estimation unit 21, the orientation of the player (specific person) may not be determined. For example, this is the case when the player is not facing straight ahead but slightly to the left. In this case, the direction estimation unit 21 may output a determination result that the player is facing forward and a determination result that the player is facing left. In such a case, the control unit 22 selects both the front image identification unit 23a and the left image identification unit 23c, and causes the image identification unit 23a and the image identification unit 23c to perform image identification. can be

Further, the orientation estimation unit 21 determines the orientation of the player (specific person) based on the likelihood of the orientation (for example, the likelihood of facing forward = 0.7, the likelihood of facing left = 0.3, etc.). You may make it select the image identification part for.

In the first embodiment, the direction of the player (specific person) in the video is estimated from the input video, and an image recognition unit specialized in identifying the direction of each player (specific person) recognizes the player from the video. Since the (specific person) is identified, the player (specific person) can be identified with higher accuracy than image recognition that does not consider the orientation of the player (specific person).

<Modification 1 of the first embodiment>
Next, Modification 1 of the above-described first embodiment will be described.

In the above-described first embodiment, when the orientation estimation unit 21 estimates the orientation of the player, the orientation of the player is estimated from the difference from the recognized movement vector of the person.

Therefore, in Modification 1 of the first embodiment, the recognized movement vector of the person is used to distinguish between the spectator (person) and the player (specific person).

Specifically, the orientation estimation unit 21 associates and stores the movement amount of the movement vector and the person calculated for each frame. Then, information in which the movement amount of the movement vector and the person calculated for each frame are associated is output to the control unit 22 .

The control unit 22 receives the direction estimation result of the direction estimation unit 21, and selects an image identification unit for identifying the player and the attributes of the player from any of the image identification units 23a to 23d. Then, to the selected image identification units 23a to 23d, information in which the amount of movement of the movement vector and the person calculated for each frame are associated is output together with the image.

The image identification units 23a to 23d regard persons whose movement amount of the movement vector is equal to or greater than a predetermined value as spectators, based on information in which the movement amount of the movement vector and the person calculated for each frame are associated with each other, and The person is excluded from the identification target (specific person) candidates, and the remaining person is identified as the identification target (specific person).

In this way, the spectator (person) and the player (specific person) can be identified with high accuracy, and as a result, the player (specific person) to be identified can be identified with high accuracy.

<Modification 2 of the first embodiment>
Next, Modification 2 of the above-described first embodiment will be described.

In the first embodiment described above, the cameras 1a to 1d are basically described as cameras that move together with the players. However, some cameras do not move with the player and are fixed (eg fixed point cameras). Therefore, in Modified Example 2 of the first embodiment, a case where the camera is a fixed-point camera that does not move with the player will be described.

FIG. 23 is a block diagram showing a configuration example of a video production system according to modification 2 of the first embodiment. A video production system according to Modification 2 of the first embodiment includes a camera 100, an orientation estimation section 101, a control section 22, and image identification sections 23a to 23d.

Modification 2 of the first embodiment differs from the first embodiment in that the camera 100 is a fixed-point camera that is installed at a certain location and the shooting direction is also fixed, and that the orientation estimation unit 101 The estimation method differs from that of the orientation estimation unit 21 .

Here, the camera 100 is installed at a position corresponding to the U-shaped vertex of the road that curves into a U-shape, and the center of the photographing direction of the camera 100 is from the U-shaped vertex toward the U-shaped center. It is assumed that the angle of view of is capable of photographing the entire U-shaped road.

Then, the camera 100 can shoot an image as shown in FIG. 24, for example. Specifically, it is an image in which the player moves from a position (point A) in the upper left direction of the U-shaped curve to point B, and reaches point C, which is the top of the U-shaped curve. Further, the image moves from point C to point D in the upper right direction, and finally reaches point E in the upper right direction.

In the case of such an image, since the camera 100, which is a fixed-point camera, does not move with the player, a situation opposite to that of the first embodiment described above occurs. That is, in the video captured by the camera 100, the player moves, but the spectator stays in place. Therefore, the player's movement can be detected between frames at different times, but the spectator's movement cannot be detected. Furthermore, the direction of movement (movement vector) differs between persons whose movements can be detected.

Specifically, with reference to FIG. 24, the movement of the spectators cannot be detected at the point A, but the movement vector of the player in the downward direction of the screen can be detected. At the point B, the movement of the spectators cannot be detected, but the movement vector of the player in the lower right direction of the screen can be detected. At point C, the movement of the spectators cannot be detected, but the movement vector of the player in the right direction of the screen can be detected. At the point D, the movement of the spectators cannot be detected, but the movement vector of the player in the upper right direction of the screen can be detected. At the point E, the movement of the spectators cannot be detected, but the movement vector of the player in the upward direction of the screen can be detected.

In this way, the direction estimating unit 101 estimates the direction of the player in the video captured by the camera 100 in view of the particularity of the video captured by the camera 100 .

Specifically, the direction estimation unit 101 detects the movement vector of the player in the downward direction of the screen in the video in which the player is present at the point A, and estimates that the player is facing forward.

In addition, the orientation estimation unit 101 detects the movement vector of the player in the lower right direction of the screen in the video in which the player is present at the point B, and compares the direction of the vector with a predetermined threshold. Then, compared with a predetermined threshold, if the direction of the vector is closer to the downward direction, it is estimated that the player is facing forward. On the other hand, if the direction of the vector is closer to the right than the predetermined threshold, then the player's orientation is assumed to be lateral to the right.

In addition, the direction estimation unit 101 detects the movement vector of the player in the right direction of the screen in the video in which the player is present at the point C, and estimates that the player is facing rightward (rightward).

In addition, the direction estimation unit 101 detects the movement vector of the player in the upper right direction of the screen in the video in which the player is present at the point D, and compares the direction of the vector with a predetermined threshold. Then, if the direction of the vector is closer to the upward direction than a predetermined threshold, it is estimated that the player is facing the back. On the other hand, if the direction of the vector is closer to the right than the predetermined threshold, then the player's orientation is assumed to be lateral to the right.

In addition, the orientation estimation unit 101 detects the movement vector of the player in the upward direction of the screen in the video in which the player is present at the point E, and estimates that the player is facing the back.

Then, the control unit 22 receives the estimation result of the orientation estimation unit 21, selects one of the image identification units 23a to 23d corresponding to the estimated orientation of the player, and provides the selected image identification unit 23a to d with the camera 100 images are output.

Note that detection of the motion vector by the direction estimation unit 101 is performed for a predetermined number of frames of the video, for example, every five frames.

In this way, even if the image is captured by a camera whose installation position and shooting direction are fixed, the orientation of the person can be estimated with high accuracy.

Furthermore, in the description of the modification 2 of the first embodiment, the front image identification unit 23a, the back image identification unit 23b, the left side image identification unit 23c, and the right side image identification unit 23c In addition to these, an image identification unit that learns from teacher data an image of a person facing a direction of 45 degrees between the front and the right side, and an image identification unit 23d Prepare an image identification unit that has learned images of a person facing a 45-degree direction between the side and the side using teacher data, and images of players at points B and D are processed by those image identification units. You can identify it. By doing so, further recognition accuracy can be obtained.

Further, as in the first modification of the first embodiment described above, it is also possible to distinguish between spectators (persons) and players (specific persons) using the recognized movement vectors of the persons.

As described above, in images captured by a fixed-point camera, a person whose movement can be detected between predetermined frames is an athlete, and a person whose movement cannot be detected is a spectator. Using such a property, the image identification units 23a to 23d set the movement amount of the movement vector to a predetermined value based on the information in which the movement amount of the movement vector and the person calculated for each frame are associated with each other. The following persons are set as spectators, the persons are excluded from candidates for identification (specific persons), and the remaining persons are identified as identification objects (specific persons).

In this way, the spectator (person) and the player (specific person) can be identified with high accuracy, and as a result, the player (specific person) to be identified can be identified with high accuracy.

<Second Embodiment>
The second embodiment will explain a video production system including an image identification device and a program for identifying the positional relationship of identification objects based on the size of the identified identification objects on the video.

In a two-dimensional image, objects in the foreground appear larger, and objects in the distance appear smaller. This is also the case when shooting a competition. For example, in marathons and relay races, athletes are often photographed from the front. In this case, the player running in front appears larger in the image, and the player in the back appears smaller in the image.

In the second embodiment, using such properties, the positional relationship of the identified identification objects is identified.

FIG. 14 is a block diagram showing a configuration example of the image identification device 2 according to the second embodiment. The image identification device 2 in the second embodiment includes a positional relationship identification section 24 in addition to the image identification device 2 in the first embodiment.

The positional relationship identification unit 24 identifies the positional relationship of each identification target from the size of the identified identification target on the image. In the first embodiment described above, in order to identify an identification target, as shown in FIG. 11, for a detected person, the distance of a straight line connecting the top of the head to the neck midpoint is L, and the distance from the neck center of the person to the downward direction is L. A region of interest of L×2L was set in .

The positional relationship identification unit 24 focuses on the size of this attention area as an index of the size of the identified identification target on the video.

Furthermore, the positional relationship identification unit 24 focuses on the direction of the player (specific person) estimated by the direction estimation unit 21 in addition to the size of the region of interest.

For example, the example in FIG. 15 is an image when the player faces the front. In this image, the attention area of person D is the largest and the attention area of person C is the smallest. Therefore, the order of the athletes is Person D (athlete of University Z), Person B (athlete of University Y), and Person C (athlete of University X), and it can be identified that they are running in that order. On the other hand, as shown in FIG. 16, in the case of an image in which the player faces the back, even if the position of the person is the same as in FIG. 15, the smaller the attention area, the higher the player's ranking. , the order of the athletes is person C (athlete of university X), person B (athlete of university Y), and person D (athlete of university Z) in that order, and it can be identified that they are running in that order.

Thus, by considering the size of the identified player on the video and the direction of the identified player, the positional relationship of the identified player on the video can be identified regardless of the shooting direction.

In the above example, the positional relationship is identified by focusing on the attention area of the identified player, but the invention is not limited to this. For example, it may be the size of the player in the video of the player himself. As another method, there is a method of paying attention to the equipment commonly worn by each player. For example, bibs and the like are suitable because they have a common size among players.

Furthermore, as an application example of this example, relative distances can also be obtained from the magnitudes used to identify the positional relationship. The camera used for filming the competition can acquire the photographing information such as the angle of view together with the photographed image. Therefore, if the relationship between the size and the distance used to identify the positional relationship is learned in advance in association with the angle of view, etc., it is possible to calculate the distance between the players (identification targets).

Furthermore, it is also possible to predict changes in the positional relationship of the players using the obtained distances between the players. For example, if the distance between players decreases over time, a trailing player may overtake a leading player. In such a case, by issuing a warning, it is possible to inform the production side of the change in the positional relationship of the players.

<Third Embodiment>
A third embodiment will be described.

The third embodiment is a teacher data classification system for making the image identification units 23a to 23d in the first embodiment learn.

The orientation estimation unit 21 described above recognizes a person from an input image, refers to the position of the recognized person on the image, and estimates the orientation of the player (specific person) on the image. By using this direction estimation unit 21, it is possible to classify the teacher data of the images to be learned by the image identification units 23a to 23d according to the player's direction.

FIG. 17 is a block diagram of a classification system according to the third embodiment.

The orientation estimator 21 performs the same operation as the orientation estimator 21 described in the first embodiment. In this example, as in the first embodiment described above, any one of front, back, left, and right is estimated as the orientation of the player (specific person) from the video.

The storage unit 30 receives the estimation result of the orientation estimation unit 21, and stores the image in any one of the front, rear, left, and right teacher data storage folders 31a to 31d.

By doing so, it is possible to automatically classify teacher data for learning the image identification units 23a to 23d.

<Fourth Embodiment>
A fourth embodiment will be described.

FIG. 18 is a block diagram of the fourth embodiment.

The fourth embodiment includes a metadata generation section 40, a metadata storage section 41, and a scene search section 42 in addition to the first and second embodiments.

The metadata generation unit 40 generates the identification results of the image identification units 23a to 23d (the attributes of the player (specific person), the position of the player (specific person) on the video, etc.) and the identification results of the positional relationship identification unit 24 (player Metadata that associates (the positional relationship of (a specific person), etc.) with the images of the cameras 1a to 1d is generated.

Each of the image identification units 23a to 23d and the positional relationship identification unit 24 outputs identification results at predetermined time intervals (for example, 1-second intervals). Therefore, the metadata generation unit 40 associates the identification results received at predetermined time intervals, the image identification information for identifying the identified images, and the time code of the images corresponding to the identification results, and stores them in the metadata storage unit 41. store in The metadata storage unit 41 may store not only metadata but also each video corresponding to the metadata.

Specifically, the image identification units 23a to 23d output identification results for each player (specific person) as shown in FIG. Furthermore, since the coordinates of the neck (midpoint of the neck) and the face length (L) of the recognized player (specific person) are calculated when setting the attention area, it is also possible to output this information. . Further, the positional relationship identification unit 24 can output the recognized positional relationship of each player (specific person). Therefore, the metadata generation unit 40 determines the degree of reliability that each identified player (specific person) belongs to each university, the coordinates of the neck (midpoint of the neck) and the length of the face ( L), the rank of the player, the video identification information of the video that has been identified, and the time code of the video are associated with each other and stored in the metadata storage unit 41 . In addition, the reliability of each identified player (specific person) belonging to each university, the coordinates and face length (L) of the neck (neck midpoint) of each identified player (specific person), and the player's The order, the video identification information of the video to be identified, and the time code of the video are hereinafter referred to as each index of metadata.

FIG. 19 is a diagram showing an example of metadata. In the example of FIG. 19, as indexes, the confidence that each identified player (specific person) belongs to each university, the coordinates of the neck (neck midpoint) of each identified player (specific person), and the length of the face ( L), the ranking of the player, the video identification information of the video that was the target of the identification, and the time code, and the reliability of each identified player (specific person) belonging to each university. The coordinates of the neck (midpoint of the neck) and face length (L) of each player (specific person), the ranking of the player, the video identification information of the video that has been identified, and the time code of the video. It is an example that is associated and configured with metadata. For example, for person B, the degree of reliability belonging to each university (university X=0.2, university Y=0.9, university Z=0.1, . . . ) and the coordinates of the neck (=(781,450 )), face length (L) (=5), rank (=2), video identification information (camera 1), and time code (=02:10:01) are associated, and metadata is configured. In addition, in the example of FIG. 19, among the identified persons, spectators other than the players are excluded.

The scene search unit 42 searches for scenes that meet predetermined search conditions using metadata stored in the metadata storage unit 41 . The search condition may be any condition that enables a search from each index of the metadata.

For example, when the scene search unit 42 is set with “a scene in which athletes from university X and university Y are shown” as a search condition, the scene search unit 42 searches the index of the metadata stored in the metadata storage unit 41 . Among them, a time code that has the same video identification information and that satisfies the condition that the reliability of "university X" and "university Y" is greater than a predetermined threshold value (for example, 0, 7) is searched. Then, the video scene corresponding to the time code of the corresponding video identification information is extracted. In the example of FIG. 19, a scene that satisfies the condition of "a scene in which athletes from university X and university Y are shown" is the time code "02:10:01" of "video from camera 1" as shown in FIG. ” is the scene. Therefore, the scene of the time code "02:10:01" of the "camera 1 video" is extracted from the "camera 1 video", and the scene is displayed. In the example of FIG. 20, the condition is "a scene in which athletes from University X and University Y are shown". gives a good example. However, if the search condition is "a scene in which only university X and university Y players are shown", it is possible to search for scenes in which only university X and university Y players are shown.

As a search condition, it is also possible to set conditions such as "a scene in which a player from University Z overtakes a player from University Y". In this case, in addition to the index described above, the ranking index is referred to, and the scenes before and after the time code where the ranking of the highly reliable person of University Z and the ranking of the highly reliable person of University Y are exchanged are retrieved. Good luck.

In the fourth embodiment, by converting the identification result into video metadata, it is possible to search for video scenes that meet specific conditions.

<Fifth Embodiment>
A fifth embodiment will be described.

The fifth embodiment describes a production support system using the image identification device for identifying the positional relationship of identification targets, the rangefinder system, and the marathon bulletin online system in the second embodiment.

First, the rangefinder system and the marathon bulletin online system will be explained.

A range finder is provided on the broadcast vehicle loaded with cameras. This rangefinder obtains the distance of the relay vehicle on the marathon course by using the distance measured by the sensor installed in the relay vehicle and the position of the relay vehicle obtained from a positioning system using GPS. can do

On the other hand, the marathon bulletin online system allocates personnel to predetermined points on the marathon course, confirms the athletes (university names) who pass the points, and reports to the counting system. The tallying system tallies the passage status of the athlete (university name) reported from each point, and estimates the approximate position and rank of each athlete (university name). Such a marathon bulletin online system has been used conventionally.

FIG. 21 is a block diagram of the fifth embodiment.

In FIG. 21, 51 is the No. 1 relay vehicle loaded with the camera 1a, 52 is the No. 2 relay vehicle loaded with the camera 1b, 53 is the No. 3 relay vehicle loaded with the camera 1c, and 54 is the camera 1d. 55 is an image identification device, 56 is a marathon bulletin online system, 57 is a positional relationship specifying unit, and 58 is a display.

The relay car 51 to the relay car 54 are provided with the range finder described above. This rangefinder calculates the distance of the relay vehicle on the marathon course by using the distance measured by the sensor installed in the relay vehicle and the position of the relay vehicle obtained from the positioning system using GPS. and output to the positional relationship specifying unit 57 . In addition, the images captured by the cameras 1 a to 1 d loaded on the relay vans 51 to 54 are output to the image identification device 55 .

The image identification device 55 is the image identification device described in the second embodiment. The image identification result and the positional relationship are output to the positional relationship specifying unit 57 .

In the marathon bulletin online system 56, the tally system tallies the passage status of the athlete (university name) reported by the personnel stationed at the predetermined points on the marathon course, and outputs the result to the positional relationship identification unit 57. .

Next, the operation of the fifth embodiment will be described using a specific example.

First, in this operation example, an Ekiden is assumed in which athletes from a total of 12 schools, from AA University to JJ University, run. At present, players from AA University to JJ University are running in 6 sections with a section of 20 km, and relay vans 51 to 54 are running in 6 sections.

The relay vehicle 51 (car No. 1) is traveling 19.5 km in the 6th section according to the rangefinder. The camera 1a loaded on the broadcast van 51 (car No. 1) takes an image of the player running in the lead. Relay van 52 (car No. 2) is traveling 19.2 km in Section 6 according to the rangefinder. Then, the camera 1b loaded on the broadcast van 52 (car No. 2) photographs the running player. Relay van 53 (car No. 3) is traveling 18.6 km in Section 6 according to the rangefinder. The camera 1c loaded on the relay van 53 (car No. 3) photographs the running player. Relay van 54 (car No. 4) is traveling 17.8 km in Section 6 according to the rangefinder. A camera 1d loaded on the relay van 54 (car No. 4) photographs the running player. , the distances at which the relay cars 51 to 54 are located, which are obtained from the rangefinders, are output to the positional relationship specifying unit 57 . Also, the images captured by the cameras 1 a to 1 d of the relay cars 51 to 54 are output to the image identification device 55 .

The image identification device 55 identifies the positional relationship between the player and the player on the video from the video captured by the camera 1a. It is assumed that the players identified here are only players from AA universities. This identification result is output to the positional relationship specifying unit 57 .

The image identification device 55 identifies the positional relationship between the player and the player in the video captured by the camera 1b. The athletes identified here are the EE University athletes and the CC University athletes, and the positional relationship is such that the EE University athletes are at the top, followed by the CC University athletes. This identification result is output to the positional relationship specifying unit 57 .

The image identification device 55 identifies the positional relationship between the player and the player on the video from the video captured by the camera 1c. The players identified here are BB University players, II University players, and GG University players, and their positional relationship is in the order of BB University players (top), II University players, and GG University players. Assume that there is This identification result is output to the positional relationship specifying unit 57 .

The image identification device 55 identifies the positional relationship between the player and the player on the video from the video captured by the camera 1d. Assume that the players identified here are only players from FF University. This identification result is output to the positional relationship specifying unit 57 .

It is assumed that the marathon bulletin online system 56 receives the following reports from each point. The order in which the athletes are listed indicates the order in which they passed each point.
(1) 19.5km point AA athletes pass (2) 19.0km point AA University athletes, EE University athletes and CC University athletes pass (3) 18.5km point AA University athletes and EE University athletes , CC University athletes, KK University athletes, BB University athletes, II University athletes, GG University athletes passing (4) 18.0km point AA University athletes, EE University athletes, CC University athletes , KK University athletes, BB University athletes, II University athletes, GG University athletes, DD University athletes, LL University athletes, HH University athletes passing (5) 17.5 km point AA University athletes, EE University Players, CC University Players, KK University Players, BB University Players, II University Players, GG University Players, DD University Players, LL University Players, HH University Players, FF University Players, JJ Passage of University Athletes The positional relationship specifying unit 57 collects the distances at which the broadcast vans 51 to 54 are located, the university athletes and positional relationships (ranking) identified by the image identification device 55, and the marathon bulletin online system 56. In response to the result, the player running position screen to be displayed on the display 58 is generated.

The positional relationship specifying unit 57 first determines that each relay car is traveling at the following positions from the rangefinders of each relay car 51 to 54 .
(1) Relay van 51 (car No. 1) travels 19.5 km in District 6 (2) Relay van 52 (car No. 2) travels 19.2 km in District 6 (3) Relay van 53 (car No. 3) travels 18 in District 6 Traveling 6 km (4) Relay van 54 (Car No. 4) Traveling 17.8 km in section 6 From the above, the positional relationship of each relay van can be determined.
(1) The leading relay van is relay van 51 (car No. 1). (2) Relay van 52 (car No. 2) is running second. (3) Relay van 53 (car No. 3) is running third, and the distance between relay van 52 (car No. 2) and relay van 53 (car No. 3) is 600 m. (4) Relay van 54 (car No. 4) is running fourth, and the distance between relay van 53 (car No. 3) and relay van 54 (car No. 4) is 800 m. It is assumed that the result of identification by the image identification device 55 from the image is as follows.
(1) Identification result from camera 1a AA athlete (2) Identification result from camera 1b image Athlete from EE University, athlete from CC University (ranking in order of description)
(3) Identification results from camera 1c video Players from BB University, Players from II University, Players from GG University (ranking in order of description)
(4) Identification Result from Video of Camera 1d FF University Athlete Based on the above results, the result of the rangefinder system and the image identification device can be generated from the athlete running position screen shown in FIG.

In the distance meter system part of the player running position screen in FIG. , the distance between each relay car, and the difference from the top (relay car 51 (car No. 1)).

Further, the identification result of the image of each camera identified by the image identification device 55 is displayed in the player running position screen of FIG. 22 . In addition, the university names of the athletes on the result screen of the image identification device are displayed in order of ranking from the top.

Next, on the screen of the marathon bulletin online system, the positional relationship specifying unit 57 selects the university athletes that the image identification device 55 could not detect. Here, the university athletes that could not be detected by the image identification device 55 are KK University athletes, DD University athletes, LL University athletes, HH University athletes, and JJ University athletes. The positions of these athletes can be estimated as follows from the aggregated results of the marathon bulletin online system.
(1) KK University athlete running between 19.0km point and 18.5km point (2) DD University athlete running between 18.5km point and 18.0km point (3) LL University athlete Athlete running between 18.5km and 18.0km (4) HH University athlete
Running between the 18.5 km point and the 18.0 km point (5) JJ University athlete Running between the 18.0 km point and the 17.5 km point From the above results, the following can be understood.
(1) Positioned between relay van 51 (car No. 1) and relay van 52 (car No. 2) without players (2) Positioned between relay van 52 (car No. 2) and relay van 53 (car No. 3) Athletes from KK University (3) Athletes located between relay van 53 (car No. 3) and relay van 54 (car No. 4) Athletes from DD University, athletes from LL University, athletes from HH University (4) Broadcast van 54 Athletes behind (Car No. 4) Athletes from JJ University From the above, it is possible to create a screen for the marathon bulletin online system portion of the athlete running position screen of FIG. 22 . In addition, the university names of the athletes on the screen of the marathon bulletin online system are displayed in order from the top.

In the example of FIG. 22, the part of the marathon bulletin online system is displayed in accordance with the position of the relay van. However, as shown in FIG. 25, the ranking of all universities obtained from the marathon bulletin online system may be displayed.

Furthermore, the player running position screen displayed on the display 58 may be made public on the Web so that it can be viewed by related parties. At this time, a chat function may be provided so that each person concerned can communicate with each other. In the example of FIG. 25, a chat screen is provided below the player's running position screen so that comments from each person concerned can be displayed. For example, if XXX, who selects a video, wants a video of a DDD university player, she will tell Car 4 to move. In car No. 4, it is possible to move to the position of the DD University athlete, which is located behind the FF University athlete, by looking at the athlete running position screen, and to give consent to the movement.

According to the fifth embodiment, by linking the distance information of each relay van, the identification result and positional relationship of the image identification device, and the tally result of the marathon bulletin online system, the position of the relay van and participating athletes can be determined. The relationship can be visually grasped, and useful information can be provided to the producer who relays the marathon or the like.

Although the present invention has been described with reference to preferred embodiments, it is not necessary to include all the configurations of the embodiments, and not only can they be combined as appropriate, but the present invention is not necessarily limited to the above embodiments. However, it can be modified and implemented in various ways within the scope of its technical ideas.

1a to 1d camera 2 image identification device 21 orientation estimation unit 22 control unit 23a to 23d image identification unit 24 positional relationship identification unit 30 storage unit 31a to 31d teacher data storage folder 40 metadata generation unit 41 metadata storage unit 42 scene search unit 51 to 54 Broadcast van 55 Image identification device 56 Marathon bulletin online system 57 Positional relationship identification unit 58 Display 100 Camera 101 Orientation estimation unit

Claims (14)

An image identification device that identifies a specific specific person from an image,
an orientation estimation unit that recognizes a person from an image, refers to the position of the recognized person on the image, and estimates the orientation of the specific person on the image;
a plurality of specific person identification units trained to identify the specific person for each orientation of the specific person;
a control unit that selects the specific person identifying unit corresponding to the orientation of the specific person estimated by the orientation estimating unit and causes the selected specific person identifying unit to identify the specific person from the image. . 2. The image identification device according to claim 1, wherein the orientation estimating unit refers to a temporal change in the position of the recognized person on the image to estimate the orientation of the specific person on the image. wherein the orientation estimating unit estimates the orientation of the specific person on the image by referring to a temporal change in the position of the recognized person on the image according to whether or not a camera that captures the image is movable. Item 3. The image identification device according to item 2. The specific person identifying unit refers to the position of the recognized person on the image determined by the orientation estimating unit, and distinguishes the recognized person between the specific person and persons other than the specific person. The image identification device according to any one of claims 1 to 3. 5. The image identification device according to any one of claims 1 to 4, wherein the specific person identification unit identifies the positional relationship of the specific person by referring to the direction of the specific person estimated by the direction estimation unit. 6. The image identification device according to any one of claims 1 to 5, wherein the orientation estimating unit estimates any one of front, back, left and right as the orientation of the specific person. 7. The apparatus according to any one of claims 1 to 6, further comprising a metadata generation unit that associates the identification result of the specific person identification unit and time information of the video in which the identification result is identified, and generates metadata of the video. 1. The image identification device according to claim 1. A scene search unit that searches for video scenes that match predetermined search conditions by referring to the metadata.
The image identification device according to claim 7 . A classification system for classifying teacher data for an image identification device that identifies a specific person from video,
an orientation estimation unit that recognizes a person from an image, refers to the position of the recognized person on the image, and estimates the orientation of the specific person on the image;
A classification system comprising: a storage unit corresponding to the orientation of the specific person estimated by the orientation estimation unit; A production support system comprising a relay van loaded with a camera, an image identification device that identifies a specific specific person from the image of the camera, and a reporting system that reports that the specific person has passed a predetermined point,
The relay vehicle is
Having a position information acquisition unit that acquires position information of the broadcast vehicle on the course,
The image identification device is
an orientation estimating unit that recognizes a person from an image captured by a camera mounted on the broadcast vehicle, refers to the position of the recognized person on the image, and estimates the orientation of the specific person on the image; a plurality of specific person identification units trained to identify the specific person and the specific person identification unit corresponding to the orientation of the specific person estimated by the orientation estimation unit are selected for each orientation of the and a control unit for identifying the specific person from the image by the selected specific person identification unit, wherein the specific person identification unit refers to the orientation of the specific person estimated by the orientation estimation unit, and the Identify the positional relationship of a specific person,
The reporting system has an aggregation unit that aggregates the results of the reports,
The production support system uses the location information of the relay vehicle, the positional relationship of the specific person identified by the image identification device from the image of the camera mounted on the relay vehicle, and the aggregated result of the reporting system, A production support system having a screen generation unit that generates a screen that visually expresses the positional relationship between the broadcast van and the specific person. An image identification method for identifying a specific person from a video,
recognizing a person from an image, referring to the position of the recognized person on the image, and estimating the orientation of the specific person on the image;
selecting a specific person identification unit corresponding to the estimated orientation of the specific person, and identifying the specific person from the image by the selected specific person identification unit;
The image identification method, wherein the specific person identifying unit is trained for identifying the specific person for each orientation of the specific person. A classification method for classifying teacher data for an image identification device that identifies a specific person from a video,
recognizing a person from an image, referring to the position of the recognized person on the image, and estimating the orientation of the specific person on the image;
A classification method for storing an image with an estimated orientation of the specific person in a storage unit corresponding to the estimated orientation of the specific person. A program for an information processing device that identifies a specific specific person from a video,
an orientation estimation unit that recognizes a person from an image, refers to the position of the recognized person on the image, and estimates the orientation of the specific person on the image;
a plurality of specific person identification units trained to identify the specific person for each orientation of the specific person;
The information processing device functions as a control unit that selects the specific person identifying unit corresponding to the orientation of the specific person estimated by the orientation estimating unit, and identifies the specific person from the image by the selected specific person identifying unit. program to make A program for an information processing device that classifies teacher data for an image identification device that identifies a specific person from a video,
an orientation estimation unit that recognizes a person from an image, refers to the position of the recognized person on the image, and estimates the orientation of the specific person on the image;
A program that causes an information processing device to function as a storage unit that stores an image obtained by estimating the direction of the specific person in a storage unit corresponding to the direction of the specific person estimated by the direction estimation unit.

Images