WO2020039473A1 - Image management system, image management method, program, and image management device - Google Patents

Abstract

Provided are an image management system, an image management method, a program, and an image management device which are capable of achieving at least any one among acceleration of information addition used for retrieving image data and incrementation of information accuracy. The image management system (10), which manages image data captured in a performance including a sports game, includes: image data registration units (14, 30, 32, 33) through which the image data is registered; an estimation information extraction unit (40) which extracts and acquires, from an official record of the game, estimation information that is applied to estimate a subject; an analysis unit (60) which analyzes the image data and classifies the subject with a plurality of models; an estimation unit (62) which estimates the subject; and an estimation result storage unit (16) which stores the estimation result in association with the image data.

Description

Image management system, image management method, program, and image management device

The present invention relates to an image management system, an image management method, a program, and an image management device.

フ ァ イ ル File management services using cloud computing are known. For example, a large amount of image data captured at a sports performance is stored in a storage device. A person who wants to use the image data searches for the image data stored in the storage device via the search system. The use applicant acquires and uses the hit image data.

利用 By adding search information such as the date and time of image capture to the image data, the applicant can search for the desired image data from a large amount of image data. Information such as the image capturing date and time and image capturing conditions of the image data can be automatically given to the image data by the image capturing apparatus. It is also possible to manually add various search information to the image data.

人 Manuals are sometimes used when information for searching image data is added to image data. Further, even when information for search of image data is automatically added to image data, verification of matching between the image data and the information for search is manually performed from the viewpoint of ensuring its accuracy. .

Patent Document 1 describes a digital camera that writes game information to image data obtained by a news photographer capturing an image in a stadium. The digital camera described in the document receives the game information transmitted from the broadcast station server, and writes the basic information and the progress information of the game information in a header portion of a captured image.

(4) The digital camera performs face detection and uniform number detection on the image, and when at least one of the face detection and the uniform number is detected, the digital camera sequentially reads the records stored in the database. If there is a record in which at least one of the face and the uniform number matches, the digital camera writes the player name in the record to the header of the image data.

Patent Literature 2 describes an image selection support system that supports selection of image data obtained by imaging in order to quickly create a photo book composed of image data obtained by imaging at a performance. I have. In the image data selection support system described in the document, the recorder terminal generates event information in which an event that has occurred at a performance is associated with date and time, and scene information that indicates the progress of a match in association with time, To the editor terminal.

(4) The digital camera transmits the image data, the information of the image capturing date, and the information of the photographer to the editor terminal. The editor terminal associates the image data with the event information based on the time stamp of the image data and the time stamp of the event information. In addition, images are arranged according to the scene of the game progress based on the scene information. The editor terminal recognizes the player's uniform number and name appearing in the image using optical character recognition, and adds the recognized character information to the image data.

Japanese Patent No. 4659569 JP 2016-86279 A

フ ァ イ ル In the file management service using cloud computing, there is a problem of automation in adding search information to a managed file. The invention described in Patent Literature 1 and the invention described in Patent Literature 2 realize automation of information addition for search with respect to image data obtained by imaging at the performance of a sport. However, it is desired to further improve the accuracy of the processing and to speed up the processing.

The present invention has been made in view of such circumstances, and an image management system and an image management method capable of realizing at least any one of high-speed information addition and high-accuracy information used for searching image data. , A program, and an image management device.

た め The following aspects of the invention are provided to achieve the above object.

An image management system according to a first aspect is an image management system that manages image data obtained by imaging using an imaging device at a performance including a sports game, and an image data registration unit that registers the image data. And extracting estimation information to be applied to the estimation of the subject in the image data from the official record of the game, and an estimation information acquisition unit for acquiring the estimation information, and analyzing the image data by applying a plurality of models to analyze the image data. An analysis unit that classifies the subject into a model; an estimation unit that estimates the subject of the image data based on the estimation information and the model in which the image data is classified; and an estimation result of the estimation unit stored in association with the image data. And an estimation result storage unit.

According to the first aspect, the image data is analyzed to classify the subject into one of a plurality of prescribed models. The subject of the image data is specified based on the classification. As a result, information on the subject in the image data can be obtained at high speed and with high accuracy.

The subject of the image data at the performance of a sport may include at least one of a person and a landscape. The person may include at least one of a player participating in the competition, a spectator of the stadium, a member of a support organization, and the like. The person may include a person wearing a stuffed toy worn by a mascot character.

モ デ ル The model into which the image data is classified can be defined according to the competition. The number of models may be two or more, and there is no limitation on the number of models. The model may be hierarchized.

The estimation information acquisition unit may extract a necessary official record from the official record storage device in advance and acquire a part of the extracted official record, or may acquire all the official records and acquire a necessary part of the official record. May be extracted.

In a second aspect, in the image management system according to the first aspect, the analysis unit may classify whether or not the subject is a player in a game.

According to the second aspect, a player in a game can be estimated as a subject of image data.

選手 The players in the game may include at least one of a player who is participating in the game and a player who is not participating in the game.

In a third aspect, in the image management system according to the second aspect, when the subject is a player in a game, the analysis unit may classify whether the player is an attacker or a defensive player.

According to the third aspect, either the attacking player or the defensive player can be estimated as the subject of the image data.

Attack means the action of a player to get points. Defensive means a player's action to prevent a goal.

In a fourth aspect, in the image management system according to the second aspect or the third aspect, when the subject is a player in a game, the analysis unit may classify a role played by the player.

According to the fourth aspect, the role of the player can be estimated as the subject of the image data.

役 割 The role played by athletes can be defined according to the competition. Examples of roles played by players when the game is baseball include pitchers, fielders, batters, and runners.

In a fifth aspect, in the image management system according to the fourth aspect, the analysis unit may classify the dominant hand of the player when the role of the player is classified.

According to the fifth aspect, the dominant hand of the player can be estimated as the subject of the image data.

The dominant hand of a player means the left and right limbs mainly used in the competition. The dominant hand in the case of a ball game may be the one who mainly operates the ball.

A sixth aspect is the image management system according to any one of the first aspect to the fifth aspect, wherein the analysis unit determines whether the subject is a person other than the player or a scene of the stadium when the subject is not a player in a game. It is good also as composition which classifies crab.

According to the sixth aspect, a person other than a player or a scenery of a stadium can be estimated as a subject of image data.

Examples of the person other than the player include a person wearing a stuffed toy worn by a mascot character and a cheerleader. The stadium landscape may include spectator spectators.

According to a seventh aspect, in the image management system according to any one of the first to sixth aspects, the image management system further includes a face detection unit that detects a face of the subject from the image data and derives a correct answer probability of the subject based on the detection result. Alternatively, the configuration may be such that, when the correct answer probability derived using the face detection unit is smaller than a prescribed reference value, the analysis unit is used to perform the analysis processing of the image data.

According to the seventh aspect, the accuracy of the estimation result can be improved by using face detection together.

According to an eighth aspect, in the image management system according to any one of the first aspect to the seventh aspect, the image management system further includes a photographing information acquisition unit that acquires information of an imaging time of image data, and the estimation information acquisition unit includes: The estimating unit obtains information on the start time of the match, the end time of the match, and the time range of the event that occurred in the match, and the estimating unit compares the imaging time of the image data with the information on the time of the estimated information, so that the image data is captured. The estimated scene may be estimated.

According to the eighth aspect, the accuracy of the estimation result can be improved by comparing the imaging time of the image data with the time of the estimation information.

Events include events during competition that can be recorded on the official record. For example, when the game is baseball, pitching of a pitcher, hitting of a batter, and running of a runner may be included.

シ ー ン A scene where image data is captured can be specified as a scene where an event has occurred.

According to a ninth aspect, in the image management system according to any one of the first to eighth aspects, the estimation information acquisition unit acquires, as estimation information, game information including information on a player who has participated in the game, and an estimation unit. May be configured to collate the analysis result of the analysis unit with the game information and estimate the subject of the image data.

According to the ninth aspect, the accuracy of the estimation result can be improved by referring to the information of the players who have participated in the match.

In a tenth aspect, in the image management system according to any one of the first to ninth aspects, the estimating unit may be configured to create a candidate list of subjects of the image data.

According to the tenth aspect, the estimation result can be grasped by referring to the candidate list.

In an eleventh aspect, in the image management system according to the tenth aspect, a configuration may be adopted in which a signal transmitting unit that transmits a signal for displaying the candidate list on the display device to the display device is provided.

According to the eleventh aspect, the candidate list can be visually recognized.

An image management method according to a twelfth aspect is an image management method for managing image data obtained by imaging using an imaging device in a performance including a sports game, wherein an image data registration step of registering image data is performed. And extracting the estimation information to be applied to the estimation of the subject in the image data from the official record of the game, and performing an estimation information acquiring step of acquiring the estimation information, and analyzing the image data by applying a plurality of models, and An analysis step of classifying the subject into a model, an estimation step of estimating the subject of the image data based on the estimation information and the model in which the image data is classified, and storing the estimation result in the estimation step in association with the image data And an estimation result storage step.

According to the twelfth aspect, the same effect as in the first aspect can be obtained.

に お い て In the twelfth aspect, the same matters as those specified in the second to eleventh aspects can be appropriately combined. In that case, a component that performs a process or a function specified in the image management system can be grasped as a component of an image management method that performs a corresponding process or function.

A program according to a thirteenth aspect is a program for managing image data obtained by imaging using an imaging device in a performance including a sports game, and an image data registration function for registering image data in a computer; Estimation information to be applied to the estimation of the subject in the image data is extracted from the official record of the game, an estimation information acquisition function to acquire the estimation information, image data is analyzed by applying multiple models, and the subject of the image data is modeled. Based on the analysis function for classifying, the estimation information, and the model on which the image data is classified, the estimation function for estimating the subject of the image data, and the estimation result derived using the estimation function are stored in association with the image data. This is a program for realizing the estimation result storage function to be performed.

According to the thirteenth aspect, it is possible to obtain the same effect as the first aspect.

１３ In the thirteenth aspect, matters similar to the matters specified in the second to eleventh aspects can be appropriately combined. In that case, the component that performs the process or function specified in the image management system can be grasped as the component of the program that performs the corresponding process or function.

An image management device according to a fourteenth aspect is an image management device that manages image data obtained by imaging using an imaging device at a performance including a sports game, and an image data registration unit that registers the image data. And extracting estimation information to be applied to the estimation of the subject in the image data from the official record of the game, and an estimation information acquisition unit for acquiring the estimation information, and analyzing the image data by applying a plurality of models to analyze the image data. An analysis unit that classifies the subject into a model; an estimation unit that estimates the subject of the image data based on the estimation information and the model in which the image data is classified; and an estimation result obtained by associating the estimation result of the estimation unit with the image data. And a storage control unit that stores the data in the storage unit.

According to the fourteenth aspect, it is possible to obtain the same effect as the first aspect.

１４ In the fourteenth aspect, the same matters as those specified in the second to twelfth aspects can be appropriately combined. In that case, the component that performs the process or function specified in the image management system can be understood as the component of the image management device that performs the corresponding process or function.

According to the present invention, the image data is analyzed to classify the subject into one of a plurality of prescribed models. The subject of the image data is specified based on the classification. As a result, information on the subject in the image data can be obtained at high speed and with high accuracy.

FIG. 1 is an overall configuration diagram of an image management system according to the embodiment. FIG. 2 is a block diagram showing a hardware configuration of the server device shown in FIG. FIG. 3 is a functional block diagram of the server device shown in FIG. FIG. 4 is a schematic diagram showing a procedure of processing in the image management system shown in FIG. FIG. 5 is an explanatory diagram of a group. FIG. 6 is a schematic diagram of a face list. FIG. 7 is an explanatory diagram of the face detection processing. FIG. 8 is an explanatory diagram illustrating an example of a result of the face detection process. FIG. 9 is a schematic diagram of an analysis process to which time estimation is applied. FIG. 10 is a schematic diagram of an analysis process using image analysis in combination with time estimation. FIG. 11 is an explanatory diagram showing an example of the game information. FIG. 12 is an explanatory diagram showing an example of the player information. FIG. 13 is an explanatory diagram showing an example of the analysis result. FIG. 14 is an explanatory diagram illustrating an example of a candidate list. FIG. 15 is a schematic diagram of machine learning. FIG. 16 is a flowchart showing the procedure of group ID registration. FIG. 17 is a flowchart illustrating the procedure of the face detection process. FIG. 18 is a flowchart showing the procedure of the analysis process. FIG. 19 is a flowchart illustrating the procedure of the estimation process.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In this specification, the same components are denoted by the same reference characters, and redundant description will be omitted as appropriate.

[Overview of Image Management System]
FIG. 1 is an overall configuration diagram of an image management system according to the embodiment. The image management system according to the present embodiment registers image data of an image captured at a performance of a sport and provides the image data to a person who wants to use the image. The use applicant applies for permission to use the image to the administrator who manages the image when using the image. The administrator searches and provides image data according to the application of the applicant.

The image management system according to the present embodiment automatically creates search information for searching for image data based on the estimation result of the subject in the image data. The search information is stored in a prescribed database in association with the image data, and is used for searching the image data. For the estimation of the subject in the image data, game information and player information extracted from the official record are referred to.

Hereinafter, the image management system according to the present embodiment will be described in detail. In the present embodiment, an official game of professional baseball is exemplified as a performance of a sport. In the present embodiment, generation of search information in image data registered and taken by a cameraman contracted by a competition manager or the like in an official game of professional baseball will be described.

で は In this specification, the term image data refers to electronic data obtained by imaging using an imaging device. The term image refers to printed objects, objects displayed on a display device, and the like. The image may include characters, figures, patterns, and the like. In this specification, image data and image are treated as readable terms.

The image management system 10 shown in FIG. 1 includes a server device 12, an image data storage device 14, a search information storage device 16, and a game information storage unit 18. The server device 12 performs registration of image data, processing of image data, search of image data, and the like. For example, image data obtained by imaging using the imaging device 20 is transmitted to the server device 12 using the terminal device 22. The server device 12 classifies the image data for each team and stores it in the image data storage device 14.

For example, when transmitting image data from the terminal device 22 to the server device 12, a site for each team in which the image data is registered is specified. The server device 12 can store the image data at the designated site for each team. Note that illustration of the image data is omitted in FIG. The image data is shown in FIG.

画像 The image management system 10 shown in FIG. 1 is applicable to both a cloud system and an on-premises system. The cloud system can apply a format in which software is used online via a network such as an ASP (Application @ Service @ Provider) and a SaaS (Software @ as @ a @ Service).

The cloud system can apply a format in which a set of hardware and a platform such as an OS (Operating System) such as PaaS (Platform as a Service) is remotely used as a service on the Internet.

The cloud system can be applied to a form in which infrastructure such as equipment and lines required for operating the system, such as IaaS (Infrastructure as a service), is remotely used as a service on the Internet.

The server device 12 automatically generates search information, which is information for searching for image data, for the registered image data. An example of the search information is a player name appearing in the image.

The server device 12 stores the automatically generated search information in the search information storage device 16 in association with the image data. Details of automatic generation of search information will be described later.

(4) The server device 12 may be a computer. The server device 12 realizes the function of the server device 12 by reading and executing the program. The program may be read from a storage device provided in the server device 12, or may be read from a storage device external to the server device 12. The server device 12 described in the embodiment is an example of an image management device.

The image data storage device 14 stores the image data registered in the image management system 10. As the image data storage device 14, a large-capacity storage device can be applied. Similarly, a large-capacity storage device can be applied to the search information storage device 16 and the game information storage unit 18.

The search information storage device 16 stores search information of image data in association with the image data. The game information storage unit 18 stores game information applied to analysis of image data and estimation of a person appearing in an image. The game information can be extracted and obtained from the official record database 24A stored in the official record storage device 24 managed by the competition manager in the sports performance. The game information is shown in FIG.

で は In the present embodiment, the official record managed by the competition manager is exemplified, but the official record may be managed by the organizer of the sports box office. Official records may also be derived from formal official records maintained by the Competition Manager, and may be included in the concept of official records, consistent with official records and conforming to official records. .

An example of official record conformance record is a record created by an external organization such as a news organization, a distribution service organization, or a game maker, using an official record managed by a competition manager or a sports box office organizer. On the other hand, records that are independently collected and managed by the records manager and are not consistent with official records are not included in official records compliant records because a certain level of reliability has not been obtained. Note that the event manager and the organizer may include the meaning of an organization.

The image management system 10 can apply a network system in which the server device 12 and the terminal device 22 are communicably connected via a network (not shown). The server device 12 can be communicably connected to at least one of the image data storage device 14, the search information storage device 16, and the game information storage unit 18 via a network.

A known public communication network can be applied to the network. The network may be a small-scale communication network such as a LAN (Local Area Network). The communication standard and the communication form applied to the network are not limited.

[Configuration of server device]
[Hardware configuration]
FIG. 2 is a block diagram showing a hardware configuration of the server device shown in FIG. The server device 12 illustrated in FIG. 2 includes a control unit 120, a memory 122, a storage device 124, and a network controller 126. The I / O shown in FIG. 2 represents an input / output interface.

The control unit 120, the memory 122, the storage device 124, and the network controller 126 are connected via a bus 136 so that data communication is possible.

<Control unit>
The control unit 120 functions as an overall control unit, various calculation units, and a storage control unit of the server device 12. The control unit 120 executes a program stored in a ROM (read only memory) provided in the memory 122.

The control unit 120 may download a program from an external storage device via the network controller 126 and execute the downloaded program. The external storage device may be communicably connected to the server device 12 via the network 140.

The control unit 120 performs various processes in cooperation with various programs by using a RAM (random access memory) provided in the memory 122 as a calculation area. Thereby, various functions of the image management system 10 are realized.

The control unit 120 controls reading of data from the storage device 124 and writing of data to the storage device 124. The control unit 120 may acquire various data from an external storage device via the network controller 126. The control unit 120 can execute various processes such as calculation using the obtained various data.

The control unit 120 may include one or more processors. Examples of the processor include an FPGA (Field Programmable Gate Array) and a PLD (Programmable Logic Device). FPGAs and PLDs are devices whose circuit configuration can be changed after manufacturing.

Another example of a processor is an ASIC (Application Specific Integrated Circuit). The ASIC has a circuit configuration specifically designed to execute a specific process.

The control unit 120 can apply two or more processors of the same type. For example, the control unit 120 may use two or more FPGAs or two PLDs. The control unit 120 may apply two or more processors of different types. For example, the control unit 120 may apply one or more FPGAs and one or more ASICs.

In the case where a plurality of control units 120 are provided, the plurality of control units 120 may be configured using one processor. As an example of configuring the plurality of control units 120 with one processor, one processor is configured using a combination of one or more CPUs (Central Processing Unit) and software, and this processor functions as the plurality of control units 120. There is a form to do. Note that software in this specification is synonymous with a program.

Another example in which the plurality of control units 120 are configured by one processor is a mode in which a processor that realizes the functions of the entire system including the plurality of control units 120 by one IC chip is used. As a typical example of a processor that realizes the functions of the entire system including the plurality of control units 120 with one IC chip, there is an SoC (System @ On \ Chip). Note that IC is an abbreviation for Integrated @ Circuit.

As described above, the control unit 120 has a hardware structure using one or more types of processors.

<memory>
The memory 122 includes a ROM (not shown) and a RAM (not shown). The ROM stores various programs executed in the image management system 10. The ROM stores parameters used for executing various programs, files, and the like. The RAM functions as a temporary storage area for data, a work area for the control unit 120, and the like.

<Storage device>
The storage device 124 temporarily stores various data. The storage device 124 may be externally provided outside the server device 12. Instead of the storage device 124 or in combination therewith, a large-capacity semiconductor memory device may be applied.

<Network controller>
The network controller 126 controls data communication with an external device. Controlling data communication may include managing data communication traffic. As the network 140 connected via the network controller 126, a known network such as a LAN can be applied.

The hardware configuration of the server device 12 shown in FIG. 2 is an example, and can be added, deleted, and changed as appropriate.

[Function block of image management system]
FIG. 3 is a functional block diagram of the server device shown in FIG. The server device 12 includes an image data acquisition unit 30, a pre-processing unit 32, and an image data storage unit 33.

The image data acquisition unit 30 acquires the image data transmitted from the terminal device 22. The image data acquisition unit 30 stores the image data in the image data storage device 14. Further, the image data acquisition unit 30 transmits the image data to the pre-processing unit 32. The image data storage device 14 and the image data acquisition unit 30 described in the embodiment are examples of components of the image data registration unit.

The pre-processing unit 32 performs pre-processing on the image data. The pre-processing includes acquisition of information on the imaging date and time added to the image data, resizing of the image data, rotation of the image data, and addition of a group ID (identification). The pre-processing unit 32 according to the embodiment is an example of a shooting information acquisition unit that acquires information on an imaging time of image data.

(4) The pre-processing unit 32 stores the image data subjected to the resizing process and the rotation process in the image data storage unit 33. The preprocessing unit 32 stores the information on the imaging date and time, the information on the imaging condition, and the group ID in the search information storage device 16 in association with the image data.

The preprocessing unit 32 and the image data storage unit 33 described in the embodiment are examples of components of the image data registration unit. The search information storage device 16 according to the embodiment is an example of an estimation result storage unit.

The server device 12 includes the official record information acquisition unit 40. The official record information acquisition unit 40 extracts and reads out the game information 25 necessary for the image data analysis process and the image data estimation process from the official record database 24A. In FIG. 3, illustration of the official record storage device 24 shown in FIG. 1 is omitted.

The official record information acquisition unit 40 can apply an API (Application Programming Interface). The official record information acquisition unit 40 stores the game information 25 read from the official record database 24A in the game information storage unit 18.

The official recording information acquisition unit 40 described in the embodiment is an example of an estimation information acquisition unit. The game information 25 shown in the embodiment is an example of estimation information.

The official record database 24A is a database created and managed by a competition manager of an official professional baseball game. The official record database 24A stores event information for each match, performance information, and the like.

The server device 12 includes a face detection unit 50, a player master acquisition unit 52, and a face list acquisition unit 54. The face detection unit 50 performs a face detection process on the image data to be processed, and detects a player's face appearing in the image. As a result of the face detection process, the correct answer probability of the specified player can be applied. A plurality of persons may be specified as a result of the face detection processing.

The player master acquisition unit 52 acquires the player master 210 from the official record database 24A. Details of the player master 210 will be described later. The player master is a list of players in which a face photograph of the player and identification information of the player are associated with each other. The player master can be prepared in advance by each team.

The face list acquisition unit 54 acquires the face list 200. The face list 200 is a list in which a plurality of correct images are stored for each player. The face list 200 can be prepared in advance by each team. The face detection unit 50 can perform face detection processing with reference to the player master 210 and the face list 200. Note that the face list 200 and the player master 210 may be integrated.

The server device 12 includes an analysis unit 60 and an estimation unit 62. The analysis unit 60 performs an analysis process on the image data and analyzes a scene or the like of the image. The estimating unit 62 refers to the detection result of the face detection unit 50, the analysis result of the analysis unit 60, and the game information 25 to estimate the person appearing in the image. The estimation result is stored in the search information storage device 16 as the candidate list 220.

The step of extracting and reading the game information 25 from the official record database 24A, and the function is to extract estimation information to be applied to the estimation of the subject in the recorded image data from the official record of the game, and to obtain the estimation information. It is an example of a process and an estimation information acquisition function.

[Detailed description of image management system]
FIG. 4 is a schematic diagram showing a procedure of processing in the image management system shown in FIG.

[Registration of image data]
The image data 23 of the image captured by the cameraman 21 using the imaging device 20 is registered in the image management system 10. That is, the image data 23 is registered on the site for each team. A site for each team is prepared in a web server device (not shown). The site for each team publishes the image data 23 stored in the image data storage device 14. A user who wants to use the image data 23 can access the site for each baseball team using the terminal device and browse the image data 23 registered in the management system using images.

[Pre-processing]
Pre-processing is performed on the image data 23 registered using the pre-processing unit 32 shown in FIG. In the pre-processing, the information of the imaging date and time added to the image data 23 is obtained, and the information of the imaging date and time and the information of the imaging condition are stored in the search information storage device 16.

{Circle around (4)} In the pre-processing, the image data 23 is resized. This can suppress an increase in processing load in the face detection processing and the analysis processing. In the pre-processing, a rotation process is performed on the image data 23. Thereby, the orientation of the image is corrected, and the detection accuracy in the face detection processing and the analysis processing can be improved. The pre-processed image data 23 is stored in the image data storage unit 33.

(4) Further, in the pre-processing, a sorting process is performed, and a group ID is assigned. In the pre-processing, a file list is generated for each group.

FIG. 5 is an explanatory diagram of a group. The group 80 represents a group of a plurality of pieces of image data 23 which are continuously shot. The group ID represents an identification symbol of the group 80. That is, a plurality of image data 23 that are continuously shot are individually given file information and managed as a file.

{Circle around (1)} A group ID is given to a group of the plurality of image data 23 which are continuously shot. In the pre-processing, the queue registration of the face detection processing is performed using the group ID. The face detection process is performed for each group 80 in the order in which the queues are registered. The same applies to analysis processing and estimation processing described later.

The information of the imaging date and time, the information of the imaging condition, the group ID, and the file list for each group ID acquired in the pre-processing are stored in the search information storage device 16 shown in FIG. After the pre-processing, the face detection processing is performed. The advance preparation described in the embodiment is an example of an image data registration step and an image data registration function.

(Face detection processing)
In the face detection process, the face detection unit 50 illustrated in FIG. 3 performs the face detection process for each group 80 for each of the image data 23 stored in the image data storage unit 33. In the face detection processing, a face list and a player master are referred to.

FIG. 6 is a schematic diagram of a face list. The face list 200 is a list in which correct images 202 for each player are registered. FIG. 6 shows a correct image of A man B belonging to the ABC team. The face list 200 may register a plurality of correct images 202 for one player. FIG. 6 shows an example in which four correct images 202 are registered for the male A and the male B.

FIG. 7 is a conceptual diagram of the face detection process. In the face detection processing, image data to be processed is read from the image data storage unit 33, a face area 23A is detected from the image data 23 to be processed, and the face area 23A is identified. In the face detection processing, image processing may be performed on the image data 23 when identifying the face area 23A.

FIG. 7 illustrates an example of image processing in which the image data 23 is divided into a plurality of regions 23B and a characteristic region 23C is extracted using the characteristic amount of each region 23B. In the example shown in FIG. 7, a plurality of regions 23B are extracted as the characteristic regions 23C.

The face area 23A is identified by referring to the face list 200, collating the face area 23A with the correct image 202 registered in the face list 200, and calculating the correct answer probability of the candidate player. A well-known technique can be applied for calculating the correct answer probability.

FIG. 8 is an explanatory diagram showing an example of the result of the face detection process. The example illustrated in FIG. 8 illustrates an example in which three players belonging to the ABC team are extracted as candidates and the correct answer probability is calculated for each player. The player with the highest correct answer probability may be used as the detection result of the face detection processing. The detection result of the face detection processing is stored in the search information storage device 16. When the detection result of the face detection processing is equal to or more than the prescribed correct answer probability, the result of the face detection processing can be used as the estimation result of the image data 23.

[Match information confirmation]
Returning to FIG. 4, the game information confirmation processing is performed after the face detection processing. In the match information confirmation process, it is determined whether or not match information on the day of the shooting is stored in the match information storage unit 18.

If the game information storage unit 18 does not store the game information on the shooting day, it is determined again whether or not the game information on the shooting day is stored again after the specified period has elapsed. When the game information storage unit 18 stores the game information on the day of the shooting, the process proceeds to the analysis process.

The step of extracting and storing the game information from the official record database 24 </ b> A shown in the embodiment and the function of extracting and applying the estimation information to be applied to the estimation of the subject in the image data from the official record of the game and obtaining the estimation information It is an example of an estimation information acquisition process and an estimation information acquisition function.

(Analysis processing)
<Analysis processing using time estimation>
FIG. 9 is a schematic diagram of an analysis process to which time estimation is applied. In the above-described face detection process, when the person appearing in the image to be processed is not sufficiently specified, an analysis process using time estimation is performed on the image data 23 applied to the face detection process.

解析 The analysis processing using the time estimation uses the information of the imaging time and the game information 25. As the information on the imaging time, the information on the imaging date and time given by the imaging device 20 to the image data 23 is applied. The information of the imaging date and time given to the image data 23 has been acquired in advance preparation.

The image management system 10 can specify a player who has participated in the shooting date and time as a person appearing in the image corresponding to the image data 23 with reference to the game information 25 stored in the game information storage unit 18. .

FIG. 9 shows an example in which the information of the shooting date and the game information 25 and the game information 25 are used together to derive an estimation result that the possibility that the person appearing in the image corresponding to the image data 23 is the male A and the male B is 80%. Is shown. Further, it is possible to improve the estimation accuracy by using the image analysis together with the time estimation.

<Analysis processing using image analysis in combination with time estimation>
FIG. 10 is a schematic diagram of an analysis process using image analysis in combination with time estimation. In the analysis processing, a plurality of analysis models are defined in advance, and the image data 23 is classified using the analysis models. Examples of the analysis model include an offense / defense model indicating whether the player is a pitcher or a batter, and a dominant hand model indicating a dominant hand. The dominant hand here means the dominant hand in the competition. When the game is baseball, the dominant hand in defense and the dominant hand in attack, such as right throw and left strike, may be different. In the case of a batter, there are right-handed, left-handed, and double-handed.

In image analysis, a characteristic region is extracted from image data, and classification of image data based on the shape, size, pixel value, and the like of the characteristic region is possible. For example, a batter has a characteristic shape of holding a bat in his hand and a characteristic shape of wearing a helmet with ears. Right or left can be specified from the position of the head and the position of the bat.

で あ れ ば A pitcher has a characteristic shape in which the dominant hand extends upward, diagonally upward, laterally, diagonally downward, or downward. From the position of the head and the position of the dominant hand, a right throw or a left throw can be specified. In this way, by extracting the characteristic region from the image data 23 and analyzing the characteristic region, the image data 23 can be classified into any of the analysis models.

FIG. 11 is an explanatory diagram showing an example of the game information. The game information 25 shown in FIG. 11 includes information on the number of innings, the inning start time, and the inning end time. The game information 25 includes a batter list indicating a batter who has participated in the inning and a runner list indicating a runner who has participated in the inning. Furthermore, the game information 25 includes a fielder list representing a fielder who has participated in the inning, a pitcher list representing a pitcher who has competed in the inning, and a catcher list representing a catcher who has competed in the inning. The period specified from the inning start time and the inning end time described in the embodiment is an example of the time range of the event that has occurred in the match.

The batter list includes information on whether the player hits right, left, or both. The pitcher list includes information on right throw or left throw. The game information 25 shown in FIG. 11 includes information such as a game day, a home team code, a visitor team code, a game start time, a game end time, a list of players participating in each home team, a list of players participating in the visitor teams, and the like. It is. Items of the game information 25 shown in FIG. 11 can be added and deleted as appropriate.

In the example shown in FIG. 10, an analysis result is obtained that the person appearing in the image corresponding to the image data 23 is a right-handed batter. Returning to FIG. 4, the result of the analysis processing is stored in the search information storage device 16.

The offense / defense model indicating whether a player is a pitcher or a batter described in the embodiment is an example of classification of whether a player is an attacker or a defensive player. The pitcher, the fielder, the batter, the runner, and the catcher shown in the embodiment are examples of the classification of the role played by the player.

(Estimation processing)
In the estimation process, a person appearing in the image to be processed is estimated based on the results of the face detection process and the analysis process. In the estimation process, the player information 240 extracted from the official record database 24A is referred to. The estimation result is stored in the search information storage device 16. In the estimation process, a candidate list is created.

FIG. 12 is an explanatory diagram showing an example of the player information. The player information 240 includes a year to which the player information is applied, and a player code as identification information of the player. The player information 240 includes information such as an affiliated team code, a uniform number, a dominant hand, and a defensive position. The player information 240 may include supplementary information such as the height, weight, and date of birth of the player.

FIG. 13 is an explanatory diagram showing an example of the analysis result. The analysis result 250 illustrated in FIG. 13 includes a group ID, file information, an imaging date and time, a face detection result, and an analysis result. A candidate list is created based on the analysis result 250 shown in FIG.

FIG. 14 is an explanatory diagram showing an example of a candidate list. FIG. 14 shows an example of a candidate list in which a list format is applied to a plurality of image data 23. The candidate list 220 illustrated in FIG. 14 includes an image 222 corresponding to the image data 23, a candidate name 224, classification information 226, and remarks 228.

The image 222 is managed using file information given to the image data 23 as identification information. When there are a plurality of candidates, the candidate names include all the candidates. The classification information 226 includes text information representing a model.

For example, the estimating unit 62 illustrated in FIG. 3 may display a candidate list indicating an estimation result on a display unit of a terminal device communicably connected to the server device 12. FIG. 14 shows a state in which the candidate list at the center in the column direction in the candidate list 220 is selected. The list of candidates in the center has a check box with a selection symbol and a highlighted background.

表示 The display signal on the display unit of the terminal device is implemented using a signal transmission unit (not shown). The signal transmission unit (not shown) according to the embodiment is an example of a signal transmission unit that transmits a signal for displaying the candidate list on the display device to the display device.

The operator of the terminal device can check the contents of the candidate list displayed on the display unit of the terminal device. If the contents of the candidate list are correct, information for determining the player name associated with the image data 23 may be transmitted from the terminal device to the server device 12.

When the contents of the candidate list need to be modified, the operator of the terminal device may modify the contents of the candidate list and transmit the modified candidate list to the server device 12 from the terminal device.

The process and the function of performing the estimation process described in the embodiment are an example of the estimation process and the estimation function of estimating the subject of the image data based on the estimation information and the model into which the image data is classified. The step of storing the estimation result and the function described in the embodiment are an example of the estimation result storage step and the estimation result storage function of storing the estimation result in association with the image data.

In the present embodiment, the aspect in which the face detection result is referred to when obtaining the estimation result is illustrated. However, the face detection result is not referred to when obtaining the estimation result, and the estimation is performed based on the analysis result and information extracted from the official record. A mode of obtaining a result is also possible.

[Machine learning]
FIG. 15 is a schematic diagram of machine learning. Machine learning is performed using the set of the image data and the estimation result as the correct answer data, and the accuracy of the face detection processing, the analysis processing, and the estimation processing can be improved. An example of machine learning is a convolutional neural network, which is an iterative process of a convolutional layer and a pooling layer.

That is, machine learning is performed using the correct answer image 202 obtained as the face list 200, the game information 25 obtained from the official record database 24A, the analysis processing result of the image data 23, and the estimation result as the correct answer data, and FIG. It is possible to improve the processing accuracy of the face detection unit 50, the analysis unit 60, and the estimation unit 62 shown in FIG.

In machine learning, machine learning using correct answer data specialized in sports competitions can be performed on learning machines that have performed general-purpose learning. As described above, by performing the two-stage machine learning, the number of correct answer data prepared in advance can be reduced.

[Procedure of image management method]
The processing procedure in the image management system shown in FIG. 4 will be described in detail.

[Group ID registration procedure]
FIG. 16 is a flowchart showing the procedure of group ID registration. The group ID registration is performed in advance preparation shown in FIG. In the group ID generation step S10 illustrated in FIG. 16, the pre-processing unit 32 illustrated in FIG. 3 generates a group ID. In the group ID generation step S10, a file list for each group 80 is generated.

In the database registration step S12, the pre-processing unit 32 registers the group ID in the search information storage device 16. That is, in the database registration step S12, a file list is registered in the search information storage device 16 for each group.

In the queue registration step S14, the pre-processing unit 32 registers the queue of the group ID. That is, in the queue registration step S14, the queue registration of the face detection processing is performed for each group. In the queue registration step S14, the group ID is registered in a storage queue (not shown) for managing the queue.

[Face detection processing procedure]
FIG. 17 is a flowchart illustrating the procedure of the face detection process. In the file list acquisition step S20, the face detection unit 50 acquires a file list of the group 80 to be processed. In the face detection processing, the processing is performed for all files stored in the file list for each file.

In the player identification step S22, the face detection unit 50 extracts the face area 23A from the image data 23, identifies the person in the image corresponding to the image data 23 with reference to the face list 200 and the player master 210. I do. It should be noted that, for the image data 23 in which no person can be specified, the person specified in the image data 23 belonging to the same group 80 may be the specified result.

That is, the group includes a plurality of image data obtained by continuous shooting. It is considered that a plurality of image data obtained by continuous shooting show the same subject. Therefore, the same player may be specified for the image data 23 belonging to the same group. Here, the image data 23 in which a player cannot be specified may include image data in which a player is specified but a correct answer probability is lower than a prescribed reference.

In the identification result registration step S24, the face detection unit 50 registers the identification result in the player identification step S22 in the search information storage device 16. When the specific results have been registered for all the files, the process proceeds to a queue registration step S26.

In the queue registration step S26, the face detection unit 50 registers the queue of the analysis processing. That is, in the queue registration step S26, the group on which the face detection processing has been performed is registered in the analysis processing queue.

[Procedure of analysis processing]
FIG. 18 is a flowchart showing the procedure of the analysis process. Hereinafter, an aspect in which the game information confirmation illustrated in FIG. 4 is included in the analysis processing will be described. In the file list acquisition step S40, the analysis unit 60 acquires a file list of the group 80 to be processed. The analysis process is performed for all files stored in the file list for each file.

In the game information acquisition step S42, the analysis unit 60 acquires the game information 25 shown in FIG. In the match condition determination step S44, the analysis unit 60 collates the shooting date and time of the image data 23 with the match information. When it is determined that the match is not performed on the date and time when the image data 23 was captured, the determination is No. If the determination is No, the process proceeds to the analysis result registration step S46.

In the analysis result registration step S <b> 46, the analysis unit 60 registers information indicating that no match is performed on the date and time of imaging of the image data 23 in the search information storage device 16. On the other hand, if it is determined in the game condition determination step S44 that the game is being played on the shooting date and time of the image data 23, the determination is Yes. If the determination is Yes, the process proceeds to the match time determination step S48.

In the game time determination step S48, the analysis unit 60 refers to the game information 25 to determine whether or not the imaging time of the image data 23 is during the game. When it is determined that the imaging time of the image data 23 is not during the game, the determination is No. If the determination is No, the process proceeds to the pre- and post-match information providing step S50.

In the pre / post-game information providing step S50, the analysis unit 60 provides the image data 23 with information indicating that the imaging time is before or after the game. After the pre- and post-match information providing step S50, the process proceeds to a mascot cheer model analysis step S52.

In the mascot cheer model analysis step S52, the analysis unit 60 analyzes whether the image data 23 is an image of at least one of a mascot and a cheerleader, or an image of a spectator seat of a stadium. The analysis result in the mascot cheer model analysis step S52 is registered in the search information storage device 16 in the analysis result registration step S46.

The mascot and cheerleader described in the embodiment are examples other than the players. The spectator seats of the stadium shown in the embodiment are examples of the scenery of the stadium.

On the other hand, in the match time determination step S48, if it is determined that the imaging time of the image data 23 is during the match, the determination is Yes. If the determination is Yes, the process proceeds to the offense / defense model analysis step S54.

In the offense / defense model analysis step S54, the analysis unit 60 classifies the person appearing in the image into one of a pitcher, a batter, and another. In the offense and defense model analysis result determination step S56, the analysis unit 60 determines whether or not the image data 23 is an image of a pitcher or a batter based on the analysis result in the offense and defense model analysis step S54. When it is determined that the image data 23 is not an image of the pitcher or the batter, the determination is No. If the determination is No, the process proceeds to the other model analysis step S58.

In the other model analysis step S58, the analysis unit 60 classifies the person shown in the image data 23 into a runner, a fielder, a catcher, and others. In the other model analysis result determination step S60, the analysis unit 60 determines whether or not the person shown in the image data 23 has been classified into another.

If it is determined in the other model analysis result determination step S60 that the person shown in the image data 23 has been classified as a runner, a fielder, or a catcher, a No determination is made. If the determination is No, the analysis unit 60 registers the analysis result in the analysis result registration step S46.

If it is determined in the other model analysis result determination step S60 that the person appearing in the image data 23 has been classified as other, a Yes determination is made. If the determination is Yes, the process proceeds to the mascot cheer model analysis step S52. The analysis unit 60 registers the analysis result of the mascot cheer model analysis step S52 in the analysis result registration step S46.

If it is determined in the offense / defense model analysis result determination step S56 that the image of the pitcher or the batter is captured, the determination is Yes. If the determination is Yes, the process proceeds to the dominant hand model analysis step S62. In the dominant hand model analysis step S62, the analysis unit 60 analyzes whether the pitcher is a right throw or left throw, or whether the batter is right or left strike. The analysis result of the dominant hand model analysis step S62 is registered in the analysis result registration step S46.

When the analysis results are registered for all the files included in the group, the process proceeds to the queue registration step S64. In the queue registration step S64, the analysis unit 60 performs queue registration of the estimation processing.

[Procedure of estimation processing]
FIG. 19 is a flowchart illustrating the procedure of the estimation process. In the file list obtaining step S100, the estimating unit 62 obtains a file list of a processing target group. The analysis process is performed for all files stored in the file list for each file.

In the face detection processing result acquisition step S102, the estimation unit 62 acquires the face detection processing result shown in FIG. In the face detection processing result determination step S104, the estimating unit 62 determines whether or not only a player whose accuracy probability is less than 80% is specified as a result of the face detection processing. In addition, 80% of the correct answer probability is an example, and an arbitrary value can be applied to the correct answer probability. When the correct answer probability is relatively high, the accuracy of the face detection processing can be improved.

場合 If it is determined in the face detection processing result determination step S104 that only players with a correct answer probability of less than 80% have not been identified, the determination is No. In other words, in the face detection processing result determination step S104, if a player with a correct answer probability of 80% or more is specified and the result of the face detection processing can be adopted as the estimation result, a No determination is made. If the determination is No, the process proceeds to the candidate list creation step S106.

In the candidate list creation step S106, the estimating unit 62 creates the candidate list 220 shown in FIG. 14 based on the result of the face detection processing. In the estimation result registration step S108, the estimation unit 62 registers the candidate list 220.

On the other hand, in the face detection processing result determination step S104, when it is determined that only the player whose correct answer probability is less than 80% is specified, the determination is Yes. If the determination is Yes, the accuracy of the face detection process is low. In this case, the process proceeds to the analysis result acquisition step S110. In the analysis result obtaining step S110, the estimation unit 62 obtains the analysis result 250 shown in FIG. Less than 80% shown in the embodiment is an example of less than the reference value.

In the player determination step S112, it is determined whether or not the analysis result 250 represents a player. In the player determination step S112, when it is determined that the analysis result 250 does not represent a player, a No determination is made. That is, when it is determined that the analysis result is a mascot, a Chigaard, a spectator seat of a stadium, or the like, the process proceeds to an estimation result registration step S108.

In the estimation result registration step S108, the estimation unit 62 registers an estimation result indicating that the person appearing in the image is a mascot, a tiagaal, a spectator seat of a stadium, or the like.

On the other hand, in the player determination step S112, when it is determined that the analysis result represents a player, a Yes determination is made. If the determination is Yes, the process proceeds to the offense / defense information acquisition step S114. In the offense and defense information acquisition step S114, the estimating unit 62 acquires home team information and offense and defense information from the game information 25 using the shooting date and time of the image data 23. That is, in the offense and defense information acquisition step S114, the estimating unit 62 identifies the home team and identifies offense and defense.

In the scene matching step S116, it is determined whether or not the scene based on the game information 25 matches the offense and defense classification based on the analysis result. For example, in the match information 25, when the player is defending, the match is determined if the analysis result is a pitcher. On the other hand, in the game information 25, when the player is defending, if the analysis result is a batter, it is determined that there is no match.

Ｎｏ If no match is found in the scene collation step S116, a No determination is made. If the determination is No, the process proceeds to the scene determination step S118. In the scene determination step S118, the estimation unit 62 determines, based on the game information 25, whether the scene at the imaging time is being defended or being attacked.

場合 If it is determined in the scene determination step S118 that the player is attacking, a No determination is made. If the determination is No, the process proceeds to the attacking player information acquisition step S120. In the attacking player information acquiring step S120, the estimating unit 62 acquires, from the game information 25, information on the player who was the batter and information on the player who was the runner during the inning specified based on the imaging time. The player information includes information such as a player name and a player code that can specify the player. When there are a plurality of runners, information on players corresponding to all the runners is obtained.

The estimating unit 62 creates the candidate list 220 by using the player information acquired from the game information 25 in the candidate list creating step S106.

On the other hand, if it is determined that the player is defending in the scene determination step S118, the determination is Yes. If the determination is Yes, the process proceeds to the defensive player information acquisition step S122. In the defensive player information acquiring step S122, the estimating unit 62 acquires, from the game information 25, information on all the defensive players during the inning specified based on the imaging time. The estimating unit 62 creates the candidate list 220 in the candidate list creating step S106 using the player information acquired from the game information 25.

If it is determined in the scene collation step S116 that they match, the determination is Yes. In the case of Yes determination, more detailed estimation is performed using the analysis result. In the attack determination step S124, the estimation unit 62 determines whether or not the classification based on the analysis result is under attack. For example, if the analysis result is a pitcher or a fielder, it is determined that the player is defending, and the determination is No. If the determination is No, the process proceeds to the pitcher determination step S126. In the pitcher determination step S126, the estimation unit 62 determines whether the analysis result is a pitcher or a fielder. If the analysis result is a fielder, a No determination is made. If the determination is No, the process proceeds to the defensive position determination step S128.

In the defensive position determining step S128, the estimating unit 62 determines whether the defensive position is a fielder other than the catcher. If it is determined to be a catcher, a No determination is made. If the determination is No, the process proceeds to catcher information acquisition step S130. In the catcher information acquiring step S130, the estimating unit 62 acquires, from the game information 25, information on the players who have participated as catchers during the inning specified based on the imaging time.

The estimating unit 62 creates the candidate list 220 in the candidate list creating step S <b> 106 using the information of the players who have participated as catchers acquired from the game information 25.

On the other hand, when it is determined in the defensive position determining step S128 that the fielder is a fielder other than the catcher, the determination is Yes. If the determination is Yes, the process proceeds to the fielder information acquisition step S132. In the fielder information obtaining step S132, the estimating unit 62 obtains, from the game information 25, information on all players who have participated as fielders other than catchers during the inning specified based on the imaging time.

The estimation unit 62 creates the candidate list 220 in the candidate list creation step S106 by using information on all players who have participated as fielders other than catchers acquired from the game information 25.

場合 If the pitcher is determined in the pitcher determination step S126, the determination is Yes. In the case of Yes determination, the flow proceeds to pitcher information acquisition step S134. In the pitcher information obtaining step S134, the estimating unit 62 obtains, from the game information 25, information on the players who have participated as pitchers during the inning specified based on the imaging time. In the pitcher dominant hand information acquisition step S136, the estimating unit 62 acquires dominant hand information from the analysis result.

The estimating unit 62 creates the candidate list 220 in the candidate list creating step S106 using the information of the players who have participated as pitchers acquired from the game information 25 and the dominant hand information.

(4) In the attack determination step S124, if the analysis result is determined to be an attack, the determination is Yes. In the case of a Yes determination, the process proceeds to a batter determination step S138. In the batter determination step S138, the estimation unit 62 determines whether the analysis result is a batter. In the batter determination step S138, if the analysis result is determined to be a runner, a No determination is made. If the determination is No, the process proceeds to the runner information acquisition step S140. In the runner information obtaining step S140, the estimating unit 62 obtains, from the game information 25, information on all players who have participated as runners during the inning specified based on the imaging time.

The estimating unit 62 creates the candidate list 220 in the candidate list creating step S106 by using information of all the players who have participated as runners obtained from the official record information.

On the other hand, in the batter determination step S138, if the analysis result is determined to be a batter, a Yes determination is made. In the case of Yes determination, the flow proceeds to batter information acquisition step S142. In the batter information obtaining step S142, the estimating unit 62 obtains, from the game information 25, information on a player who has participated as a batter during the inning specified based on the imaging time. Furthermore, in the batter dominant hand information acquisition step S144, the estimating unit 62 acquires batter dominant hand information from the analysis result.

The estimating unit 62 creates the candidate list 220 in the candidate list creating step S <b> 106 using the information of the player who has participated as the batter obtained from the game information 25 and the dominant hand information.

After the candidate list 220 is created for all the files belonging to the group and the candidate list 220 is specially recorded, the process proceeds to the adjusting step S146.

In the adjustment step S146, the estimation unit 62 adjusts the estimation result for the files belonging to the group. For example, all files belonging to the same group should match in the candidate list 220. However, when there is a file in which the candidate list 220 does not match in the group, the candidate list 220 of the mismatching file is adjusted.

推定 The estimation result derived in this way is stored in the search information storage device 16 in association with the file.

[Verification of estimation results]
The estimation result for each file stored in the search information storage device 16 may be verified. For example, when there are a plurality of candidates, the operator may visually check the image and specify a person appearing in the image from the plurality of candidates. Verification of the estimation result can be performed using a terminal device communicably connected to the image management system 10.

That is, the operator accesses the image management system 10 using the terminal device, downloads the candidate list 220 shown in FIG. 13, specifies the person appearing in the image 222 for each image 222, and manages the specified result in the image management. It may be stored in the system 10.

[Effects]
According to the image management system, the image management method, and the program configured as described above, the following effects can be obtained.

[1]
When classifying the subject of the image data, a model such as a pitcher, a fielder, a batter, and a runner is applied. This makes it possible to analyze the image data and estimate the subject of the image data at high speed and with high accuracy.

[2]
When the subject of the image data is a pitcher or a batter, the dominant hand is classified. Thus, the analysis of the image data and the estimation of the subject in the image data can be performed with high accuracy.

[3]
With reference to the match start time, the match end time, and the imaging time of the image data, it is specified whether or not the imaging of the image data is being performed. This makes it possible to analyze the image data and estimate the subject of the image data at high speed and with high accuracy.

[4]
With reference to the start time of the inning, the end time of the inning, and the imaging time of the image data, it is specified whether the imaging of the image data is under attack or under defense. Thereby, the scene of the image data and the event that occurred during the game can be specified. Thus, the estimation of the subject in the image data can be performed with high accuracy.

[5]
Obtain match information and player information from official records. This makes it possible to analyze image data and obtain information applied to estimating a subject in the image data.

[6]
An official record, which is highly reliable big data, is applied to analysis of image data and estimation of a subject of the image data. As a result, it is possible to analyze image data with a certain high accuracy and to estimate a subject of the image data.

[Example of application to other sports]
In the present embodiment, professional baseball is illustrated as a performance of a sport, but the image management system, the image management method, and the program described in the present embodiment can be applied to other sports. That is, a game that can acquire an official record, analyze image data from the official record, and extract information applied to estimating the subject of the image data is a game in which the image management system, the image management method, and the program according to the present embodiment are used. Can be applied. In particular, it is relatively easy to apply to ball games and sports in which offense and defense can be easily distinguished.

For example, the softball similar in the game rules to the baseball shown in the present embodiment can be applied to the image management system shown in the present embodiment. In sports using rackets such as tennis, table tennis, and badminton, estimation using dominant hand information is effective. For throwing balls, such as football, volleyball, basketball, handball, and water polo, estimation using dominant hand information is also effective.

競技 In sports such as tennis, where the net is interposed, court information can be applied as offense and defense information. The court information means information indicating whether the player is a player on one side of the net or a player on the other side. In addition, in games where it is difficult to distinguish between offense and defense, such as soccer, basketball, handball, and water polo, information such as the direction of movement of the ball and the direction of movement of players can be obtained to distinguish between offense and defense. Note that an attack means a player's action for obtaining a score, and a defense means a player's action for preventing a goal.

[Example of application to program]
The function of each unit of the image management system 10 and the function of each step of the image management method described above can be realized by executing a program using a computer. For example, an image data registration function for registering image data, an estimation information acquisition function for extracting estimation information to be applied to the estimation of a subject of image data from an official record of a game, and acquiring estimation information, and an image by applying a plurality of models. Analyze data and classify objects in image data into models.Estimation information.Estimation function to estimate objects in image data based on model into which image data is classified. A program that causes a computer to implement an estimation result storage function of storing the estimated result in association with image data may be configured. It is also possible to configure a non-transitory and computer-readable recording medium such as a CD-ROM (Compact Disk-Read Only Memory) or a flash ROM storing this program.

In the embodiment of the present invention described above, constituent elements can be appropriately changed, added, or deleted without departing from the spirit of the present invention. The present invention is not limited to the embodiments described above, and many modifications can be made by those having ordinary knowledge in the art within the technical concept of the present invention.

REFERENCE SIGNS LIST 10 image management system 12 server device 14 image data storage device 16 search information storage device 18 match information storage unit 20 imaging device 21 photographer 22 terminal device 23 image data 23A face region 23B region 23C feature region 24 official record storage device 24A official record database 25 Game information 30 Image data acquisition unit 32 Preprocessing unit 33 Image data storage unit 40 Official record information acquisition unit 50 Face detection unit 52 Player master acquisition unit 54 Face list acquisition unit 60 Analysis unit 62 Estimation unit 80 Group 102 Image processing device 104 Display device 105 Input device 120 Control unit 122 Memory 124 Storage device 126 Network controller 128 Power supply device 130 Display controller 132 Input / output interface 134 Input controller 136 Bus 140 Net Each step of chromatography click 200 face list 202 correct images 210 players master 220 candidate list 222 image 224 candidate's name 226 classification information 228 Remarks 240 S146 image managing method from player information S10

Claims (15)

1. An image management system that manages image data obtained by imaging using an imaging device in a performance including a sports game,
   An image data registration unit for registering image data;
   An estimation information acquisition unit that extracts estimation information to be applied to the estimation of the subject of the image data from the official record of the game, and acquires the estimation information;
   An analysis unit that analyzes the image data by applying a plurality of models, and classifies a subject of the image data into the model.
   An estimating unit that estimates a subject of the image data based on the estimation information and a model in which the image data is classified;
   An estimation result storage unit that stores the estimation result of the estimation unit in association with the image data,
   Image management system with
2. The image management system according to claim 1, wherein the analysis unit classifies whether the subject is a player in a game.
3. The image management system according to claim 2, wherein, if the subject is a player in a game, the analysis unit classifies the player as an attacker or a defensive player.
4. 4. The image management system according to claim 2, wherein the analysis unit classifies a role played by the player when the subject is a player in a game. 5.
5. The image management system according to claim 4, wherein the analysis unit further classifies the dominant hand of the player when the role of the player is classified.
6. The image management system according to any one of claims 1 to 5, wherein the analysis unit classifies the subject as a person other than a player or a scene of a stadium when the subject is not a player in a game. .
7. A face detection unit that detects a face of the subject from the image data and derives a correct answer probability of the subject based on the detection result,
   7. The image processing apparatus according to claim 1, wherein when the correct probability derived using the face detection unit is less than a predetermined reference value, the analysis unit performs the analysis process on the image data. 8. Image management system.
8. An imaging information acquisition unit that acquires information on the imaging time of the image data,
   The estimation information acquisition unit acquires, as the estimation information, a start time of the match, an end time of the match, and information on a time range of an event that occurred in the match,
   The image according to any one of claims 1 to 7, wherein the estimation unit estimates a scene in which the image data is captured by comparing a time at which the image data is captured with information on a time of the estimation information. Management system.
9. The estimation information acquisition unit acquires, as the estimation information, match information including information on players who have participated in the match,
   9. The image management system according to claim 1, wherein the estimation unit estimates a subject of the image data by comparing an analysis result of the analysis unit with the game information. 10.
10. The image management system according to any one of claims 1 to 9, wherein the estimation unit creates a candidate list of subjects of the image data.
11. The image management system according to claim 10, further comprising: a signal transmission unit that transmits a signal for displaying the candidate list on a display device to the display device.
12. An image management method for managing image data obtained by imaging using an imaging device in a performance including a sports game,
    An image data registration step of registering image data;
    An estimation information acquisition step of extracting estimation information to be applied to the estimation of the subject of the image data from the official record of the game, and acquiring the estimation information;
    Analyzing the image data by applying a plurality of models, an analysis step of classifying a subject of the image data into the model,
    An estimation step of estimating a subject of the image data based on the estimation information and a model in which the image data is classified;
    An estimation result storing step of storing the estimation result in the estimation step in association with the image data,
    An image management method including:
13. A program for managing image data obtained by imaging using an imaging device in a performance including a sports game,
    On the computer,
    Image data registration function for registering image data,
    An estimation information acquisition function for extracting estimation information to be applied to the estimation of the subject in the image data from the official record of the game, and acquiring the estimation information;
    Analyzing the image data by applying a plurality of models, an analysis function of classifying a subject of the image data into the model,
    An estimation function for estimating a subject of the image data based on the estimation information and a model in which the image data is classified, and an estimation result derived using the estimation function is stored in association with the image data. A program for realizing the estimation result storage function.
14. An image management device that manages image data obtained by imaging using an imaging device in a performance including a sports game,
    An image data registration unit for registering image data;
    An estimation information acquisition unit that extracts estimation information to be applied to the estimation of the subject of the image data from the official record of the game, and acquires the estimation information;
    An analysis unit that analyzes the image data by applying a plurality of models, and classifies a subject of the image data into the model.
    An estimating unit that estimates a subject of the image data based on the estimation information and a model in which the image data is classified;
    A storage control unit that stores the estimation result of the estimation unit in the estimation result storage unit in association with the image data,
    An image management device comprising:
15. A non-transitory and computer-readable recording medium, wherein the instructions stored in the recording medium are read by a computer that manages image data obtained by using an imaging device at a performance including a sports game. If
    Image data registration function for registering image data,
    An estimation information acquisition function for extracting estimation information to be applied to the estimation of the subject in the image data from the official record of the game, and acquiring the estimation information;
    Analyzing the image data by applying a plurality of models, an analysis function of classifying a subject of the image data into the model,
    An estimation function for estimating a subject of the image data based on the estimation information and a model in which the image data is classified, and an estimation result derived using the estimation function is stored in association with the image data. Estimation result storage function,
    A recording medium that causes a computer to realize.

Images