JP2018206085A - Event evaluation support system, event evaluation support device, and event evaluation support program - Google Patents

Abstract

To provide an event evaluation support system, an event evaluation support device and an event evaluation support program that allow improvement of accuracy in evaluating events that may occur in future on the basis of video data including images and sounds.SOLUTION: An event evaluation support system to support evaluation of events that may occur in future on the basis of video data including images and sounds comprises: acquisition means S110 to acquire the video data for the evaluation; a reference database that stores past video data acquired in advance, reference data used in evaluating the past video data, and 3 or more degrees of association between the past video data and the reference data; evaluation means S120 to refer to the reference database to acquire evaluation data including a first degree of association between the video data for the evaluation and the reference data; and output means S130 to generate an evaluation result on the basis of the evaluation data and to output the evaluation result.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an event evaluation support system, an event evaluation support device, and an event evaluation support program that support evaluation of events that may occur in the future based on video data including images and sounds.

  2. Description of the Related Art Conventionally, as a system for recording and analyzing conversations between people and analyzing the acceptability of the conversation partner of the user and the skill of the conversation of the user, for example, the conversation analysis system of Patent Document 1 is disclosed. . Further, as a system capable of estimating emotions of learners and the like with high accuracy and high reliability, for example, an emotion estimation system disclosed in Patent Document 2 is disclosed.

  The conversation analysis system disclosed in Patent Document 1 is a wearable terminal, which is an important word database for extracting important words uttered during conversation, and a topic for extracting influencers for a specific topic in conversation. A word database, a microphone that collects voice information of conversation, a camera that collects facial expressions and head movements of the conversation partner as video, a voice evaluation unit that recognizes topic words and important words from voice information, and video A video evaluation unit that recognizes the smile time and number of whispers of the conversation partner per unit conversation time, and the weight of the other party for the conversation and the skill of the conversation by predetermined weighting from the recognition results of the voice evaluation unit and the video evaluation unit A conversation analysis unit for analyzing, and a display unit for feeding back the acceptance level and skill of the conversation to the user.

  The emotion estimation system disclosed in Patent Document 2 includes a camera capable of acquiring face image data, an average change rate calculating unit that calculates an average change rate from face image data of a subject photographed by the camera, and faces of a plurality of subjects. The facial feature points corresponding to a plurality of different emotion pairs used for emotion estimation are extracted from the correlation matrix between the average change rate of feature points and emotions, and the feature points are narrowed down. An estimation reference amount database configured by calculating an average value and a variance-covariance matrix in each group, and determination means for determining to which group the estimator belongs based on face image data obtained by the camera And an emotion estimation means for estimating an emotion using the data of the group to which the estimated person of emotion belongs belongs based on the determination result of the determination means.

JP, 2006-103081, A JP 2011-39934 A

  Here, video data including images and sounds released on TV, the Internet, etc. can estimate, for example, the characteristics and emotions of the person who made the sound, and affect the emotions and actions of others who viewed the video data It has the potential. For this reason, there is a possibility that it may lead to evaluation of an event that may occur in the future based on the video data. However, there is a risk that evaluators and other subjects will intervene in the evaluation of events that may occur in the future. For this reason, the quantitative evaluation based on video data cannot be performed, but the subject that the precision of evaluation is low is mentioned.

  In this regard, the technique disclosed in Patent Document 1 merely extracts users who have a large influence on others on a certain topic, and what kind of influence the video data actually has on others and the like will occur in the future. It is not possible to make a quantitative assessment of the events obtained.

  The technique disclosed in Patent Document 2 is premised on real-time processing of only the face image of an e-learning learner, and video data including sound is not used as information. Moreover, in the technique disclosed in Patent Document 2, in order to acquire a face image of a learner who is performing e-learning, the technique is limited to emotion during learning. For this reason, it is difficult to fully evaluate the influence on the emotions and actions of others. This makes it impossible to quantitatively evaluate events that may occur in the future.

  Due to the above-described circumstances, it is desired to improve the accuracy in evaluating future events based on video data.

  Therefore, the present invention has been devised in view of the above-described problems, and the object of the present invention is to improve accuracy in evaluating future events based on video data including images and sounds. An event evaluation support system, an event evaluation support device, and an event evaluation support program are provided.

  An event evaluation support system according to a first aspect of the present invention is an event evaluation support system that supports evaluation of an event that may occur in the future based on video data including images and sounds, an acquisition unit that acquires video data for evaluation, A reference database in which acquired past video data, reference data used for evaluation of the past video data, and three or more levels of association between the past video data and the reference data are stored; , Referring to the reference database, and evaluating means for acquiring evaluation data including first relations of three or more stages between the video data for evaluation and the reference data, and generating an evaluation result based on the evaluation data And an output means for outputting the evaluation result.

  The event evaluation support system according to a second aspect of the present invention is the event evaluation support system according to the first aspect, wherein the evaluation video data includes image data including at least one of a photographed person, an animal, a background, and explanatory material; Sound data including at least one of a voice, an animal voice, a sound effect, and an inserted music, and the evaluation means refers to the reference database, the combination of the image data and the sound data, and the reference The first association degree with data is calculated.

  The event evaluation support system according to a third invention is the event evaluation support system according to the second invention, wherein the evaluation means refers to the reference database, and the evaluation image data calculated in time series with respect to the evaluation video data photographed in a certain period of time. The evaluation data including one degree of association is acquired.

  The event evaluation support system according to a fourth aspect of the present invention is the event evaluation support system according to the third aspect, wherein the acquisition means corresponds to the time at which the video data for evaluation is filmed and shows at least one of stock, currency exchange, and futures price movements An evaluation price movement is obtained, and the reference data includes a time course of a past price movement of at least one of a past stock, an exchange rate, and a futures transaction, and the output means is for the evaluation based on the first association degree. The evaluation result including the evaluation of the time course of the price movement is generated.

  The event evaluation support system according to a fifth aspect of the present invention is characterized in that, in the fourth aspect, the evaluation video data and the past video data are images relating to economic information.

  The event evaluation support system according to a sixth aspect of the present invention is the event evaluation support system according to the third aspect, wherein the output means outputs the evaluation result based on a result obtained by comparing a preset notification reference value with the first association degree. It is characterized by.

  In the event evaluation support system according to a seventh aspect of the present invention, in the sixth aspect of the invention, the past video data includes images before and after the crime at the crime scene, the reference data is information relating to a crime type, and the output means includes: The evaluation result including information for identifying a person who is likely to commit a crime among persons photographed in the video data for evaluation is generated.

  In an event evaluation support system according to an eighth aspect based on the third aspect, the past video data is a video released to a plurality of viewers, and the reference data corresponds to the released past video data. The evaluation video data is undisclosed video data, and the output means generates the evaluation result including an evaluation of a temporal transition of the audience rating based on the first association degree. Features.

  In an event evaluation support system according to a ninth aspect based on the third aspect, the past video data includes progress information relating to the progress of the game, and the evaluation video data includes development progress information relating to the progress of the game under development. The reference data includes at least one of a charge amount for the game progress information, a number of logins, and a play time, and the output means charges the development progress information based on the first association degree. The evaluation result including the evaluation of the temporal transition of at least one of the number of logins and the play time is generated.

  The event evaluation support system according to a tenth aspect of the present invention is the event evaluation support system according to the second aspect, wherein the evaluation unit refers to the reference database and calculates the first calculated video data for the whole evaluation for a predetermined period. The evaluation data including the degree of association is acquired.

  An event evaluation support system according to an eleventh aspect of the present invention is the event evaluation support system according to the tenth aspect, wherein the evaluation video data and the past video data are a security camera video, a video in an examination room, a video during conversation, a video during customer service, a counseling It includes a video at the time, a video at the performance, a video at the examination, or a video at the time of surgery.

  The event evaluation support system according to a twelfth aspect of the present invention is the event evaluation support system according to any one of the first to eleventh aspects, wherein the past video data includes at least one of a person photographed in the past, an animal, and a background. Synthetic data having the association degree between the reference data and past sound data including at least one of human image data, animal voice, sound effects, and inserted music recorded in the past And three or more similarities between the past image data and the synthesized data and between the past sound data and the synthesized data, and the reference database includes the synthesized data And the association degree between the reference data and the reference data is stored.

  In any one of the first to twelfth inventions, the event evaluation support system according to a thirteenth aspect of the present invention is configured such that, when a relationship between the past video data and the reference data is newly acquired, It further comprises update means for reflecting the relationship in the association degree.

The event evaluation support system according to a fourteenth aspect of the present invention is the event evaluation support system according to any one of the second to thirteenth aspects, wherein the evaluation video data includes text data corresponding to the content of the sound data.
The evaluation unit refers to the reference database, and calculates the first association degree between the reference data and a combination of the image data, the sound data, and the text data.

  An event evaluation support apparatus according to a fifteenth aspect of the present invention is an event evaluation support apparatus that supports evaluation of an event that may occur in the future based on video data including images and sound, and an acquisition unit that acquires evaluation video data; A reference database in which acquired past video data, reference data used for evaluation of the past video data, and three or more levels of association between the past video data and the reference data are stored; , Referring to the reference database, and generating an evaluation result based on the evaluation data, and an evaluation unit that acquires evaluation data including first relations of three or more stages between the video data for evaluation and the reference data And an output unit for outputting the evaluation result.

  An event evaluation support apparatus according to a sixteenth aspect of the present invention is an event evaluation support program for supporting evaluation of an event that may occur in the future based on video data including images and sound, and acquires video data for evaluation, which is acquired in advance. Referring to past video data, reference data used to evaluate the past video data, and a reference database in which three or more levels of association between the past video data and the reference data are stored; A computer that obtains evaluation data including three or more stages of first association between the evaluation video data and the reference data, generates an evaluation result based on the evaluation data, and outputs the evaluation result It is made to perform.

  According to the first to fourteenth aspects, the evaluation means refers to the reference database and acquires evaluation data including the first association degree between the video data for evaluation and the reference data. For this reason, reference data is linked to the evaluation video data, and a quantitative evaluation result of an event that may occur in the future can be acquired based on the evaluation video data. As a result, it is possible to improve the accuracy in the evaluation of events that may occur in the future.

  In particular, according to the second to fourteenth aspects, the evaluation means calculates a first relationship between the combination of the image data and the sound data and the reference data. For this reason, compared with the case where either one of image data and sound data is used, the feature of the video or the estimation of the emotion of the photographed person, or the affect of the other person who viewed the video or the behavior of the other person is affected. The accuracy of evaluating the possibility of giving can be improved. As a result, it is possible to further improve the accuracy of evaluation of events that may occur in the future.

  In particular, according to the third to ninth inventions and the twelfth to fourteenth inventions, the evaluation means calculates the first association degree in time series for the video data for evaluation photographed in a certain period. For this reason, the time-dependent change of evaluation with respect to video data for evaluation can be shown. As a result, it is possible to perform evaluation along with the temporal change of the video data for evaluation, and it is possible to perform detailed evaluation of events that may occur in the future.

  In particular, according to the fourth invention, the fifth invention, and the twelfth to fourteenth inventions, the output means generates an evaluation result including an evaluation of the temporal transition of the evaluation value movement based on the first association degree. For this reason, it is possible to easily obtain the evaluation of the temporal transition of the price movement based on the influence of the evaluation video data. This makes it possible to examine a business plan of a company etc. using the evaluation result as a material, and to increase the options of business strategy.

  In particular, according to the sixth invention, the seventh invention, and the twelfth invention to the fourteenth invention, the output means outputs the evaluation result based on the result of comparing the preset notification reference value and the first association degree. To do. For this reason, it is possible to control whether or not to output the evaluation result according to the setting condition of the notification reference value. Thereby, it is possible to output only when the evaluation result is necessary, and the user or the like does not always need to confirm the evaluation result. Accordingly, it is possible to reduce the work load of the user or the like.

  In particular, according to the seventh invention and the twelfth to fourteenth inventions, the output means specifies information for identifying a person who is likely to commit a crime among those photographed in the video data for evaluation based on the evaluation data. An evaluation result including is generated. For this reason, it becomes possible to suppress a crime in advance. In addition, it is possible to quantitatively evaluate a place where a large number of people come and go, and it is possible to suppress misperceptions and oversights including subjectives.

  Particularly, according to the eighth invention and the twelfth to fourteenth inventions, the output means generates an evaluation result including an evaluation of a temporal transition of the audience rating based on the first association degree. For this reason, the temporal transition of the audience rating can be evaluated in advance for unpublished evaluation video data. As a result, improvement points and the like can be grasped before the video data is released, and the quality of the video data can be improved.

  In particular, according to the ninth invention and the twelfth to fourteenth inventions, the output means generates an evaluation result including an evaluation of a change over time such as a charge amount for the development progress information based on the first association degree. For this reason, it is possible to evaluate the timing, frequency, etc. charged in a game under development. As a result, improvement points and the like can be grasped before the game is sold, and the quality of the game can be improved.

  In particular, according to the tenth to fourteenth inventions, the evaluation means calculates the first association degree with respect to the entire evaluation image data photographed in a certain period. For this reason, it is possible to comprehensively evaluate the evaluation video data including the image data and the sound data captured during a certain period. This makes it possible to greatly expand the selection range of events that can be quantitatively evaluated.

  In particular, according to the twelfth to fourteenth inventions, past video data has three or more levels of similarity between past image data and synthesized data, and between past sound data and synthesized data. Including. For this reason, when past image data and past sound data are newly stored (updated) in the reference database, it is not necessary to update the association degree each time. Thereby, it is possible to suppress the load for updating the association degree with the update of the reference database.

  In particular, according to the thirteenth and fourteenth inventions, when the updating means newly acquires the relationship between the past video data and the reference data, the updating means reflects the relationship in the association degree. For this reason, the relevance can be easily updated, and the accuracy of evaluation can be improved.

  In particular, according to the fourteenth aspect, the first relation between the combination of the image data, the sound data, and the text data and the reference data is calculated. For this reason, by adding text data to the combination, it is possible to further improve the accuracy of estimating the characteristics or emotions of the person or evaluating the possibility of affecting the emotions or actions of others. As a result, it is possible to further improve the accuracy of evaluation of events that may occur in the future.

  According to the fifteenth aspect, the evaluation unit refers to the reference database, and acquires evaluation data including the first association degree between the video data for evaluation and the reference data. For this reason, reference data is linked to the evaluation video data, and a quantitative evaluation result of an event that may occur in the future can be acquired based on the evaluation video data. As a result, it is possible to improve the accuracy in the evaluation of events that may occur in the future.

  According to the sixteenth aspect of the invention, the computer is caused to acquire the evaluation data including the first association degree between the evaluation video data and the reference data with reference to the reference database. For this reason, reference data is linked to the evaluation video data, and a quantitative evaluation result of an event that may occur in the future can be acquired based on the evaluation video data. As a result, it is possible to improve the accuracy in the evaluation of events that may occur in the future.

FIG. 1 is a block diagram showing an overall configuration of an event evaluation support system in the embodiment. FIG. 2A and FIG. 2B are schematic diagrams illustrating an example of evaluation results in the embodiment. FIG. 3 is a diagram illustrating an example of evaluation video data, past video data, reference data, and evaluation results in the embodiment. FIG. 4 is a diagram illustrating an example of evaluation video data, past video data, reference data, and evaluation results in the embodiment. FIG. 5A is a schematic diagram illustrating an example of the configuration of the event evaluation support apparatus according to the embodiment, and FIG. 5B is a schematic diagram illustrating an example of the function of the event evaluation support apparatus according to the embodiment. FIG. 6 is a schematic diagram illustrating an example of a reference database according to the embodiment. FIG. 7 is a schematic diagram illustrating a first modification of the reference database according to the embodiment. FIG. 8 is a schematic diagram illustrating a second modification of the reference database according to the embodiment. FIG. 9 is a schematic diagram illustrating a third modification of the reference database according to the embodiment. FIG. 10 is a schematic diagram illustrating a fourth modification of the reference database according to the embodiment. FIG. 11 is a flowchart illustrating an example of the operation of the event evaluation support system in the embodiment. FIG. 12A and FIG. 12B are schematic diagrams illustrating an example of evaluation results in the embodiment.

  Hereinafter, an example of an event evaluation support system 100 according to an embodiment to which the present invention is applied will be described with reference to the drawings.

(Embodiment: Configuration of Event Evaluation Support System 100)
With reference to FIG. 1, an example of a configuration of an event evaluation support system 100 in the present embodiment will be described. FIG. 1 is a block diagram showing the overall configuration of the event evaluation support system 100 in the present embodiment.

  The event evaluation support system 100 includes an event evaluation support device 1. The event evaluation support device 1 may be connected to at least one of the server 2 and the imaging device 3 via, for example, the public communication network 4.

  The event evaluation support system 100 is used to support evaluation of events that may occur in the future based on video data including images and sounds. For example, as shown in FIG. 2A, the event evaluation support apparatus 1 includes evaluation image data including image data D1 including a photographed person and sound data D2 corresponding to the voice of the photographed person. And outputs an evaluation result based on the video data for evaluation. The evaluation result includes information on an event that may occur in the future, such as “the Nikkei Stock Average may increase by XX yen”. Based on the evaluation result, for example, the user can determine a future event.

  For example, as shown in FIG. 2B, the event evaluation support apparatus 1 may periodically acquire evaluation video data from the imaging apparatus 3 fixed at a predetermined position. In this case, for example, a person who has a possibility of committing a crime can be identified, and an evaluation result such as “There is a possibility of shoplifting (broken line part)” can be output. In the evaluation information, as information for specifying a person, for example, the specified person is surrounded by a broken line, and the person may be specified by an arrow or the like. In addition to the above, the event evaluation support apparatus 1 evaluates economic information, sales, and the like based on, for example, video of a TV program acquired via the public communication network 4, video during customer service, video during conversation, etc. Can do.

  The event evaluation support apparatus 1 refers to the reference database when outputting the evaluation result. The reference database stores past video data acquired in advance, reference data used for evaluation of past video data, and a degree of association between past video data and reference data. The event evaluation support apparatus 1 refers to the reference database, calculates a first association between the evaluation video data and the reference data, and generates an evaluation result based on the evaluation data including the first association. .

  The above-described evaluation image data, past video data, reference data, and evaluation results include, for example, examples of data shown in FIGS. The evaluation video data includes image data D1 and sound data D2. The past video data includes past image data including a person photographed in the past, and may include past sound data corresponding to a voice of a person photographed in the past, for example.

  The image data D1 and the past image data include still images as well as moving images, and at least humans, animals, backgrounds, and explanatory materials (for example, explanatory graphs, flip boards, presentation materials) captured in each video data. Including either That is, the image data D1 and the past image data indicate data of the entire image including the photographed person. The image data D1 and the past image data include, for example, a movement tendency of a person or an animal, for example, a facial expression corresponding to the emotion of a photographed person, as well as a mouth corner position, a state between eyebrows, and the like. In addition, a part of the face that affects the facial expression may be the target of the data. In addition to the above, the image data and the past image data may be subject to data such as a gesture of a photographed person, a behavior, a behavior, a hand gesture, and a behavior such as a walking position and a stop time.

  The sound data D2 and the past sound data include at least one of a photographed human voice, animal voice, sound effect, and inserted music. That is, the sound data D2 and the past sound data indicate all sound data including the voice of the photographed person. The sound data D2 and the past sound data include, for example, words and intonation in addition to voice and sound volume and speed, and may be classified into categories such as preset emotions. . Note that the sound data D2 and the past sound data may be classified into voices for each sound source or each person, or may be in a state where a plurality of sound sources or voices of a plurality of people are mixed.

  By evaluating the image data D1 and the sound data D2 in combination, it is possible to accurately evaluate events that may occur in the future. For example, by combining a person's action in the image data D1 and a person's tone in the sound data D2, for example, emotions such as emotions that cannot be determined only by the image data D1 can be taken in as evaluation targets.

  The evaluation video data includes text data corresponding to the content of the sound data, for example. The past video data includes, for example, past text data corresponding to the contents of the past sound data. The text data and the past text data include, for example, the whole sentence of the sound, for example, only the characteristic morpheme, word, phrase, clause, etc. may be extracted from the whole sentence of the sound. Note that the text data and past text data may be, for example, sound effects other than language converted into onomatopoeia.

  The image data D1, the past image data, the sound data D2, the past sound data, the text data, and the past text data may be digitized, for example, and may be set to 3 to 10-level evaluation, respectively. Details of each data will be described later.

<Configuration of event evaluation support apparatus 1>
FIG. 5A is a schematic diagram illustrating an example of the configuration of the event evaluation support apparatus 1. An electronic device such as a personal computer (PC) is used as the event evaluation support apparatus 1. The event evaluation support apparatus 1 may be formed integrally with an arbitrary apparatus that can acquire video data for evaluation, for example. The event evaluation support apparatus 1 includes a housing 10, a CPU 101, a ROM 102, a RAM 103, a storage unit 104, and I / Fs 105 to 107. Each component 101 to 107 is connected by an internal bus 110.

  A CPU (Central Processing Unit) 101 controls the event evaluation support apparatus 1 as a whole. A ROM (Read Only Memory) 102 stores an operation code of the CPU 101. A RAM (Random Access Memory) 103 is a work area used when the CPU 101 operates. The storage unit 104 stores various information such as cell images. As the storage unit 104, for example, an HDD (Hard Disk Drive), a data storage device such as an SSD (solid state drive) or a floppy disk is used. For example, the event evaluation support apparatus 1 may include a GPU (Graphics Processing Unit) (not shown). By having a GPU, it is possible to perform higher-speed arithmetic processing than usual.

  The I / F 105 is an interface for transmitting and receiving various types of information to and from the server 2 and the like via the public communication network 4. The I / F 106 is an interface for transmitting / receiving information to / from the input unit 108. For example, a keyboard is used as the input part 108, and a user of the event evaluation support system 100 inputs various information or a control command of the event evaluation support apparatus 1 through the input part 108. The I / F 107 is an interface for performing transmission / reception of various information with the output unit 109. The output part 109 outputs various information stored in the storage unit 104 or the processing status of the event evaluation support apparatus 1. A display is used as the output portion 109, and for example, a touch panel type may be used.

  FIG. 5B is a schematic diagram illustrating an example of functions of the event evaluation support apparatus 1. The event evaluation support apparatus 1 includes an information DB 11, an acquisition unit 12, an evaluation unit 13, an output unit 14, and an input unit 15. The event evaluation support apparatus 1 may include an update unit 16, for example. The function shown in FIG. 5B is realized by the CPU 101 executing a program stored in the storage unit 104 or the like using the RAM 103 as a work area. Moreover, each structure 11-16 may be controlled by artificial intelligence, for example.

<Information DB11>
The information DB 11 stores a reference database in which past video data acquired in advance and reference data used for past video data are stored. The information DB 11 includes, for example, evaluation video data, evaluation data including a first association degree between the evaluation video data and reference data, an evaluation result generated based on the evaluation data, a format for displaying the evaluation result, and the like. Various information is stored. The reference database and various types of information are stored as a database of various types of information in the storage unit 104 embodied by an HDD, SSD, or the like. Each structure 12-16 memorize | stores various information in information DB11 as needed, or takes out various information.

  For example, as shown in FIG. 6, the reference database stores three or more levels of association between past video data and reference data. The reference database is formed by an algorithm that can calculate the association degree, for example. The past video data and reference data have a plurality of data, and the relationship between each past data and each reference data is associated with each other by the degree of association.

  For example, the video A included in the past video data indicates the association degree “80%” with the evaluation A included in the reference data, and the association degree “15%” with the evaluation B included in the reference data. Indicates. That is, the “association degree” indicates the degree of connection between the data, and the higher the association degree, the stronger the connection between the data.

  The past video data and reference data are stored in the reference database in the form of video, and may be stored in the form of numerical values, matrices (vectors), histograms, or the like.

  The association degree is calculated using, for example, machine learning. For example, deep learning is used for machine learning. In particular, in the event evaluation support system 100 in this embodiment, it is preferable to use a convolutional neural network.

  The past video data may be stored in the reference database by dividing past image data and past sound data, for example, as shown in FIG. In this case, the association degree is calculated based on the relationship between the combination of the past image data and the past sound data and the reference data. For example, the combination of the image A included in the past image data and the sound A included in the past sound data indicates the association degree “90%” with the evaluation A, and the association degree “20” with the evaluation B. % ". In this case, the past image data and past sound data can be stored independently of each other. For this reason, when generating an evaluation result based on video data for evaluation described later, it is possible to improve accuracy and expand the range of options.

  The past video data may include composite data and similarity as shown in FIG. 8, for example. The synthesized data is indicated by three or more levels of similarity with past image data or past sound data. In addition to being stored in the reference database in the form of video, the synthesized data may be stored in the form of numerical values, matrices (vectors), histograms, or the like.

  The past video data may include past text data, for example. When the past video data includes past text data, for example, as shown in FIG. 9, based on the combination of the past image data, the past sound data, and the past text data and the reference data, The degree is calculated. In addition, the past video data may include a similarity indicating the relationship between the text data and the synthesized data, for example, as shown in FIG.

<Acquisition unit 12>
The acquisition unit 12 acquires evaluation video data to be evaluated. The acquisition unit 12 acquires evaluation video data captured by the imaging device 3 for a certain period, and also, for example, evaluation video captured for a certain period via a storage medium such as the public communication network 4 or a portable memory. Data may be acquired. The format of the evaluation video data is arbitrary, and for example, the acquisition unit 12 may convert it into an arbitrary file format.

  In addition to the evaluation video data, the acquisition unit 12 may acquire, for example, personal information about a person photographed in the evaluation video data. Thereby, it is possible to output an evaluation result corresponding to the person information.

<Evaluation unit 13>
The evaluation unit 13 acquires evaluation data including the first association degree of three or more levels between the evaluation video data and the reference data. The evaluation unit 13 refers to, for example, a reference database, selects past video data that matches or is similar to the evaluation video data, and uses the degree of association associated with the selected past video data as the first degree of association. calculate. In addition, the evaluation unit 13 may calculate a first association degree between the evaluation video data and the reference data using, for example, a reference database as an algorithm of the classifier.

  For example, when the reference database shown in FIG. 6 is used, when the evaluation video data matches or is similar to video A, “80%” for evaluation A, “15%” for evaluation B, and evaluation C On the other hand, the first association degree of “1%” is calculated. Further, when the evaluation video data is similar to the video A and the video B, for example, a value obtained by multiplying the association degree between the video A and the video B and the reference data by an arbitrary coefficient is calculated as the first association degree. Is done.

  After calculating the first association degree, the evaluation unit 13 acquires evaluation data including evaluation video data, reference data, and the first association degree. Note that the evaluation unit 13 may calculate the first association degree with reference to, for example, the reference database illustrated in FIG. 7 or FIG.

<Output unit 14>
The output unit 14 generates an evaluation result based on the evaluation data and outputs the evaluation result. The output unit 14 generates an evaluation result for the evaluation video data based on, for example, the first association degree of the evaluation data. Further, the output unit 14 may generate the evaluation result without performing the processing of the evaluation data, for example.

  The output unit 14 outputs the generated evaluation result. The output unit 14 may output the evaluation result to the output unit 109 via the I / F 107, or may output the evaluation result to an arbitrary device via the I / F 105, for example.

<Input unit 15>
The input unit 15 receives the evaluation video data transmitted from the imaging device 3 via the I / F 105 and receives various types of information input from the input part 108 via the I / F 106, for example. In addition, the input unit 15 may receive evaluation video data stored in the server 2, for example. The input unit 15 may receive evaluation video data or the like via a storage medium such as a portable memory. The input unit 15 receives, for example, update data created by the user based on the evaluation result, learning data used to update the association degree, and the like.

<Update unit 16>
For example, when the relationship between the past video data and the reference data is newly acquired, the update unit 16 reflects the relationship in the association degree. As data to be reflected in the association degree, for example, update data including video data newly created by the user and reference data corresponding to the video data is used. In addition, as the data to be reflected in the association degree, for example, learning data created by the user based on the evaluation result is used.

<Server 2>
The server 2 stores data (database) related to various types of information. In this database, for example, information sent via the public communication network 4 is accumulated. The server 2 may store information similar to the information DB 11, for example, and may transmit / receive various information to / from the event evaluation support apparatus 1 via the public communication network 4. As the server 2, for example, a database server on a network may be used. The server 2 may be used instead of the storage unit 104 and the information DB 11 described above.

<Photographing device 3>
The imaging device 3 captures the evaluation video data and transmits it to the event evaluation support device 1. As the image capturing device 3, for example, a device such as a security camera or a pinhole camera that can acquire images including images and sounds is used. As the imaging device 3, for example, a drone equipped with a camera, a mobile terminal with a camera, or the like may be used. Note that the imaging device 3 may include, for example, the configuration of the event evaluation support device 1.

<Public communication network 4>
The public communication network 4 (network) is an Internet network or the like to which the event evaluation support device 1 and the imaging device 3 are connected via a communication circuit. The public communication network 4 may be a so-called optical fiber communication network. The public communication network 4 is not limited to a wired communication network, and may be realized by a wireless communication network.

(Embodiment: Operation of Event Evaluation Support System 100)
Next, an example of operation | movement of the event evaluation assistance system 100 in this embodiment is demonstrated. FIG. 11 is a flowchart showing an example of the operation of the event evaluation support system 100 in the present embodiment.

<Acquisition means: S110>
First, evaluation video data to be evaluated is acquired (acquisition means S110). The acquisition unit 12 acquires the evaluation video data generated by the imaging device 3 via the input unit 15, and also receives the evaluation video data via a storage medium such as the public communication network 4 or a portable memory. You may get it. The acquisition unit 12 may acquire, for example, person information in addition to the evaluation video data. The acquisition unit 12 may store the acquired evaluation video data and the like in the information DB 11.

  For example, the acquisition unit 12 acquires video data for evaluation captured during a certain period. When acquiring from the imaging device 3, the acquisition unit 12 may acquire evaluation video data taken intermittently as well as evaluation video data taken continuously. The timing at which the acquisition unit 12 acquires the evaluation video data from the imaging device 3 is arbitrary. Note that the shooting conditions and the like of the evaluation video data acquired by the acquisition unit 12 are arbitrary.

<Evaluation means: S120>
Next, with reference to the reference database, evaluation data including the first degree of association between the evaluation video data and the reference data is acquired (evaluation means S120). The evaluation unit 13 acquires evaluation video data from the acquisition unit 12 or the information DB 11 and acquires a reference database from the information DB 11.

  The evaluation unit 13 can calculate the first association degree between the video data for evaluation and the reference data by referring to the reference database. The evaluation unit 13 selects, for example, past video data that matches, partially matches, or is similar to the evaluation image data, calculates the first association degree based on the corresponding association degree, and, for example, classifies the reference database as a classifier The first association degree may be calculated using the above algorithm. The evaluation unit 13 may store the calculated first association degree and the acquired evaluation data in the information DB 11.

  The evaluation unit 13 refers to, for example, a reference database, calculates a first association degree in time series with respect to evaluation video data shot in a certain period, and obtains evaluation data including the first association degree. Note that the unit of time series in the calculation is arbitrary.

  The evaluation unit 13 refers to, for example, a reference database, calculates a first association degree with respect to the entire evaluation video data photographed in a certain period, and acquires evaluation data including the first association degree. Note that the fixed period during which the video data for evaluation is photographed is arbitrary.

  For example, the evaluation unit 13 may calculate a first association between the combination of the image data and the sound data and the reference data with reference to the reference database shown in FIG.

  For example, the evaluation unit 13 refers to the reference database illustrated in FIG. 8 and calculates the first similarity between the image data and the synthesized data and the second similarity between the sound data and the synthesized data. A first degree of association between the combined data and the reference data may be calculated. Note that the calculation method of the first similarity and the second similarity is the same as the first relevance.

  For example, the evaluation unit 13 may refer to the reference database illustrated in FIG. 9 to calculate the first association degree between the combination of the image data, the sound data, and the text data and the reference data.

  The evaluation unit 13 refers to, for example, the reference database illustrated in FIG. 10, and combines the first similarity between the image data and the synthesized data, the second similarity between the sound data and the synthesized data, and the text data. After calculating the third similarity with the data, the first association between the combined data and the reference data may be calculated.

<Output means: S130>
Next, an evaluation result is generated based on the evaluation data, and the evaluation result is output (output unit S130). The output unit 14 may acquire evaluation data from the evaluation unit 13 or the information DB 11, and may acquire a format for displaying the evaluation result from the information DB 11, for example.

  The output unit 14 generates an evaluation result based on the evaluation data with reference to, for example, a format. The output unit 14 generates an evaluation result in a text format as shown in FIG. 2A, for example, and also evaluates in a format using a part of video data for evaluation as shown in FIG. 2B, for example. Results may be generated.

  Thereafter, the output unit 14 outputs the evaluation result. The output unit 14 outputs the evaluation result to the output part 109. The output unit 14 may output an evaluation result based on, for example, a result obtained by comparing a preset notification reference value with the first association degree. In this case, for example, when the notification reference value is set to “90% or more”, the evaluation result is output only when the first association degree is 90% or more. That is, the notification reference value is an arbitrary threshold value, and conditions such as the notification reference value greater than or less than the notification reference value can be arbitrarily set.

<Update means: S140>
Thereafter, for example, when a new relationship between past video data and reference data is acquired, the relationship may be reflected in the association degree (update means: S140). For example, the update unit 16 acquires update data newly created by the user and reflects the update data in the similarity. In addition, for example, the update unit 16 acquires learning data created by the user based on the evaluation result, and reflects it in the similarity. The update unit 16 calculates and updates the similarity using, for example, machine learning. For example, a convolutional neural network is used for the machine learning. Note that the update unit 16 newly acquires, for example, at least one of a relationship between image data and synthesized data, a relationship between sound data and synthesized data, and a relationship between text data and synthesized data. In some cases, the relationship may be updated to the similarity. When updating the similarity, it is not necessary to update the association degree. For this reason, it is possible to minimize the load accompanying the update of the reference database.

  Thereby, the operation of the event evaluation support system 100 in the present embodiment is completed. Note that whether or not the above-described updating unit S140 is performed is arbitrary. Moreover, you may make a computer perform the said operation | movement as an event evaluation assistance program in this embodiment.

(Embodiment: Evaluation video data, past video data, and reference data)
Next, evaluation video data, past video data, and reference data will be described. The evaluation video data, the past video data, and the reference data include different data for each use environment (category) of the event evaluation support system 100 as shown in FIGS. In the reference database, data corresponding to each category is stored according to the situation. Hereinafter, characteristics by category will be described. 3 and 4 show examples of data corresponding to each category.

<TV program production>
In the category “television program production”, the event evaluation support system 100 outputs an audience rating prediction based on the television program being produced (for example, FIG. 12A). Here, the “TV program” indicates a program such as an Internet TV distributed via the Internet, in addition to a program broadcast by terrestrial digital TV broadcasting.

  In this case, as the evaluation video data, a video related to the television program being produced is used. That is, the evaluation video data is unpublished video data. At this time, the image data includes, for example, the image of the television program being produced, the action of the photographed animal and the performer, and also includes, for example, the background, flip board, explanatory material, telop, wipe, and the like. The sound data includes the sound that flows in the television program being produced, the voice and tone of the performers, the cry of animals, and the like. The video data for evaluation may include, for example, text data corresponding to sounds flowing in a television program being produced, contents said by a performer, animal calls, and the like.

  In addition, as past video data, a video related to a broadcast television program is used. That is, the past video data is a video released to a plurality of viewers. At this time, the past image data includes, for example, an image of a television program that has already been broadcast, the action of a photographed animal or performer, and the like, for example, a background, a flip board, explanatory material, a telop, a wipe, and the like. Past sound data includes sounds that flow in already broadcast television programs, performers' voices and tone, animal calls, and the like. The past evaluation video data may include past text data corresponding to, for example, sound flowing in a television program that has already been broadcast, contents said by a performer, and animal calls.

  As reference data, an audience rating (for example, 0% to 100%) corresponding to past video data that has already been released is used. At this time, the evaluation result includes prediction of the audience rating for the video data for evaluation, and includes, for example, the transition of the predicted audience rating, the tendency and characteristics of the program that can obtain the audience rating, and the like.

  In the above case, for example, in the evaluation unit S120, the evaluation unit 13 divides the evaluation video data into two or more arbitrary periods, for example, and between the divided evaluation video data and the reference data (viewing rate) The first association degree is calculated, and evaluation data is acquired. Thereafter, in the output unit S130, the output unit 14 generates an evaluation result including an evaluation of the temporal transition of the audience rating based on the first association degree. For this reason, it is possible to confirm the prediction of the temporal transition of the audience rating for the evaluation video data. As a result, improvement points and the like can be grasped before the television program is released, and the quality of the television program can be improved.

  For example, in the output unit S130, the output unit 14 may include, for example, at least one of image data, sound data, and text data having a high first association degree with respect to a high audience rating in the evaluation result. For this reason, it is possible to easily grasp the tendency and characteristics of the program that can obtain the audience rating with reference to the evaluation result. As a result, it is possible to easily grasp improvement points and the like before releasing the television program, and it is possible to further improve the quality of the television program.

<Economic information>
In the category “economic information”, the event evaluation support system 100 outputs economic information prediction or the like based on the public video (for example, FIG. 12B). Here, the “economic information” indicates a change over time in stocks, exchange rates, commodity transactions, or the like. “Public video” refers to a video related to economic information such as economic news programs, company meetings during IPO (Initial Public Offering), meetings at FOMC (Federal Open Market Committee), and the like.

  In this case, as the evaluation video data, an image related to economic information during broadcasting (during transmission) or immediately after broadcasting (immediately after transmission) is used. That is, the evaluation video data is a public video that is being broadcast live, and is a video that can be used for stock trading or the like based on the result of viewing by the viewer. At this time, the image data may include a background video, a flip board, explanatory material, a telop, a wipe, etc., in addition to the image of the public video and the action of the photographed performer. The sound data includes the sound that flows in the public video, the voice and tone of the performers, and the like. The evaluation video data may include text data corresponding to, for example, sound flowing in the public video, contents said by the performer, or the like.

  Also, as past video data, a video related to economic information released in the past is used. In other words, the past video data is a public video that may affect the economic situation, as it is assumed that stock trading or the like has been performed based on the result viewed by the viewer. At this time, the past image data may include, for example, a background, a flip board, explanatory material, a telop, a wipe, and the like, in addition to the image of the video released in the past and the action of the photographed performer. The past sound data includes sounds flowing in videos released in the past, voices and tone of performers, and the like. The past video data may include past text data corresponding to, for example, sound flowing in a video released in the past, contents said by a performer, or the like.

  Also, as the reference data, past economic information trends with respect to published past video data (for example, up, down, or no change of any numerical value) are used. For example, past stocks, exchange rates, and futures transactions Of at least one of the past price movements. The reference data may include, for example, company performance (consolidated settlement, interim settlement, etc.). At this time, the evaluation result includes prediction of economic information for the video data for evaluation, for example, price fluctuation of the predicted stock price, characteristics when the stock price rises, predicted corporate performance transition, corporate performance is good And the like. Note that the time and period during which price movements such as stock prices can be predicted can be arbitrarily set, and may be set, for example, on the next day in addition to immediately after 30 minutes of broadcasting a public video.

  In the above case, for example, in the acquisition unit S110, the acquisition unit 12 acquires the price movement for evaluation indicating the price movement of at least one of stocks, exchange rates, and futures transactions corresponding to the time when the video data for evaluation is taken. Also good. The evaluation price movement indicates, for example, a time course of the market price of at least one of stocks, exchange rates, and futures transactions immediately before the evaluation video data is released. At this time, in the output unit S130, the output unit 14 generates an evaluation result including the evaluation of the temporal transition of the evaluation value movement based on the first association degree. For this reason, it is possible to easily obtain the evaluation of the temporal transition of the price movement based on the influence of the evaluation video data. This makes it possible to examine a business plan of a company etc. using the evaluation result as a material, and to increase the options of business strategy.

  Further, for example, in the output unit S130, the output unit 14 may include at least one of image data, sound data, and text data having a high first association with the past price movement in the evaluation result. For this reason, it is possible to easily grasp the characteristics of the public video when the stock price increases with reference to the evaluation result. This makes it possible to further increase business strategy options using the characteristics of the public video as one material.

  In addition, for example, instead of the above-described price fluctuation of the predicted stock price and the feature when the stock price rises, the evaluation result may include the transition of the predicted corporate performance and the feature when the corporate performance improves. . In this case as well, as described above, it is possible to further increase business strategy options using the characteristics of the public video as one material.

<Crime scene>
In the category “crime scene”, the event evaluation support system 100 outputs, for example, information for identifying a person who is likely to commit a crime based on a video imaged by the imaging device 3 such as a security camera (for example, FIG. 2 (b)). Here, “crime scene” refers to, for example, shoplifting in a convenience store, and indicates a crime that can occur in a place where a security camera or the like can be installed.

  In this case, for example, a video photographed by the photographing device 3 is used as the evaluation video data. The evaluation video data is a video in real time or near real time. At this time, the image data includes an image captured by the image capturing device 3 and a human action, and also includes, for example, a captured environment, brightness, displayed merchandise, and the like. The sound data includes a sound photographed by the photographing device 3, a human voice, a tone, and the like. The evaluation video data may include, for example, text data corresponding to sound captured by the image capturing device 3, contents spoken by a person, and the like.

  Further, as the past video data, a video photographed by the photographing device 3 at the crime scene before and after a crime occurred in the past is used. At this time, the past image data includes images taken before and after the crime at the crime scene and actions of the criminal, as well as, for example, the photographed environment, brightness, displayed products, and the like. The past sound data includes sounds taken before and after the crime at the crime scene, the voice and tone of the offender, the voice and tone of the victim, and the like. The past video data may include past text data corresponding to, for example, sounds taken before and after the crime at the crime scene, content said by the criminal, content said by the victim, and the like.

  In addition, information regarding the type of crime (for example, shoplifting, robbery, etc.) is used as reference data. At this time, the evaluation result includes information for identifying a person who may possibly commit the crime.

  In the above case, in the output unit S130, the output unit 14 may output the evaluation result based on, for example, a result obtained by comparing a preset notification reference value with the first association degree. Thereby, the output part 14 can output only when the person who may reach a crime is specified among the persons image | photographed by the video data for evaluation. Thereby, generation | occurrence | production of a crime can be prevented.

  For example, in the output unit S130, the output unit 14 may include, for example, at least one of image data, sound data, and text data having a high first association degree with respect to the crime type in the evaluation result. For this reason, it is possible to easily grasp the characteristics of the video when a crime can occur with reference to the evaluation result. This makes it possible to easily consider improvement plans for environments and places where crimes can occur.

<Game production>
The event evaluation support system 100 outputs, as an evaluation result, a prediction of a charge amount for a game based on a video shot when a user is playing a game under development in the category “game production”.

  In this case, for example, a video photographed by the photographing device 3 such as a user portable terminal is used as the evaluation video data. At this time, the image data includes images taken when a game under development is being performed, user actions, and the like. The sound data includes sounds taken when a game under development, a user's voice, tone, and the like are taken. The evaluation video data may include, for example, text data corresponding to a sound taken when a game under development is being performed, a content said by the user, or the like.

  Further, as the past video data, a video shot by the shooting device 3 when the user is playing a game that has been released or sold is used, and is associated with, for example, the user's charge amount, the number of logins, the play time, and the like. Recorded images are used. At this time, the past image data includes an image taken when a game that has been released or sold is being executed, an action of the user, and the like. The past sound data includes a sound taken when a game that has been released or sold is being implemented, a user's voice or tone, and the like. The past video data may include past text data corresponding to, for example, a sound captured when a game that has been released or sold is being executed, a content that the user has said, and the like.

  In addition, as reference data, a charge amount (for example, a total of XX yen, □□ name is charged) associated with past video data, the number of logins, play time, and the like are used. That is, the reference data includes at least one of a charge amount for game progress information, the number of logins, and a play time. At this time, in addition to the estimation of the charge amount as an evaluation result, for example, the prediction of the number of logins, the prediction of the play time, the feature when the charge amount increases, the feature when the login number increases, the case where the play time becomes long Features etc. are included. As reference data, for example, the number of user comments associated with video data may be used.

  In the above case, the past video data includes progress information regarding the progress of the game, and the evaluation video data includes development progress information regarding the progress of the game under development. In the output unit S130, the output unit 14 generates an evaluation result including an evaluation of a temporal transition of at least one of the charge amount for the development progress information, the number of logins, and the play time based on, for example, the first association degree. For this reason, it is possible to confirm the prediction of the charge amount for the evaluation video data. As a result, the improvement points of the game under development can be grasped, and the quality of the game under development can be improved.

  For example, in the output unit S130, the output unit 14 may include, for example, at least one of image data, sound data, and text data having a high first relation with respect to a high charge amount or the like in the evaluation result. For this reason, it is possible to easily grasp the tendency and characteristics of obtaining the charge amount by referring to the evaluation result. As a result, the improvement points of the game under development can be easily grasped, and the quality of the game under development can be easily improved.

<Interrogation>
The event evaluation support system 100 outputs an estimation result of whether the suspect is guilty or not guilty based on the video photographed in the investigation room in the category “interrogation”.

  In this case, as the evaluation video data, for example, a video shot by the shooting device 3 such as a security camera or pinhole camera glasses is used. At this time, the image data includes an image taken at the time of the examination, the action of the suspect, the action of the interrogator, and the like. The sound data includes sounds taken at the time of investigation, suspect's voice and tone, interviewer's voice and tone, and the like. The video data for evaluation may include, for example, text data corresponding to the sound photographed at the time of investigation, the contents said by the suspect, the contents said by the interrogator, and the like.

  In addition, as the past video data, a video or the like taken when a survey (past survey) is performed in a case where a judgment is finalized is used. At this time, the past image data includes an image taken at the time of past examination, a suspect's action, an interrogator's action, and the like. The past sound data includes sounds taken during past investigations, suspect's voice and tone, interviewer's voice and tone, and the like. The past video data may include past text data corresponding to, for example, sounds taken during past investigations, contents said by the suspect, contents said by the interrogator, and the like.

  Further, as reference data, a case corresponding to past video data is used, and for example, authenticity of past survey information is used. At this time, the evaluation result includes information for estimating whether the suspect is guilty or not guilty, and for example, includes information for estimating the authenticity of the suspect's testimony. This makes it possible to obtain a quantitative evaluation result based on the suspect's motion, sound, and the like, and to improve the efficiency of the investigation.

<Relationship>
The event evaluation support system 100 outputs, as an evaluation result, a prediction of an impression received by the evaluator based on a video taken during the conversation between the evaluation target person and the evaluator in the category “human relation”.

  In this case, as the evaluation video data, for example, a video shot by the shooting device 3 such as a security camera or pinhole camera glasses is used. As the evaluation video data, for example, a video shot for conversation simulation or the like for improving communication ability is used. At this time, the image data includes an image taken at the time of the conversation simulation, the operation of the evaluation subject, the operation of the evaluator, and the like. The sound data includes a sound photographed during the simulation of the conversation, the voice and tone of the evaluation subject, the voice and tone of the evaluator, and the like. The evaluation video data may include text data corresponding to, for example, a sound photographed during a conversation simulation, a content said by the evaluation subject, a content said by the evaluator, and the like.

  In addition, as the past video data, a video photographed during a past conversation is used, and a video associated with an impression received by the evaluator with respect to the conversation of the evaluation subject is used. At this time, the past image data includes an image taken at the time of conversation, an action of the evaluation subject, an action of the evaluator, and the like. The past sound data includes sounds taken during conversation, the voice and tone of the evaluation subject, the voice and tone of the evaluator, and the like. The past video data may include past text data corresponding to, for example, a sound captured during conversation, a content said by the evaluation subject, a content said by the evaluator, and the like.

  Further, as reference data, information related to an impression received by an evaluator associated with past video data is used, and for example, a method for forming a good human relationship is used. At this time, the evaluation result includes a prediction of an impression received by the evaluator, and a feature analysis result that improves the impression, for example. This makes it possible to quantitatively evaluate the impression given to the other party during conversation, and to easily propose an improvement plan for the communication method of the evaluation target person, thereby efficiently improving the communication ability of the evaluation target person It becomes possible. In addition, for example, when the person to be evaluated in the town invites a person who does not know (evaluator) such as a meal, a prediction as to whether or not the evaluator agrees can be output as an evaluation result.

<Sales, customer service>
The event evaluation support system 100 outputs a sales forecast or the like as an evaluation result based on a video taken when a salesperson is serving a customer in the category “sales, customer service”.

  In this case, as the evaluation video data, for example, a video shot by the shooting device 3 such as a security camera or pinhole camera glasses is used. As the evaluation video data, for example, a video taken for a customer service simulation or the like is used. At this time, the image data includes an image taken at the time of customer service simulation, salesperson's operation, customer's operation, and the like. The sound data includes sounds taken during a customer service simulation, salesperson's voice and tone, customer's voice and tone, and the like. The video data for evaluation may include, for example, text data corresponding to sounds taken during a customer service simulation, contents said by a salesperson, contents said by a customer, and the like.

  Further, as past video data, a video taken at the time of customer service in the past is used, and a video linked to a sales result of a salesperson is used. At this time, the past image data includes images taken at the time of customer service, salesperson's actions, customer's actions, and the like. The past sound data includes sounds taken at the time of customer service, salesperson's voice and tone, customer's voice and tone, and the like. The past video data may include, for example, past text data corresponding to sounds taken at the time of customer service, contents said by salespersons, contents said by customers, and the like.

  Further, as reference data, sales of salespersons associated with past video data are used, for example, typical behavior of salespersons (for example, a customer service manual that leads to sales improvement), and the like. At this time, the evaluation result includes a sales prediction for the customer service simulation, and also includes, for example, a characteristic analysis result for increasing sales. As a result, it is possible to quantitatively evaluate the impact of customer service, and it is also possible to easily propose proposals for improving the customer service method, leading to operational improvements and sales increases. .

<Counseling>
In the category “counseling”, the event evaluation support system 100 outputs, as an evaluation result, a prediction of the counselor's condition based on the video taken during counseling between the counselor and the counselor.

  In this case, as the evaluation video data, for example, a video shot by the shooting device 3 such as a security camera or pinhole camera glasses is used. At this time, the image data includes an image taken at the time of counseling, an operation of a counselor, an operation of a counselor, and the like. The sound data includes sounds taken at the time of counseling, voice and tone of a consultant, voice and tone of a counselor, and the like. The video data for evaluation may include text data corresponding to, for example, sound taken at the time of counseling, content said by a consultant, content said by a counselor, and the like. Note that “counseling” includes, for example, when the counselor and the counselor have a dialogue, for example, when the counselor is watching a rehabilitation video or the like.

  In addition, as the past video data, a video taken in the past counseling is used, and a video linked to the contents of the consultant's medical record or the like is used. At this time, the past image data includes an image taken at the time of past counseling, an action of a counselor, an action of a counselor, and the like. The past sound data includes sounds taken during past counseling, voices and tone of the consultant, voices and tone of the counselor, and the like. The past video data may include past text data corresponding to, for example, sounds taken during past counseling, content said by a consultant, content said by a counselor, and the like.

  Further, as the reference data, the content of the consultant's medical chart associated with the past video data (for example, the degree of improvement of the consultant's symptoms) is used. At this time, the evaluation result includes prediction of the consultant's condition, and includes, for example, prediction of a counseling method that improves without worsening symptoms, and characteristic analysis of the counseling method that improves symptoms. As a result, the condition of the consultant can be predicted quantitatively, and an improvement plan for the counseling method can be easily proposed to the consultant, which can lead to an early recovery of the consultant. .

<Performance>
The event evaluation support system 100 outputs a prediction of audience satisfaction as an evaluation result based on the video shot at the performance in the category “performance”. Here, “performance” refers to a performance, music concert, lecture, seminar, etc. that mobilizes multiple audiences.

  In this case, as the evaluation video data, for example, a performance video shot by the shooting device 3 such as a security camera is used. At this time, the image data includes an image taken at the time of the performance, an action of the audience, an action of a performer (for example, an actor, a speaker, etc.) and the like. The sound data includes sounds taken at the performance, audience voice and tone, performer voice and tone, and the like. The video data for evaluation may include, for example, text data corresponding to sounds taken at the performance, contents said by the audience, contents said by the performers, and the like.

  In addition, as the past video data, a video taken at the past performance is used, and a video linked to the contents of the questionnaire described by the audience is used. At this time, the past image data includes an image taken at the past performance, an action of the audience, an action of the performer, and the like. Past sound data includes sounds taken during past performances, audience voice and tone, performer voice and tone, and the like. The past video data may include past text data corresponding to, for example, sounds taken at past performances, contents said by the audience, contents said by the performers, and the like.

  In addition to the contents of the audience questionnaire as reference data, for example, performance indicators that attract interest, performance indicators that increase box office performance, performance performance indicators, emotional information (joyful, fun, pleasant, refreshing, Proud, fear, etc.) are used. At this time, the evaluation results include prediction of audience satisfaction, and include, for example, feature analysis of performances that increase revenue, feature analysis of performances that excite, feature analysis of performances that attract interest, and the like. As a result, the satisfaction level of the audience with the content of the performance can be quantitatively evaluated, and an improvement plan for improving the quality of the performance can be easily proposed, and the quality of the performance can be improved. Become.

<Diagnosis>
The event evaluation support system 100 outputs, as an evaluation result, a prediction of the patient's condition and the like based on the video taken at the time of the patient's examination in the category “diagnosis”.

  In this case, as the video data for evaluation, for example, a video at the time of examination photographed by the photographing device 3 such as a security camera or pinhole camera glasses is used. At this time, the image data includes an image taken at the time of the examination, the action of the patient, the action of the examiner, and the like. The sound data includes sounds taken at the time of examination, patient voice and tone, examiner voice and tone, and the like. The evaluation video data may include, for example, text data corresponding to sounds taken at the time of examination, contents said by the patient, contents said by the examiner, and the like.

  In addition, as the past video data, a video taken at the past examination is used, and a video linked to the contents of the patient's medical record and the like is used. At this time, the past image data includes an image taken at the past examination, a patient's action, and an examiner's action. The past sound data includes sounds taken at the past examination, voice and tone of the patient, voice and tone of the examiner, and the like. The past video data may include past text data corresponding to, for example, sounds taken at the past examination, contents said by the patient, contents said by the examiner, and the like.

  Further, as reference data, the contents of the patient's medical record (symptom improvement, etc.) are used, and a medication method, a treatment method and the like are used. At this time, the evaluation result includes prediction of the patient's condition, for example, prediction of a medication method that improves without worsening symptoms, prediction of a treatment method that improves without worsening symptoms, and treatment method that improves symptoms Feature analysis and the like. As a result, the patient's condition can be predicted quantitatively, and an improvement plan for the examination method can be easily proposed to the examiner, which can lead to an early recovery of the patient.

<Containment after surgery>
The event evaluation support system 100 outputs, as an evaluation result, a prediction of a patient's condition based on an image taken at the time of a patient's operation in the category “condition after operation”.

  In this case, as the evaluation video data, for example, a video at the time of surgery photographed by the photographing device 3 such as a security camera or pinhole camera glasses is used. At this time, the image data includes an image taken at the time of surgery, patient motion, surgeon motion, and the like. The sound data includes sounds taken at the time of surgery, patient voice and tone, surgeon voice and tone, and the like. The video data for evaluation may include text data corresponding to, for example, sounds taken at the time of surgery, contents said by a patient, contents said by an operator, and the like.

  In addition, as the past video data, a video taken at the time of past surgery is used, and a video linked to the contents of the patient's medical record or the like is used. At this time, the past image data includes images taken at the time of past surgery, patient motion, surgeon motion, and the like. The past sound data includes sounds taken at the time of past surgery, patient voice and tone, operator voice and tone, and the like. The past video data may include past text data corresponding to, for example, sounds taken at the time of past surgery, contents said by the patient, contents said by the surgeon, and the like.

  Further, as the reference data, the contents of the patient's medical chart (symptom improvement, etc.) are used, and for example, an operation method, a medication method, a treatment method, and the like are used. At this time, the evaluation result includes prediction of the patient's condition, for example, prediction of a medication method that improves without worsening symptoms, prediction of a treatment method that improves without worsening symptoms, and surgical method that improves symptoms Analysis of characteristics, analysis of characteristics of treatment methods for improving symptoms, and the like. As a result, the patient's condition can be predicted quantitatively, and an improvement plan of the surgical method can be easily proposed to the surgeon, leading to early recovery of the patient.

<Video content>
In the category “video content”, the event evaluation support system 100 outputs a prediction or the like of content that is interesting based on the video content being produced. Here, “video content” refers to movies, document videos, promotional videos, company introduction videos, and the like recorded and sold on a recording medium such as a DVD.

  In this case, a video relating to the video content being produced is used as the video data for evaluation. That is, the evaluation video data is unpublished video data. At this time, the image data includes the image of the video content being produced, the action of the photographed animal and the performer, and also includes, for example, the background, flip board, explanatory material, telop, wipe, and the like. The sound data includes the sound that flows in the video content being produced, the voice and tone of the performers, the cry of animals, and the like. The video data for evaluation may include, for example, text data corresponding to the sound flowing in the video content being produced, the content said by the performer, the cry of animals, and the like.

  Also, as the past video data, a video related to the published video content is used. That is, the past video data is a video released to a plurality of viewers. At this time, the past image data includes, for example, the image of the video content that has already been released, the action of the photographed animal or performer, and the like, for example, the background, flip board, explanatory material, telop, wipe, and the like. The past sound data includes the sound that flows in the video content that has already been released, the voice and tone of the performers, the cry of animals, and the like. The past evaluation video data may include past text data corresponding to, for example, sound flowing in a content video that has already been made public, the content said by a performer, the cry of an animal, and the like.

  Further, as the reference data, the degree of interest of the viewer is used, and for example, information related to emotions (joyful, fun, pleasant, refreshing, proud, fearful, etc.) is used. At this time, the evaluation result includes prediction of content that is of interest, and includes, for example, feature analysis of a method that evokes emotions. As a result, the evaluation of the video content being produced can be quantitatively evaluated, and an improvement plan for the video content being produced can be easily proposed, and the quality of the video content can be improved. Become.

<Traffic jam>
In the category “traffic jam”, the event evaluation support system 100 outputs a traffic jam prediction or the like as an evaluation result based on a photographed road image.

  In this case, as the evaluation video data, for example, a road image shot by the shooting device 3 such as a security camera is used. At this time, the image data includes a photographed road image, a car traveling on the road, and the like. The sound data includes exhaust sound of a vehicle traveling on the photographed road. The evaluation video data may include, for example, text data corresponding to the onomatopoeia of a sound produced by a car traveling on a photographed road.

  Further, as past video data, a video of a road photographed in the past is used, and a video associated with traffic congestion information or the like is used. At this time, the past image data includes an image of a road photographed in the past, a car traveling on the road, and the like. The past sound data includes the exhaust sound of a vehicle traveling on a photographed road. The past video data may include, for example, past text data corresponding to the onomatopoeia of sound produced by a car traveling on a photographed road.

  Moreover, traffic state information is used as reference data. At this time, prediction of traffic congestion is included as an evaluation result. As a result, traffic congestion can be predicted quantitatively, and a route for avoiding congestion, a congestion mitigation plan, and the like can be easily proposed.

<Aerial photography>
The event evaluation support system 100 outputs, as an evaluation result, a prediction of a harvest rate of cereals, vegetables, fruits, and the like based on an aerial image captured in the category “aerial imaging”.

  In this case, as the evaluation image data, for example, an image relating to a grain or the like taken aerial by the imaging device 3 such as a drone is used. At this time, the image data includes an aerial image, the degree of growth of grains, and the like.

  In addition, as past video data, a video of a grain or the like associated with a harvest rate taken in the past is used. At this time, the past image data includes images taken aerial in the past, the degree of growth of grains, and the like.

  As the reference data, a crop harvest rate is used. At this time, the evaluation result includes a prediction of the harvest rate of grains and the like. Thereby, it is possible to quantitatively predict the harvest rate of cereals and the like, and it is possible to easily propose a plan for improving the harvest rate of cereals and the like.

  Of the categories described above, the case where the output unit 14 generates the evaluation result including the evaluation of the temporal transition with respect to the evaluation video data is described in the case where the output unit 14 applies to the entire evaluation video data. An evaluation result including an evaluation predicted (for example, in the case of audience rating, average audience rating, highest audience rating, lowest audience rating) may be generated.

  According to the present embodiment, the evaluation unit S120 refers to the reference database, and acquires evaluation data including the first degree of association between the evaluation video data and the reference data. For this reason, reference data is linked to the evaluation video data, and a quantitative evaluation result of an event that may occur in the future can be acquired based on the evaluation video data. As a result, it is possible to improve the accuracy in the evaluation of events that may occur in the future.

  Further, according to the present embodiment, the evaluation unit S120 calculates the first relationship between the combination of the image data and the sound data and the reference data. For this reason, compared with the case where either one of image data and sound data is used, the feature of the video or the estimation of the emotion of the photographed person, or the affect of the other person who viewed the video or the behavior of the other person is affected. The accuracy of evaluating the possibility of giving can be improved. As a result, it is possible to further improve the accuracy of evaluation of events that may occur in the future.

  Further, according to the present embodiment, the evaluation unit S120 calculates the first association degree in time series for the evaluation video data photographed in a certain period. For this reason, the time-dependent change of evaluation with respect to video data for evaluation can be shown. As a result, it is possible to perform evaluation along with the temporal change of the video data for evaluation, and it is possible to perform detailed evaluation of events that may occur in the future.

  Further, according to the present embodiment, the output unit S130 generates an evaluation result including an evaluation of the temporal transition of the evaluation value movement based on the first association degree. For this reason, it is possible to easily obtain the evaluation of the temporal transition of the price movement based on the influence of the evaluation video data. This makes it possible to examine a business plan of a company etc. using the evaluation result as a material, and to increase the options of business strategy.

  Further, according to the present embodiment, the output unit S130 outputs an evaluation result based on a result obtained by comparing a preset notification reference value with the first association degree. For this reason, it is possible to control whether or not to output the evaluation result according to the setting condition of the notification reference value. Thereby, it is possible to output only when the evaluation result is necessary, and the user or the like does not always need to confirm the evaluation result. Accordingly, it is possible to reduce the work load of the user or the like.

  Further, according to the present embodiment, the output unit S130 generates an evaluation result including information for identifying a person who is likely to commit a crime among persons photographed in the evaluation video data based on the evaluation data. . For this reason, it becomes possible to suppress a crime in advance. In addition, it is possible to quantitatively evaluate a place where a large number of people come and go, and it is possible to suppress misperceptions and oversights including subjectives.

  Further, according to the present embodiment, the output unit S130 generates an evaluation result including an evaluation of the temporal transition of the audience rating based on the first association degree. For this reason, the temporal transition of the audience rating can be evaluated in advance for unpublished evaluation video data. As a result, improvement points and the like can be grasped before the video data is released, and the quality of the video data can be improved.

  Further, according to the present embodiment, the output unit S130 generates an evaluation result including an evaluation of a temporal transition such as a charge amount for the development progress information based on the first association degree. For this reason, it is possible to evaluate the timing, frequency, etc. charged in a game under development. As a result, improvement points and the like can be grasped before the game is sold, and the quality of the game can be improved.

  Further, according to the present embodiment, the evaluation unit S120 calculates the first association degree with respect to the entire evaluation image data photographed in a certain period. For this reason, it is possible to comprehensively evaluate the evaluation video data including the image data and the sound data captured during a certain period. This makes it possible to greatly expand the selection range of events that can be quantitatively evaluated.

  According to the present embodiment, the past video data includes three or more levels of similarity between the past image data and the synthesized data and between the past sound data and the synthesized data. For this reason, when past image data and past sound data are newly stored (updated) in the reference database, it is not necessary to update the association degree each time. Thereby, it is possible to suppress the load for updating the association degree with the update of the reference database.

  In addition, according to the present embodiment, when the relationship between the past video data and the reference data is newly acquired, the update unit S140 reflects the relationship in the association degree. For this reason, the relevance can be easily updated, and the accuracy of evaluation can be improved.

  Further, according to the present embodiment, the first relationship between the combination of image data, sound data, and text data and the reference data is calculated. For this reason, by adding text data to the combination, it is possible to further improve the accuracy of estimating the characteristics or emotions of the person or evaluating the possibility of affecting the emotions or actions of others. As a result, it is possible to further improve the accuracy of evaluation of events that may occur in the future.

  Further, according to the present embodiment, the evaluation unit 13 refers to the reference database, and acquires evaluation data including the first association degree between the video data for evaluation and the reference data. For this reason, reference data is linked to the evaluation video data, and a quantitative evaluation result of an event that may occur in the future can be acquired based on the evaluation video data. As a result, it is possible to improve the accuracy in the evaluation of events that may occur in the future.

  According to the present embodiment, the computer is caused to acquire evaluation data including the first association degree between the evaluation video data and the reference data by referring to the reference database. For this reason, reference data is linked to the evaluation video data, and a quantitative evaluation result of an event that may occur in the future can be acquired based on the evaluation video data. As a result, it is possible to improve the accuracy in the evaluation of events that may occur in the future.

  In addition, the present embodiment is characterized in that the evaluation video data is evaluated based on the degree of association (first degree of association) set in three or more stages. The association degree can be described by a numerical value of 0 to 100%, for example, but is not limited thereto, and may be configured at any stage as long as it can be described by a numerical value of three or more levels.

  Based on such association, reference data can be displayed in descending order of association in reference data selected as evaluation result candidates for evaluation video data. By displaying in the order of association in this way, the user or the like can preferentially evaluate an event that is likely to occur in the future. On the other hand, since events that are unlikely to occur in the future can be displayed without being excluded, the user or the like can evaluate without overlooking.

  In addition to the above, according to the present embodiment, evaluation can be performed without overlooking even when the relevance is as extremely low as 1%. Even reference data with a very low degree of association indicates that it is connected as a slight sign, and it is possible to suppress oversight and misidentification.

  Although an embodiment of the present invention has been described, this embodiment is presented as an example and is not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

1: Event evaluation support device 2: Server 3: Imaging device 4: Public communication network 10: Housing 12: Acquisition unit 13: Evaluation unit 14: Output unit 15: Input unit 16: Update unit 100: Event evaluation support system 101: CPU
102: ROM
103: RAM
104: Storage unit 105: I / F
106: I / F
107: I / F
108: Input part 109: Output part 110: Internal bus DB11: Information S110: Acquisition means S120: Evaluation means S130: Output means S140: Update means

Event evaluation support system according to the fourteenth invention, Oite the second shot bright, the evaluation image data includes the text data corresponding to the contents of the sound data,
The evaluation unit refers to the reference database, and calculates the first association degree between the reference data and a combination of the image data, the sound data, and the text data.

Claims (16)

An event evaluation support system that supports evaluation of future events based on video data including images and sounds,
Acquisition means for acquiring video data for evaluation;
Reference database in which past video data acquired in advance, reference data used for evaluation of the past video data, and three or more levels of association between the past video data and the reference data are stored When,
An evaluation unit that refers to the reference database and obtains evaluation data including first relations of three or more stages between the evaluation video data and the reference data;
Output means for generating an evaluation result based on the evaluation data and outputting the evaluation result;
An event evaluation support system comprising: The video data for evaluation is
Image data including at least one of photographed person, animal, background, and explanatory material;
Sound data including at least one of a photographed human voice, animal voice, sound effect, and inserted music;
Including
The event evaluation according to claim 1, wherein the evaluation unit calculates the first association degree between the reference data and a combination of the image data and the sound data with reference to the reference database. Support system. The evaluation means refers to the reference database, and acquires the evaluation data including the first relevance calculated in time series with respect to the video data for evaluation taken in a certain period. The event evaluation support system according to claim 2. The acquisition means, corresponding to the time when the evaluation video data was taken, acquires an evaluation price movement indicating a price movement of at least one of a stock, a currency exchange, and a futures transaction,
The reference data includes a historical trend of at least one of past stocks, exchange rates, and futures transactions,
The event evaluation support system according to claim 3, wherein the output unit generates the evaluation result including an evaluation of a temporal transition of the evaluation value movement based on the first association degree. The event evaluation support system according to claim 4, wherein the evaluation video data and the past video data are images related to economic information. The event evaluation support system according to claim 3, wherein the output unit outputs the evaluation result based on a result obtained by comparing a preset notification reference value with the first association degree. The past video data includes images before and after the crime at the crime scene,
The reference data is information on the type of crime,
The event evaluation according to claim 6, wherein the output unit generates the evaluation result including information for identifying a person who is likely to commit a crime among persons photographed in the video data for evaluation. Support system. The past video data is a video released to a plurality of viewers,
The reference data is an audience rating corresponding to the published past video data,
The evaluation video data is unpublished video data,
The event output support system according to claim 3, wherein the output unit generates the evaluation result including an evaluation of a temporal transition of the audience rating based on the first relevance. The past video data includes progress information regarding the progress of the game,
The video data for evaluation includes development progress information regarding the progress of the game under development,
The reference data includes at least one of a charge amount for the progress information of the game, the number of logins, and a play time,
The output means generates the evaluation result including an evaluation of a temporal transition of at least one of a charge amount, a number of logins, and a play time for the development progress information based on the first association degree. Item 3. The event evaluation support system according to Item 3. The evaluation means refers to the reference database, and acquires the evaluation data including the first relevance calculated for the entire evaluation video data photographed in a certain period. Item 2. The event evaluation support system according to Item 2. The video data for evaluation and the past video data are a video of a security camera, a video in an investigation room, a video during conversation, a video during customer service, a video during counseling, a video during performance, a video during medical examination, or The event evaluation support system according to claim 10, further comprising an image during surgery. The past video data is
Past image data including at least one of a person, an animal, and a background photographed in the past;
Past sound data including at least one of a human voice, an animal voice, a sound effect, and an inserted song recorded in the past;
Synthetic data having the association degree with the reference data;
Three or more similarities between the past image data and the synthesized data, and between the past sound data and the synthesized data,
Including
The event evaluation support system according to any one of claims 1 to 11, wherein the association degree between the composite data and the reference data is stored in the reference database. The update means for reflecting the relationship in the association degree when a relationship between the past video data and the reference data is newly acquired is further provided. The event evaluation support system according to claim 1. The evaluation video data includes text data corresponding to the content of the sound data,
The evaluation means refers to the reference database, and calculates the first association degree between the reference data and a combination of the image data, the sound data, and the text data. The event evaluation support system according to any one of 2 to 13. An event evaluation support apparatus that supports evaluation of events that may occur in the future based on video data including images and sounds,
An acquisition unit for acquiring video data for evaluation;
Reference database in which past video data acquired in advance, reference data used for evaluation of the past video data, and three or more levels of association between the past video data and the reference data are stored When,
An evaluation unit that refers to the reference database and obtains evaluation data including the first association degree in three or more stages between the video data for evaluation and the reference data;
Generating an evaluation result based on the evaluation data, and outputting the evaluation result; and
An event evaluation support apparatus comprising: An event evaluation support program that supports evaluation of future events based on video data including images and sounds,
Obtain video data for evaluation,
Reference database in which past video data acquired in advance, reference data used for evaluation of the past video data, and three or more levels of association between the past video data and the reference data are stored And obtaining evaluation data including the first association degree of three or more stages between the video data for evaluation and the reference data,
An event evaluation support program that causes a computer to generate an evaluation result based on the evaluation data and output the evaluation result.

Images