JP2016053891A - Confirmation device and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a confirmation device capable of assisting confirmation of difference of evaluation to a subject between a site and a remote area, in a remote group interview.SOLUTION: A video acquisition part 21 acquires an interview video of a site, a video distribution part 23 distributes it to a remote area, and the distribution video is recorded as a viewing video. An evaluation storage part 33 designates a subject, receives evaluation information which evaluates a state of the subject, and then store the evaluation information, from each of the site and the remote area, in a state that a time stamp following to progress of the interview is applied. In association of the evaluation information to the distribution video on a time axis, a confirmation object video creation part 42 identifies a scene which is determined so that, the pieces of evaluation information from the site and from the remote area to the subject are different each other, as a confirmation object scene.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a confirmation device and a program that enable an effective review of an interview between an interviewer moderator and a remote analyst in a remote group interview.

  The remote group interview is conducted in cooperation with the moderator at the interview site and the analyst at the remote location. FIG. 1 is a diagram schematically showing a remote group interview environment. As shown in the figure, in the remote group interview, unlike the normal group interview, the analyst exists in the remote location L2 instead of the interview site L1. The moderator is present at the site L1 together with the subject, as in a normal group interview. Thus, in remote group interviews, there is an advantage that it is possible to collect subjects more easily because the analyst and the subjects do not need to coexist in the same place.

  However, as shown in FIG. 1, when viewing the interview site L1, the analyst views the video through a plurality of video cameras installed at the site L1 and relayed via the network N. At this time, since the analyst views the relay video of the selected camera by switching the video camera, the analyst may fail to grasp the test subject's behavior. This can cause analysts to miss or misinterpret the scene being interviewed.

  In order to prevent this situation, the interview is generally reviewed between the moderator and the analyst after the interview. In order to look back on the interview efficiently, it is necessary to extract only scenes where a difference in consciousness was born from the vast interview video shot with the video camera.

  Patent Document 1 is a related art related to improving the efficiency of such work. Patent Document 1 proposes a method for efficiently looking back on a missed scene in a conference from the line of sight of a conference participant. In this method, overlooking is determined based on whether or not the line of sight is directed to the gaze target.

JP 2006-185187 A JP 2010-102595 A

  However, since the method of Patent Document 1 is based on the position of a conference participant, it looks back on the missed scene that occurs because the video camera could not catch the target object during the remote group interview. I can't. In addition, a recognition error scene cannot be found.

  Here, in remote group interviews, it is effective to allow comments to be posted during the interview between the moderator and the analyst, and to compare the comments of both after the interview. It is considered one of the means.

  As a conventional technique related to the use of such a comment, Patent Document 2 proposes a method of determining the similarity of a comment between users from a pointing position in a video designated at the time of comment posting.

  However, the method of Patent Document 2 is intended to assist the exchange of opinions among a large number of users regarding the design and the like of an image such as a three-dimensional model in a blog or the like. For this reason, the method of patent document 2 is not directly applicable to the analysis of the recognition difference between the moderator side and the analyst side in the remote group interview.

  That is, in Patent Document 2, an image of each part of an image such as a target three-dimensional model is prepared in advance by associating it with structure, color, pattern, and other attribute information, and the part is designated by pointing to the user. Then, after inputting a comment, similarity between comments is determined by considering attribute information. In the method of Patent Document 2, it is highly likely that the comments are similar if the pointing positions are close, and the similarity of the comments is not determined in consideration of the content of the comments. Assuming that the technique of Patent Document 2 is applied to a remote group interview, in a scene where there is a recognition difference between the moderator side and the analyst side (recognition error scene), there is a comment with low content similarity even if the pointing position is close Although it may be posted, the method of Patent Document 2 cannot find such a recognition error scene.

  In remote group interviews, it is necessary to analyze video images. However, the method of Patent Document 2 is intended primarily for images and is not a method for video images.

  The present invention has been made in view of the above-described problems of the prior art, and it is an object of the present invention to provide a confirmation device and program capable of assisting in confirming a difference in evaluation with respect to the state of a subject between a site and a remote site in a remote group interview. .

  In order to achieve the above object, the present invention distributes a video of a group interview conducted for a subject at a site to a remote location, and the subject in the group interview at the site and / or a remote location after the end of the group interview. A confirmation device for assisting confirmation of the state of the image, a video acquisition unit that acquires the video of the group interview, a video distribution unit that distributes the video to a remote location based on the acquired video, and the distributed video A viewing image storage unit for storing viewing images at a remote location, and a time stamp corresponding to the progress of the group interview was given, and an evaluation was performed by designating a subject from each of the site and the remote location. An evaluation storage unit for receiving and storing information; and An evaluation-giving video generation unit that generates an evaluation-giving video by linking at the position of the group, and the evaluation-giving video is divided into a plurality of scenes, and evaluation information from the field and evaluation information from a remote place for the subject A confirmation target video generation unit that identifies a scene determined to be different as a confirmation target scene.

  In addition, the present invention is a program that causes a computer to function as the confirmation device.

  According to the present invention, the evaluation information from the site for the subject is received and stored from each of the site and the remote location in accordance with the progress of the group interview, and the evaluation information for evaluating the state is received and stored. Since the scene that is judged to be different from the evaluation information from the remote location is identified as the scene to be confirmed, the remote group interview assists in confirming the difference in evaluation with respect to the state of the subject between the site and the remote location can do.

It is a figure which shows a remote group interview environment typically. It is a functional block diagram of the confirmation apparatus concerning one embodiment. It is a flowchart of operation | movement of the confirmation apparatus which concerns on one Embodiment. It is a flowchart of the production | generation process of the confirmation object video | video by a confirmation object video generation part. It is a figure which shows typically the confirmation object image | video produced | generated by the confirmation object image | video production | generation part. It is a functional block diagram of the confirmation apparatus which concerns on one Embodiment different from FIG. It is a flowchart of operation | movement of the camera selection part which concerns on one Embodiment. It is a figure which shows the example of the 1st surface of a group interview. It is a figure which shows the example of a recognition error scene. It is a figure which shows the example of a miss scene. It is a figure for demonstrating the example of the screen which switches and confirms a camera image as an example in the case of the recognition error scene in FIG.

  FIG. 2 is a functional block diagram of a confirmation device according to an embodiment. The confirmation device 10 includes a video acquisition unit 21, an all video storage unit 22, a video distribution unit 23, a viewing video storage unit 24, an on-site evaluation input unit 31, a remote evaluation input unit 32, an evaluation storage unit 33, and an evaluation-added video generation Unit 41, confirmation target video generation unit 42, confirmation target video storage unit 43, video search unit 51, and video display unit 52.

  As shown in FIG. 2, the confirmation device 10 implements various data mediations between the interview site L1 and the remote site L2, thereby realizing remote group interviews, as well as video and evaluation information to be described later. By preserving the data, the moderator (and clerk) at the site L1 and the analyst (and clerk) at the remote site L2 will automatically identify the locations where the subject's reactions are different from each other. To detect. In this way, the moderator and / or the analyst can examine the location to be confirmed after the remote group interview. At this time, the moderator and the analyst can communicate with each other while communicating with each other, or the moderator and the analyst can individually consider them.

  2 can exist on the network N that connects the site L1 and the remote location L2 in FIG. 1, but any part of the verification device 10 can be configured at the site L1 and / or the site L1. It may exist in the remote location L2. In this case, the units of the confirmation apparatus 10 that are present at a distance may communicate with each other via the network N and perform necessary data exchange.

  FIG. 3 is a flowchart of the operation of the confirmation apparatus according to the embodiment. Hereinafter, the function of each part in FIG. 2 will be described while explaining each step in FIG. 3.

  In step S1, the moderator at the interview site L1 takes the lead, and a group interview is conducted for one or more (generally multiple) subjects at the site L1. At this time, in particular, the confirmation device 10 performs the following three processes in parallel: (1) video distribution process, (2) video recording process, and (3) evaluation information recording process. The three processes (1) to (3) are as follows.

  Process (1) is a process for distributing the video of the group interview at the site L1 to the remote location L2. By the processing (1), the analyst at the remote location L2 can observe the state of the group interview in real time and observe the response of the subject. That is, the remote group interview as described in FIG. 1 is performed.

  The configuration for realizing the process (1) is as follows. First, the video acquisition unit 21 acquires videos from a plurality of cameras (video cameras) arranged at the site L1 and passes them to the video distribution unit 23. It is preferable that the plurality of cameras at the site are arranged so as to be able to photograph each of the subjects participating in the group interview as a whole. An example of the arrangement will be described later with reference to FIG.

  Next, the video distribution unit 23 transmits the video of one camera selected by the switcher operated by the analyst existing in the remote location L2 from the video of the multiple cameras obtained from the video acquisition unit 21. By distributing to the display that exists in L2, the analyst can view the video. That is, the analyst at the remote location L2 views a video of the selected camera after selecting a desired camera with the switcher. When the subject to be noticed changes as the group interview progresses, the analyst switches the camera to the one that makes it easier to confirm the state of the subject to be newly noticed by manually operating the switcher, Watch the group interview.

  Note that the video is accompanied by audio recorded at the site L1, and the video is distributed and the audio is simultaneously distributed. In the following, “video” will be referred to as “audio” accompanying it.

  Process (2) is a process of recording each video in process (1). As described below, the entire video storage unit 22 and the viewing video storage unit 24 perform the processing (2).

  First, the entire video storage unit 22 receives all the videos of the plurality of cameras acquired by the video acquisition unit 21 and stores them individually. In addition, the viewing video storage unit 24 is a video distributed by the video distribution unit 23 to the remote location L2, that is, a viewing video that an analyst at the remote location L2 is watching while appropriately switching from all camera video using a switcher. Is received and memorized.

  Process (3) is the subject of attention at the time, such as by receiving questions from the moderator from the moderator side of the site L1 and the analyst side of the remote site L2 in accordance with the progress of the interview. This is a process of receiving information to be evaluated regarding the state of the subject, recording the evaluation information after giving a time stamp.

  The configuration for realizing the process (3) is as follows. First, regarding the acceptance of the input of evaluation information, the scribe who is at the site L1 is concerned with the subject who is focused on the progress of the interview at the site L1 with respect to the site side terminal 11 arranged at the interview site L1 Enter the evaluation information manually. The clerk at the site L1 inputs the evaluation information while directly viewing the interview scene.

  The site-side terminal 11 sends a time stamp to the site-side evaluation input unit 31 after adding a time stamp when it is input to the input evaluation information. The site-side evaluation input unit 31 that has received the evaluation information with the time stamp stores the information in the evaluation storage unit 33.

  Similarly, the clerk at the remote location L2 also receives evaluation information given to the subject who is the subject of attention in the progress of the interview by the analyst who views the interview video distributed by the video distribution unit 23. Input to the remote terminal 12.

  The remote terminal 12 adds a time stamp at the time of input to the input evaluation information, and transmits it to the remote evaluation input unit 32. The remote-side evaluation input unit 32 that has received the evaluation information with the time stamp stores the information in the evaluation storage unit 33.

  As described above, the evaluation information is accumulated in the evaluation storage unit 33 every time the evaluation information is input and transmitted from both the site L1 and the remote location L2 by the process (3). It becomes. The evaluation storage unit 33 stores the evaluation information including the distinction between the evaluation information transmitted from the site L1 or the remote location L2.

  Here, the evaluation information includes: [1] information for identifying who is the subject of the evaluation, and [2] information for evaluating a specific state of the subject in the progress of the group interview. Can be configured as information. Each clerk in the field L1 and the remote site L2 shall input the evaluation information in a form according to the configuration. The information [1] and [2] are as follows.

  [1] For information identifying who is the subject of evaluation, prepare a list of subjects to participate in preparation for the organizer before starting the group interview. You can enter it in the form of selection. For example, if there are a total of three participants, A, B, and C, you can create a list of A, B, and C.

  Alternatively, on the remote location L2 side, the subject may be specified by clicking on the screen on the distribution video. Also in this case, a list of subjects to participate in is prepared in advance, and by applying a known person detection technique from an image, it is possible to specify who is the subject at the clicked position.

  [2] The information for evaluating the state of the subject can be various embodiments as follows.

  In one embodiment, it is possible to input information for evaluating the state in the form of describing the state of the subject subject as a normal text. In one embodiment, predetermined emotion evaluation items can be prepared in advance, and information for evaluating the state can be input in a form selected from the items. For example, as the predetermined emotion evaluation item, two items are set in advance as to whether the emotion corresponds to “positive” or “negative”, and it is input by selecting which of the two items corresponds Can be.

  Further, in one embodiment, information for evaluating the state of the subject may be provided by inputting both the text and the predetermined emotion evaluation item. In this case, the text may be configured as information that explains why the selected emotion evaluation item is selected. For example, when “positive” is selected from the above-described two items, the specific behavior of the subject may be entered as text indicating that it is determined to be “positive”.

  In addition, the decision on when to enter the evaluation information is made in accordance with the prescribed business arrangements according to the purpose of the group interview, and the clerk (or clerk) at each of the site L1 and the remote site L2. To the analyst who gives instructions).

  For example, in the case of a group interview with the aim of gathering opinions about the terminal from the subject in the planning stage, the moderator at the site L1 introduces the new functions of the smartphone in sequence, Group interviews proceed in the form of visiting the operation feelings. In this case, since the user-friendliness, etc. of the new function is the information that the user wants to collect in the group interview, when reading how the subject feels usability, it is decided that the read emotion will be input as evaluation information. Just keep it.

  In particular, the evaluation information is determined as input regarding the subject's non-verbal responses (responses such as having a pleasant face, or conversely a troubled face). May be.

  In addition, the subject who inputs the above evaluation information to the terminals 11 and 12 in each of the site L1 and the remote site L2 has been described as a clerk. Reasons for the clerk to input are as follows. In other words, since the moderator serves as a moderator for the group interview at the site L1, it is difficult to perform the task of inputting evaluation information sequentially as the interview progresses. For this reason, the clerk plays the role of inputting at the site L1.

  Similarly, in the remote location L2, the analyst needs to continue watching the interview video, so the clerk is responsible for the input work. At this time, the evaluation information to be input may be communicated from the analyst to the scribe by dictation or the like.

  As described above, when step S1 of FIG. 3 in which the processes (1) to (3) are performed in parallel is completed for an entire group interview, a series of information (video and evaluation information) about the group interview is accumulated. It will be in a state and will progress to step S2.

  In step S2, with reference to the viewing video of the viewing video storage unit 24 accumulated in the above step S1 and the evaluation information of the evaluation storage unit 33, the evaluation imparted video generation unit 41 generates the evaluation imparted video, Proceed to step S3.

  The evaluation imparted video generated in step S2 means that each of the evaluation information stored in the evaluation storage unit 33 is the viewing video storage unit at a position corresponding to the time stamp assigned to the evaluation information. This is a video linked to 24 viewing videos on the time axis. In other words, as the group interview progresses with respect to the viewing video, the evaluation-added video is appropriately added with the evaluation information input at the site L1 or the remote location L2 in a form linked to the input time. is there.

  In step S3, by analyzing the evaluation imparted video generated in step S2, the confirmation target video generation unit 42 determines whether each scene of the evaluation imparted video is a scene to be confirmed. Then, the confirmation target video is generated and stored in the confirmation target video storage unit 43, and then the process proceeds to step S4.

  The generated confirmation target video may be configured as a video in which information indicating whether each scene is a scene to be confirmed is given to the evaluation-giving video, or each scene of the evaluation-giving video. It may be configured as an image in which only scenes determined as scenes to be confirmed are cut out and connected. Regardless of the configuration of the confirmation target video, the confirmation target video generation unit 42 (a) processes that divide the evaluation-added video into scenes, and (b) the divided scenes are scenes to be confirmed. And a process for determining whether or not there is. Each processing (a), (b) is as follows.

(a) Regarding the scene division of evaluation-given video, the scene is appropriately divided according to the video content for general video (ordinary video that is not associated with evaluation information such as evaluation-given video) Since many technologies have been proposed, such existing technologies may be used. For example, the existing technology disclosed in [Patent Document 3] below may be used.
[Patent Document 3] Japanese Unexamined Patent Application Publication No. 2008-283229 (Video Segmentation System, NEC Corporation)

  (b) Regarding the determination as to whether each divided scene is a scene to be confirmed, it is given from the moderator attached to the scene (that is, input from the on-site evaluation input unit 31) The evaluation information is compared with the evaluation information given from the analyst side (that is, input from the remote-side evaluation input unit 32), and the determination is made based on whether or not the information can be determined to be different from each other. do it.

  Here, since the evaluation information is assigned to the evaluation imparted video on the time axis, it is linked on the time axis of each scene (on the same time axis as the original evaluation imparted video) even after the scene is divided. The above judgment can be made by referring to the evaluation information.

  Whether or not the evaluation information is different may be determined by a predetermined determination method according to the embodiment in which the evaluation information is configured. As described above, the evaluation information is composed of [1] information on who is the subject subject and [2] contents of evaluating the subject. Although various embodiments are possible with respect to [2], in the following, as described above, the state is evaluated by specifying whether the emotion evaluation item is “positive” or “negative”. A case will be described as an example.

  In this case, in one embodiment, a scene determined to be confirmed can be further distinguished from a difference in meaning to be confirmed. That is, it is possible to provide two types of distinction: “missing scene” or “recognition error scene”.

  Here, the missed scene is defined as a scene in which evaluation information is input only from a moderator side to a subject. That is, the information of [1] above in the evaluation information given to the scene, that is, information on which subject is the target, and even one subject whose evaluation information is given only from the moderator side If it exists, it is determined that the scene is a scene to be confirmed and the type is a missed scene.

  Since the missed scenes include some response of the subject who was noticed only by the clerk at the site L1 and not (not missed) by the analyst at the remote location L2, it should be confirmed from this point of view. Become.

  In contrast to the above-described missed scenes, the recognition error scene is [A] a scene in which evaluation information is input only from the analyst side for a subject, or [B] a moderator for a subject. Evaluation information is input from both the analysis side and the analyst side, but it is defined as a scene in which the content evaluation in the evaluation information, that is, “positive” or “negative” results are different from each other.

  Of the two conditions [A] and [B], it is possible to determine whether [A] is satisfied in the same manner as the determination of the missed scene. For [B], after finding out what the subject in [1] mentioned above has in common between the evaluation information on the moderator side and the evaluation information on the analyst side, the “positive” evaluation in [2] above Can be determined that the condition [B] is satisfied when the other is given a “negative” evaluation.

  A recognition error scene is a scene in which the analyst has made a judgment that is different from the correct judgment under the assumption that the moderator has made the correct judgment (the analyst has identified a “recognition error”. The scene to be confirmed.

  It should be noted that the “moderator side” and “analyzer side” may be interchanged in the above definition to further define the “moderator side” recognition error scene.

  FIG. 5 is a diagram schematically showing the confirmation target video generated by the confirmation target video generation unit 42 in step S3. [1] is given in advance on the time axis by dividing the entire SC (scene set SC) of the initial evaluation imparted video into the series of scenes sc1 to sc6 on the time axis by the above process (a). The evaluation information represents a state associated with the corresponding scene. In [1], one of the two pieces of evaluation information given on the moderator side is evaluation information E1 [sc2] associated with the scene sc2, and the other is evaluation information E1 [sc4] associated with the scene sc4. Yes. Also, one piece of evaluation information given by the analyst is evaluation information E2 [sc2] associated with the scene sc2.

  As shown in the figure, the evaluation information E1 [sc2] evaluates the subject A “negative”, and the evaluation information E1 [sc4] evaluates the subject C “positive”. Evaluation information E2 [sc2] is used to evaluate subject A “positively”. Accordingly, as shown in the drawing, the scene sc2 is determined to be a recognition error scene because the evaluation of the scene sc2 is different between the moderator side and the analyst side for the same subject A. Further, since the scene sc4 is given evaluation information only from the moderator side with respect to the subject C, it is determined as a missed scene. The other scenes sc1, sc3, sc5, and sc6 are neither recognition-recognized scenes nor overlooked scenes, but are determined not to correspond to the confirmation target scene.

  In FIG. 5, the image SC ′ newly generated by cutting out the scenes sc2 and sc4 to be confirmed in the above [1] is shown in [2]. As described above, in one embodiment, the video SC ′ may be a confirmation target video. In one embodiment, the information of [1] in a state where information indicating that the scenes sc2 and sc4 are confirmation target scenes is added. The video SC may be a confirmation target video.

  FIG. 4 is a flowchart of the confirmation target video generation processing by the confirmation target video generation unit 42 in the above step S3. In step S11, the confirmation target video generation unit 42 receives the evaluation imparted video to be processed as an input, and proceeds to step S12. In step S12, the confirmation target video generation unit 42 divides the evaluation imparted video for each scene to form a scene set SC, and the process proceeds to step S13. Step S13 is a dummy step for representing a loop of the scene determination process between Step S13 and Step S18, and the process proceeds to Step S14 which is the first step in the loop process.

  In step S14, one element sc (that is, scene sc) is extracted from the scene set SC as a processing target, and the process proceeds to step S15. In step S15, the moderator side and analyzer side evaluation information E1 and E2 assigned in the scene sc are acquired from the scene sc, and the process proceeds to step S16. In some cases, the evaluation information E1 and E2 is not assigned to the scene sc. In this case, information indicating that the evaluation information E1 and E2 is not assigned is acquired.

  In step S16, the evaluation information E1, E2 is compared to determine whether the scene sc is a confirmation target. As described above, the determination can be performed in accordance with how the evaluation information is configured, but in one embodiment, the scene to be confirmed is determined by substeps S21 to S24 drawn in the frame F16. It is possible to determine whether the type is “missing scene” or “recognition error scene”. Sub-steps S21 to S24 are as follows.

  In step S21, the evaluation information E1 and E2 of the scene sc is examined to determine whether or not there is a subject who only has the evaluation information E1 on the moderator side. If there is, the process proceeds to step S22 and exists. If not, skip step S22 and proceed to step S23. In step S22, the scene sc is classified as a missed scene as a type of confirmation target scene, and the process proceeds to step S23.

  In step S23, the evaluation information E1 and E2 of the scene sc is checked, and "there is a subject who only has evaluation information E2 on the analyst side" or "evaluation information E1 and E2 are given by both the moderator and the analyst. However, if at least one of “there are subjects with different contents” is satisfied, the process proceeds to step S24, and if not, the process skips step S24 to step S17. Proceed with In step S24, the scene sc is classified as a recognition error scene as a kind of confirmation target scene, and the process proceeds to step S17.

  The sub-steps S21 to S24 as one embodiment of step S16 have been described above. When a negative determination is made in both steps S21 and S23 (when both steps S22 and S24 are skipped), it is determined that the scene sc is not a confirmation target scene.

  In step S17, when it is determined in step S16 immediately before that scene sc is a confirmation target, the scene sc is added to the set SC ′ constituting the confirmation target video, and the process proceeds to step S18. For the set SC ′, an empty set φ may be set as an initial value when the flow of FIG. 4 is started.

  In step S18, it is determined whether or not the loop processing in steps S13 to S18 has been completed. Here, when the set SC is the empty set φ, it is determined that the loop processing has ended, and the process proceeds to step S19. On the other hand, if the set SC is not the empty set φ, there remains an element sc (scene sc) that has not yet been processed in the set SC, and the process returns to step S14 to continue the loop processing for these elements SC.

  In step S19, the confirmation target video generation unit 42 combines the scenes included in the set SC ′ to generate a confirmation target video, stores the video in the verification target video storage unit 43, and the flow in FIG. 4 ends. . When the confirmation target video is generated as the set SC ′, the confirmation target video is generated in the format [2] shown in the example of FIG. As another embodiment, the confirmation target video is displayed in the format of [1] in FIG. 5, that is, in a format in which information indicating whether or not the scene is the confirmation target is added to each scene in the video SC as the initial scene set. It may be generated.

  Returning to FIG. 3, when the above step S3 is completed, in the next step S4, the moderator and / or the analyst performs a confirmation operation (review of the group interview) with reference to the generated confirmation target video. In order to assist the work, the confirmation device 10 displays an image of a location to be confirmed to the moderator and / or the analyst in response to a request from the moderator and / or the analyst. Specifically, as described below, the video search unit 51 receives a request from a moderator and / or an analyst, searches for a video to be displayed, and the video display unit 52 displays the searched video as a moderator and / or Or display it to the analyst.

  The video search unit 51 receives a search request for a scene to be confirmed from the field side terminal 11 on the moderator side and the remote side terminal 12 on the analyst side, and searches the confirmation target video storage unit 43 for a scene that matches the request. In addition, the scenes of all video cameras corresponding to the scene are acquired from the entire video storage unit 22 and passed to the video display unit 52.

  In the video display unit 52, the scene in the searched confirmation target video is displayed on the moderator-side terminal 11 and / or the analyzer-side remote terminal 12 that transmitted the search request. When the display is performed, the evaluation information that is given to the scene and is the basis for determining the scene as the confirmation target scene in the confirmation target video generation unit 42 may be displayed together. The evaluation information may be displayed by being superimposed in a known telop format in the scene screen, or may be displayed separately from the scene screen.

  Further, since the video display unit 52 acquires scenes of all the video cameras corresponding to the scene, when displaying the scene in the confirmation target video, the moderator and / or the analyst can use any video on each terminal. By selecting a camera, the scene of the designated video camera can be viewed.

  In other words, when observing the group interview at the remote location L2, the analyst views the video of any one video camera switched by the switcher, but when performing the confirmation work in step S4 In addition to the video camera that the analyst actually viewed the target scene, any video camera can be designated and viewed, and the analyst and / or moderator can confirm the target scene. An oversight or a recognition error in a scene may be caused by the fact that an appropriate video camera has not been selected, so that the confirmation operation can be efficiently performed by the selection.

  The search request received from the site side terminal 11 and the remote side terminal 12 in the video search unit 51 can be various embodiments as follows.

  In one embodiment, the configuration information of the confirmation target video stored in the confirmation target video storage unit 43 in advance is transmitted to the terminals 11 and 12 side, and a scene to be viewed is designated from the configuration information. The search request may be configured with.

  That is, the terminals 11 and 12 have, on the side of the configuration information of the confirmation target video as described in FIG. 5 (information for distinguishing a series of scenes divided in time series and information on whether each scene is a confirmation target or not). And “information for distinguishing“ recognition error scene ”or“ missing scene ”) may be transmitted so that a desired location is selected from the terminals 11 and 12 side.

  In addition to directly selecting individual scenes as described above, in one embodiment, all scenes corresponding to “recognized scenes” are requested, or all scenes corresponding to “missing scenes” are requested, etc. Search requests can be configured by specifying conditions. Also in this case, the configuration information of the confirmation target video may be transmitted to the terminals 11 and 12 in advance, or a predetermined menu item for specifying conditions may be transmitted to the terminals 11 and 12. .

  In the above, each step of FIG. 3 and each part of FIG. 2 were demonstrated. FIG. 6 is a functional block diagram of the confirmation apparatus 10 according to an embodiment different from FIG. Differences from the configuration of FIG. 2 are as follows. That is, in the configuration of FIG. 2, the analyst operates the switcher at the remote location L2, and selects the video of one desired camera from all the camera videos acquired by the video acquisition unit 21, and the video Input was made to the distribution unit 23. On the other hand, in the configuration of FIG. 6, the switcher is omitted, and therefore the analyst does not need to operate the switcher, and instead of the switcher, the process of selecting one camera video to be input to the video distribution unit 23 is the target of gaze. The acquisition unit 25 and the camera selection unit 26 are responsible. Other configurations and operations are exactly the same as those described with reference to FIGS.

  The gaze target acquisition unit 25 and the camera selection unit 26 may be configured in the confirmation device 10 in the embodiment of FIG. 6, or one or both of them may be an external configuration of the confirmation device 10. The functions of the units 25 and 26 are as follows.

  The gaze target acquisition unit 25 acquires the gaze target (that is, the subject) of the moderator writer by using known face recognition and eye tracking technology, and passes the acquisition result to the camera selection unit 26. As a specific device for realizing eye tracking, a glass-type wearable device or the like can be used, and the gaze target acquisition unit 25 is realized by the writer wearing the device. Alternatively, the clerk only wears a device that records the first person video, and a separate device performs face recognition and eye tracking technology on the first person video, thereby realizing the gaze target acquisition unit 25. May be.

  The camera selection unit 26 selects a camera that captures an image in which the target (that is, the subject) received from the gaze target acquisition unit 25 is included in the largest proportion among all the camera images received from the video acquisition unit 21, The camera image is transferred to the video distribution unit 23. Thus, the function similar to the switcher of FIG. 2 is realized by the camera selection unit 26 selecting one of the camera images at each time point of the live-distributed interview video.

  FIG. 7 is a flowchart of the operation of the camera selection unit 26 according to an embodiment. In step S31, the camera selection unit 26 receives the gaze target o acquired by the gaze target acquisition unit 25 as an input, and proceeds to step S32. In step S32, it is determined whether or not the gaze target o is one of the subjects, and if it is any subject, the process proceeds to step S33, and if it is not any subject, the flow ends.

  When it is determined in step S32 that the subject is not any subject, the camera selection unit 26 determines to continue using the image of the camera selected in the frame at time t-1 immediately before the time t. Then, the camera image may be passed to the video distribution unit 23.

  In step S33, all camera image sets C = {c1, c2,..., Cn} (n is the number of cameras) at the time t are received as input, and the process proceeds to step S34. In step S34, the target maximum occupation ratio pmax and the selected camera number k are both set to initial values 0 as variables used in the subsequent steps, and then the process proceeds to step S35.

  Steps S35 to S39 constitute a camera selection loop process. Step S35 is a dummy step indicating the start of the loop, and the process proceeds to step S36. Note that i is used as a loop control variable. i stores the number of rounds in the loop. Therefore, i = 1 is set as an initial value.

  In step S36, the ratio p in which the object o is included in the camera image ci is calculated, and the process proceeds to step S37. For example, the ratio p can be calculated as an area ratio occupied by the face of the object o, that is, the subject o in the image. In addition, it may be calculated including a part other than the face. A known person recognition technique can be used for the calculation.

  In step S37, it is determined whether or not the ratio p is larger than the target maximum occupation ratio pmax. If larger, the process proceeds to step S38, and if not larger, the process skips step S38 and proceeds to step S39. In step S38, the value of the ratio p is substituted into the target maximum occupation ratio pmax and updated, and the value of the variable i is substituted and updated in the selected camera number k. Then, the process proceeds to step S39.

  In step S39, it is determined whether or not the camera selection processing loop has been completed based on whether or not the number of loops i has reached the number n of cameras. If i = n, the process proceeds to step S40. In the case (if i <n), i is incremented by 1, and then the process returns to step S36 to continue the loop processing.

  In step S40, as a final result of the flow of FIG. 7, the camera selection unit 26 passes the camera image ck whose camera number is the variable k at the time point to the video distribution unit 23, and the flow ends.

  The operation of the confirmation device 10 in each embodiment has been described above. Below, some typical examples of images and the like during the operation will be introduced.

  FIG. 8 is a diagram showing an example of the first surface of the group interview. As shown in the figure, there are a moderator, a scribe, and subjects A, B, and C at the interview site L1, and the state of the interview is shot with video cameras A, B, and C. Images IA, IB, and IC are images taken by video cameras A, B, and C, respectively. The interview will be described as the purpose of having the subject operate the smartphone terminal at the planning stage and collecting opinions about the terminal from the subject. Note that the examples shown in FIGS. 9 to 11 to be described later also assume the same interview environment as in FIG.

  First, a schematic example of the process of distributing the interview video to the analyst (step S1 in FIG. 3) will be introduced on the premise of the embodiment in FIG.

  During the interview, if the moderator-side writer is looking at the subject A, the gaze target acquisition unit 25 acquires the subject A as a gaze target and sends the information to the camera selection unit 26. The video acquisition unit 21 acquires the current camera images IA, IB, and IC of the video cameras A, B, and C, and sends them to the camera selection unit 26. Examples of camera images IA, IB, and IC of video cameras A, B, and C are as shown in FIG.

  The camera selection unit 26 selects a video camera that has captured an image with the largest percentage of subjects A included. Here, the video IC is selected because the image IC shown in FIG. Finally, the video distribution unit 23 distributes the image of the video camera A to the analyst side, and stores the distribution image in the viewing video storage unit 24.

  The processing as described above is performed at each time point when the group interview progresses. In the above example, the case where an image is selected by the gaze target acquisition unit 25 and the camera selection unit 26 in the embodiment of FIG. 6 is taken as an example, but also in the embodiment of FIG. The same processing is performed by selecting.

  Next, an example of discriminating a recognition error scene from the evaluation imparted video will be described.

  Here, as described above, it is assumed that the evaluation information is selected from the two items “positive” or “negative” as the emotion evaluation items, and the text describes why such selection is made. An explanation will be given by taking the case where it is configured as an example.

  The moderator side and the analyst side clerk asked the subject A in the scene from the terminal devices 11 and 12, respectively. “I swipe the smartphone screen frequently, but it seems to be in trouble because I cannot find any icons. "(Negative emotions)", "I am crazy about the operation of the smartphone screen, I am intrigued." (Positive emotions) Post the evaluation information as a comment with emotion information.

  Each of the site-side input unit 31 and the remote-side input unit 32 includes the evaluation information (time stamp, subject A to be evaluated, emotion information selected from two items of “positive” or “negative”, comments, that is, emotion information). A text explanation for the selection, input person information) is received and stored in the evaluation storage unit 33. In addition, during the interview, interview videos taken by all video cameras are stored in the entire video storage unit 22.

  The evaluation-added video generation unit 41 refers to the evaluation storage unit 33 and the viewing video storage unit 24, and generates an evaluation-added video based on the time stamp given to the evaluation information. The confirmation target video generation unit 42 compares the evaluation information on the moderator side and the analyst side with respect to the subject for each scene in the video to which the evaluation information is given, and is a missed scene or a recognition error scene as a confirmation target. Or whether it does not correspond to the scene to be confirmed.

  About the scene which illustrated the said comment content, since the emotion information provided to evaluation information is different, it is judged that there is a possibility that the analyst has mistaken recognition, and it will discriminate | determine from a recognition error scene. An example of an image IM [sc2] with a comment in the evaluation information of the scene sc2 is shown in FIG. Note that the evaluation information E1 [sc2] and E2 [sc2] in FIG. 9 correspond to the example in FIG.

  In the following, an example in which the scene is determined as a missed scene will be introduced.

  FIG. 10 shows only the comment “Appreciation of Emotion of the terminal and the terminal design very much” (positive emotion) is given as the evaluation information E1 [sc4] to the subject C on the moderator side. An image IM [sc4] in the rendered scene sc4 is shown. Since this scene sc4 is given a comment from the moderator only to the subject C, it is determined that the analyst may have missed it, and is determined to be a missed scene. Note that the evaluation information E1 [sc4] in FIG. 10 corresponds to the example in FIG.

  Finally, an example of processing for displaying the recognition error scene retrieved from the confirmation target video storage unit 43 on the terminals 11 and / or 12 will be described.

  A “recognition error scene” is designated as a search target from the terminals 11 and 12 to the video search unit 51. Then, the video search unit 51 acquires all recognition error scenes and scenes of all video cameras corresponding to those scenes from the confirmation target video storage unit 43 and the entire video storage unit 22, and stores them in the video display unit 52. send. The video display unit 52 displays the received scene on the terminals 11 and 12.

  At this time, a display example of the search scene on the terminals 11 and 12 is shown in FIG. The display example of FIG. 11 corresponds to the example of FIG.

  In FIG. 11, the video of camera B is currently displayed as video IM [sc2] on terminals 11 and 12, and camera A of a different viewpoint from camera B is further shown on the right side as video IM100. And the video IM [A] and IM [C] are also displayed. Currently, it is shown as highlighted EM that the video IM [B] of camera B is selected, but when video IM [A] or IM [C] of camera A or C is selected, video IM [sc2] can also be switched to the corresponding camera image.

  Hereinafter, supplementary matters in the present invention will be described.

  (1) The confirmation device 10 shown in FIG. 2 or FIG. 6 may be configured as a confirmation system by arbitrarily dividing each configured part and configuring each part as a server. For example, it may be configured as a distribution / recording server responsible for step S1 in FIG. 3, a confirmation target video generation server responsible for steps S2 and S3, and a search / display server responsible for step S4.

  Similarly, any configuration of the site L1 and the remote location L2 may be included in the confirmation device 10. Conversely, any configuration of the confirmation device 10 may be disposed at the site L1 and the remote location L2.

  (2) After the confirmation target video is generated, the terminals 11 and 12 that make a search request are not necessarily present at the site L1 and the remote location L2. It is possible to make a search request from any other point and display the corresponding video.

  (3) When the evaluation information is composed only of text and does not include a predetermined emotion evaluation item such as “positive” or “negative”, the confirmation target video generation unit 42 uses the known text analysis technique to Processing (b), that is, whether or not each scene is a confirmation target can be determined.

  For example, by applying a support vector machine that has been learned in advance, the texts are classified into categories, and if the categories are common, the evaluation contents are the same, and if the categories are different, the evaluation contents are different. According to the determination of the evaluation content, “missing scene” and “recognition error scene” may be identified in the same manner as in the case of determination with two items “positive” or “negative”.

  (4) The present invention can also be provided as a program that causes a computer to function as the confirmation device 10. The computer can be configured with known hardware such as a CPU (Central Processing Unit), a memory, and various I / Fs, and the CPU that reads and executes the program functions as each unit of the confirmation device 10.

  10 ... confirmation device, 21 ... video acquisition unit, 22 ... all video storage unit, 23 ... video distribution unit, 24 ... viewing video storage unit, 25 ... gazing target acquisition unit, 26 ... camera selection unit, 31 ... site side evaluation input 32: Remote side evaluation input unit, 33 ... Evaluation storage unit, 41 ... Evaluation imparted video storage unit, 42 ... Confirmation target video generation unit, 43 ... Confirmation target video storage unit, 51 ... Video search unit, 52 ... Video display Part

Claims (9)

This is a confirmation device that distributes video of group interviews conducted on subjects in the field to remote locations, and assists in confirming the status of subjects in the group interviews at the site and / or remote locations after the group interview is completed. And
A video acquisition unit for acquiring video of the group interview;
Based on the acquired video, a video distribution unit that distributes the video to a remote location;
A viewing video storage unit for storing the distributed video as a viewing video at a remote location;
After being given a time stamp according to the progress of the group interview, from each of the site and the remote place, an evaluation storage unit that receives and stores evaluation information that specifies the subject and evaluates the state thereof,
For the viewing video, an evaluation-added video generation unit that generates an evaluation-added video by associating the evaluation information at the time stamp position;
Confirmation video generation that divides the evaluation-giving video into a plurality of scenes and identifies a scene determined to have different evaluation information from the field for the subject and evaluation information from a remote location as a scene to be confirmed And a confirmation device.   The confirmation apparatus according to claim 1, wherein the evaluation information includes information that is evaluated by selecting the state of the designated subject from predetermined emotion evaluation items.   The confirmation apparatus according to claim 1, wherein the evaluation information includes information that describes the state of the designated subject as text. The confirmation target video generation unit
For a common subject, both the evaluation information from the site and the evaluation information from a remote location have been received, but the scene where it is determined that the evaluation contents in the two are different,
Alternatively, a scene in which there is a subject who receives only evaluation information from a remote location and has not received evaluation information from the site is identified as a recognition error scene in the remote location as a kind of the scene to be confirmed. The confirmation apparatus according to any one of claims 1 to 3, wherein The confirmation target video generation unit
A scene in which there is a subject who receives only evaluation information from the site and has not received evaluation information from a remote location is identified as a scene to be overlooked as a type of the confirmation target scene. The confirmation device according to any one of claims 1 to 4. The confirmation target video generation unit generates a confirmation target video from a scene identified as the confirmation target,
A video search unit that receives a search request from the site and / or a remote location, and searches for a scene that matches the search request from the confirmation target video;
The confirmation apparatus according to any one of claims 1 to 5, further comprising a video display unit that displays the searched scene on a site and / or a remote place where the search request is transmitted. The video acquisition unit acquires the video of the group interview from each of a plurality of cameras,
The video distribution unit distributes the video of one camera selected at each time point in time series from a plurality of cameras in the acquired video to a remote place,
The confirmation according to claim 6, wherein when displaying the searched scene, the video display unit also displays videos of a plurality of cameras other than the video of the selected one camera. apparatus. A gaze target acquisition unit that acquires which subject is the target of the group interviewer at the site;
A camera selection unit that determines a camera that has captured an image including the acquired subject to be watched at the highest ratio as the one camera to be selected in the video distribution unit; The confirmation device according to claim 7.   A program causing a computer to function as the confirmation device according to any one of claims 1 to 8.