JP2004187043A - Video processor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To realize video control processing based on a natural movement of an object in video by eliminating the need for applying special equipment to the object. <P>SOLUTION: A physical characteristic amount extracting part 2 extracts a characteristic action aspect C of the object M from the video inputted by a video inputting part 1 such as a video camera. When a matching part 4 determines matching between a gesture model J set in a model describing part 3 and the extracted action aspect C, a cut point extracting part 5 makes a video editing part 6 perform the editing that eliminates a corresponding part in the video by using this extraction point as a start point or end point and makes a video storing part 7 store the edited video. <P>COPYRIGHT: (C)2004,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a technique for adding video control processing such as input control and editing to video captured by a video camera or the like, and in particular, based on a characteristic operation directly specifying the control processing from the behavior mode of a subject in the video. Regarding the technology to be performed.
[0002]
[Prior art]
For example, in corporate activities, meetings and lectures are shot and recorded with a video camera, and the recorded images are used later. There is a scene where it is not possible to leave a record partially. In such a scene, the speaker serving as the subject of the video sometimes intentionally wants to stop recording the video in order to prevent the scene including the confidential information from being recorded.
Conventionally, the above situation has been dealt with by instructing a cameraman operating a video camera to stop shooting of a video, or after shooting, an editor performs an editing operation to delete a corresponding portion in the video.
[0003]
Here, as a technique for coping with the above-mentioned circumstances, as described below, in light of the idea of the present invention, a behavior mode (for example, a gesture) based on a body motion of a speaker serving as a subject or a behavior mode (for example, a sound output operation). , A predetermined audio output such as "cut"), the video control processing such as input control and editing processing can be performed on the video to be shot. It is considered to be extremely effective in practice without interrupting the continuity of conversation and the like.
However, heretofore, such a technology has not been realized, and no idea has been made.
[0004]
2. Description of the Related Art Conventionally, the following technologies have been known as device control using gestures.
2. Description of the Related Art There is known an electronic camera system in which a photographer wears a device for detecting a myoelectric signal indicating a body movement on an arm and a hand, and controls a camera by detecting a motion of the photographer as a myoelectric signal. In this electronic camera system, the camera control command is associated with a predetermined gesture consisting of a combination of the movement of the hand or finger and the movement of the upper arm from the wrist, and the photographer simply moves the arm and hand to perform the movement. The signal is detected by a signal detection unit of the device that detects the myoelectric signal, and the camera is controlled by a command defined in the gesture (see Patent Document 1).
[0005]
Also, various techniques using unique characteristic parameters extracted from human gestures and hand gestures are known.
In order to make the motion of the video character natural, the gesture of the operator is acquired from the video, the motion is parameterized based on the reproducible reference, the parameters are tagged and stored, and the performance video gesture used for the character motion is acquired and acquired. A method and apparatus for reproducing a moving image character is known (see Patent Document 2).
[0006]
To efficiently describe motion information for objects in the video, generate a cumulative motion histogram that accumulates motion histograms, generate a motion descriptor that efficiently describes motion information for the subject in the video, and use it for video search. There is known a motion descriptor generation device using a cumulative motion histogram to be used and a method thereof (see Patent Document 3).
[0007]
There is also known a technique for constructing a model for performing gesture recognition and accurately estimating the motion and structure of a subject from continuous frame images in a video.
Each of a plurality of frame images constituting a moving image is regarded as one point on a vector space, and the motion trajectory of the point is set as a feature parameter for each type of gesture, and the extracted feature parameter and the feature parameter of the reference pattern are Are compared, a gesture moving image recognition method for performing gesture recognition is known (see Patent Document 4).
[0008]
Concerning the gesture performed by the subject in the video, construct a semantic label attached to the semantic unit of the gesture, such as asking (leaning) or agreeing (nodding), extracting the meaning of the gesture from the semantic label, There is known a gesture video reconstruction method and apparatus for generating a script describing a start time and an end time of a gesture and a recording medium on which the method is recorded (see Patent Document 5).
[0009]
Moving image processing for inputting a plurality of image frames constituting a moving image, estimating affine deformation between image frames from changes in positions of at least three feature points between the image frames, and detecting movement and structure of a target object An apparatus is known (see Patent Document 6).
[0010]
[Patent Document 1]
JP 2000-138858 A
[Patent Document 2]
JP 2001-229398 A
[Patent Document 3]
JP 2000-222586 A
[Patent Document 4]
JP-A-9-245178
[Patent Document 5]
JP-A-11-238142
[Patent Document 6]
JP-A-6-89342
[0011]
[Problems to be solved by the invention]
However, conventionally, there have been the following various problems.
A common problem with the prior art described above is that the user can take a picture without attaching a special device or device for recognizing a gesture or without using a remote control for operating the photographing apparatus. There was a problem that it was not possible to perform video control processing operations.
[0012]
More specifically, the technique described in Patent Document 1 requires a user to wear a recognition device for operating a camera on the body, and the technique described in Patent Document 2 identifies a gesture. Therefore, the sensor must be worn by the user on the body, the user must be prevented from acting in a natural state, or special equipment must be provided.
Further, the technologies described in Patent Documents 3 to 6 only disclose a technology for recognizing a gesture of a subject, and are only elemental technologies applicable to video input control and editing control.
[0013]
SUMMARY OF THE INVENTION The present invention has been made in view of the above-described conventional circumstances, and has as its object to realize a video control process based on a natural motion of a subject in a video without having to equip a subject with special equipment.
Further objects of the present invention will be apparent in the following description.
[0014]
[Means for Solving the Problems]
In the video processing device according to the present invention, a video control processing unit specifies a characteristic behavior mode of a subject by a physical motion or a sound output operation from a video input by a video input device such as a video camera, and a predetermined characteristic And outputting a control signal for applying a corresponding control process to the video based on the target behavior.
Then, the video processing device according to the present invention, in response to the control signal, the video editing unit inputs a corresponding editing process such as deleting a certain scene from the video input unit and stores the video in a memory or the like. Or a video control process in which the video input control means adds corresponding control such as fading out the video camera to the video input from the video input means.
[0015]
Therefore, it is possible to perform video control processing such as video editing control and video input control based on the behavior of the subject in the video due to the natural action without having to provide the subject with special control equipment. That is, such video control processing can be performed based on features directly extracted from the subject image itself in the video and features directly extracted from the subject sound itself in the video.
[0016]
More specifically, the video processing device according to the present invention includes, as a video control processing unit, a feature amount extraction unit that specifies a characteristic behavior mode of a subject by a body motion or a sound output operation from an input video; Model storage means holding an action mode model based on the operation or the sound output operation, and matching means for determining consistency between the action mode specified by the feature amount extraction means and the action mode model held in the model storage means. And outputting the control signal based on the determination result by the matching means.
As described above, the present invention can be grasped as a video processing device of various aspects. For example, a video control method implemented by operating the video processing device and a program for realizing the video processing device by a computer are also grasped. Is done.
[0017]
BEST MODE FOR CARRYING OUT THE INVENTION
The present invention will be specifically described based on examples.
FIG. 1 shows the configuration of a video processing apparatus according to a first embodiment of the present invention.
Here, in the present example, an editing process such as scene deletion is added to a video input based on a gesture of a subject in a video (an action mode by a body motion). Control processing such as switching of an input video is performed based on a gesture (behavior mode by a body motion), or the above-described editing process based on audio output content of a subject in a video (behavior mode by a sound output operation) or Various embodiments such as performing input video control or combining them can be performed.
[0018]
As shown in FIG. 1, the video processing apparatus according to the present embodiment includes a video input unit 1, a body characteristic amount extraction unit 2, a gesture model description unit 3, a matching unit 4, a cut point extraction unit 5, a video editing unit 6, A storage unit 7 is provided.
The same applies to the above-described various embodiments. For example, a video control processing unit (in this example, the body characteristic amount extraction unit 2, the matching unit 4, and the cut point extraction unit 5) which is a main function of the present invention. May be configured by causing a computer to execute a program according to the present invention, whereby a characteristic behavior mode of a subject is specified from an input video, and a video is generated based on a predetermined characteristic behavior mode. Alternatively, a control signal for performing a corresponding control process may be output.
[0019]
The video input unit 1 is configured by a video camera, a computer equipped with a video capture device, or an interface for inputting a video signal captured by another device, etc., and is used to input a video (moving image) of a conference including a backer who is a subject. Capture the image or take it into the device.
The captured video is moving image data composed of a number of continuous image frames 9 as shown in FIG. 2. At the time of shooting, each image frame has a frame number (001, 002,...) And video time information as identifiers. (T1, t2...) Are added.
[0020]
A video is input from the video input unit 1 to the body feature extraction unit 2 in image frame units. As shown in FIG. (A), a body feature (line segment model C representing a characteristic behavior mode by body motion) is identified ((b) in the figure), and the body feature C is extracted ((c) in the figure). ), And outputs the extracted body feature value C to the matching unit 4.
[0021]
As will be described later, as shown in FIG. 4, the body characteristic amount C is processed as a line segment model including line segments L1 to L7 that characterize the gesture of the subject M. More specifically, in this example, as shown in FIG. 3A, the subject M spreads both arms and opens two fingers of both hands in a V-shape (a gesture imitating a crab). Therefore, the line segment model C is used as the control timing of the editing process, so that the line segment model C has the part line segment data corresponding to the two fingers of the subject M as shown in FIG. (L1 and L2 and L6 and L7), part line segment data (L3 and L5) corresponding to two arms, and part line segment data (L4) corresponding to the head and body.
[0022]
The body characteristic amount extraction unit 2 outputs the extracted body characteristic amount C, the frame number 11 of the extracted image frame, and the body characteristic amount ID 11 for identifying the extracted body characteristic amount C to the matching unit 4 as shown in FIG. I do.
The gesture model description unit 3 has a memory for storing predetermined gesture model data and a function of describing a gesture model J as shown in FIG. 6 based on the data. In response to the output of the body feature C, the gesture model J is described for matching and evaluation with the body feature C, and the gesture model J is output to the matching unit 4.
[0023]
The matching unit 4 compares the body feature C input from the body feature extraction unit 2 with the gesture model J input from the gesture model description unit 3, and determines whether the body feature C and the gesture model J match. Then, as shown in FIG. 7, the determination result (matching or mismatching) 12 for each image frame is output to the cut point extracting unit 5 in association with the frame number 10.
[0024]
The cut point extraction unit 5 determines a range in the video to be deleted and edited (cut), and outputs a control signal corresponding to the determination result to the video editing unit 6. Specifically, based on the determination result 12 input from the matching unit 4, the cut point extraction unit 5 determines whether the body feature C and the gesture model J match, and The frame number 10 corresponding to the middle cut start time and cut end time is obtained, and an instruction to delete a portion sandwiched between these frame numbers from the video data is output to the video editing unit 6.
[0025]
Note that this instruction may be performed using a control signal that directly specifies the image frame time in the video instead of the frame number. In addition, this instruction may be performed by specifying the cut start time and causing the video editing unit 6 to perform a predetermined operation from the start time. The cut processing may be ended after a certain time, or the cut end time may be designated and the video editing unit 6 may start the cut processing from a time that is a predetermined time earlier than the end time.
[0026]
The video editing unit 6 edits the video data input from the video input unit 1 based on the control information input from the cut point extraction unit 5, outputs the video edited data to the video storage unit 7, Is stored in the database of the video storage unit 7 composed of
Specifically, in response to the control information from the cut point extracting unit 5, the corresponding video portion of the video data is deleted and stored in the video storage unit 7.
[0027]
Next, the processing relating to the extraction of the body characteristic amount of the present example will be described in detail.
FIG. 8 shows a processing procedure of the body feature amount extraction of the present example performed by the body feature amount extraction unit 2. First, when an image frame is input from the image input unit 1, the image frame is binarized. As shown in FIG. 9, the binarized image data 16 is stored in the internal memory 15 provided in the video processing device as a work area of the body characteristic amount extraction unit 2 (step S1).
[0028]
Then, the body characteristic amount extraction unit 2 extracts so-called thin line image data 17 of the subject M from the binarized image data 16, and further performs line segment division from the thin line image data 17 to obtain two fingers, The part line segment data 18 corresponding to the two arms, the head, and the torso are extracted, the part line segment data 18 is stored in the internal memory 15, and the body characteristic amount extraction processing ends (step S2). This process is repeated each time an image frame is input from the image input unit 1, and the part line segment data 18 stored in the internal memory 15 is sequentially output to the matching unit 4 as the feature amount C as described above. You.
[0029]
That is, in the body characteristic amount extraction processing, an image of the subject M is extracted and binarized as shown in FIG. 3A, and, for example, as shown in FIG. The skeleton line is extracted by connecting the positions, and furthermore, as shown in FIG. 4C, more specifically, as shown in FIG. 4, the part line data corresponding to the two fingers, the two arms, the head, and the body is obtained. This is the process of decomposing.
[0030]
Next, the gesture model J will be described.
FIG. 6 shows the concept of the gesture model J. The gesture model J of the present example is configured with descriptions for specifying two fingers, two arms, a head, and a body. The right arm is R_ARM, the first finger of the right hand is R_FINGER1, and the second finger of the right hand is R_FINGER2. The left arm is L_ARM, the first finger of the left hand is L_FINGER1, and the second finger of the left hand is L_FINGER2. The head including the torso is referred to as HEAD.
[0031]
That is, assuming that the entire body is BODY, the gesture model J is a description based on the following combinations.
BODY (t): = (HEAD (t), R_ARM (t), L_ARM (t))
R_ARM (t): = (R_FINGER1 (t), R_FINGER2 (t))
L_ARM (t): = (L_FINGER1 (t), L_FINGER2 (t))
[0032]
Here, the gesture model J of the present example is described including a time parameter t that also represents a temporal change of the model. This is because, by giving a time change to the form of the gesture model J, if a certain image frame in the motion of the subject M changing with time matches the gesture model J, the editing process is performed. This allows the speaker as the subject M to easily give a desired operation instruction by providing a range of consistency detection.
[0033]
FIG. 10 shows the concept of the opening / closing operation of two fingers, and R_FINGER1 and R_FINGER2 (L_FINGER1 and L_FINGER2) are separated from each other with time passing around the connection part. More specifically, at a certain time t1 shown in FIG. 9A, two fingers R_FINGER1 (t1) and R_FINGER2 (t1) are at a certain angle (for example, about 30 degrees) about the junction with R_ARM. ), And at a certain time t2 thereafter, that two fingers of R_FINGER1 (t2) and R_FINGER2 (t2) are closed at the center of the junction of R_ARM. Represents.
Therefore, if the part line segment data corresponds to each element of the gesture model J such as R_FINGER1, it is determined that the matching is performed by the matching unit 4.
[0034]
Next, a description will be given of a process of determining matching between the part line segment data and the gesture model J by the matching unit 4.
The matching unit 4 determines that the matching has been performed if the respective segment line data (L1 to L7) as shown in FIG. 4 correspond to the elements of the gesture model J such as R_FINGER1 as shown in FIG. judge.
[0035]
Specifically, as shown in FIG. 4, L1, L2, and L3 are connected at one point, L3, L4, and L5 are connected at one point, and L5, L6, and L7 are connected at one point. I have.
L1, L2 and L3 correspond to R_FINGER2, R_FINGER1 and R_ARM, L3, L4 and L5 correspond to R_ARM, HEAD and L_ARM, and L5, L6 and L7 correspond to L_ARM, L_FINGER1 and L_FINGER2.
When the connection relations and positions of these part line segment data L1 to L7 and the elements of the gesture model J are matched, it is determined that the body feature C and the gesture model J match.
[0036]
The determination of the match between the body feature C and the gesture model J is sequentially performed on the continuous image frames 9, and the matched image frames 9 are set as candidates for cut points (deletion edit points).
Here, in this example in which the gesture model J is given a time width, in particular, it is assumed that a plurality of matched image frames 9 are consecutively set as candidates. It is sufficient to specify an image frame image that defines the cut point, such as the image frame image, the last image, or the middle image, and perform the video editing process using the image frame number.
[0037]
Note that, in the present invention, based on the image frame position matching the gesture model J in the video, the video portion at a predetermined time from the position is deleted, or the video from the position retroactive for a predetermined time from the position to the relevant position is deleted. A portion is deleted, or a video portion in which matching image frames are consecutively deleted, or, as shown in FIG. 11, a deletion process is started from a position of a certain matching image frame A in a video, and The editing process range may be set in various modes, such as terminating the deletion process when the matching image frame B appears.
[0038]
In the above example, the editing process is performed based on a characteristic behavior mode of a body motion called a gesture of the subject M. However, the present invention provides a feature based on an acoustic output operation such as a sound emitted by the subject in a video. The editing process may be performed based on a typical behavior mode.
For example, as shown in FIG. 12, when the video includes video data (video track) 20 and audio data (audio track) emitted by the subject, the start and end points of each of the audio tracks 21a and 21b, or Alternatively, the switching points of the audio tracks 21a and 21b may be used as the start and end positions of the editing process.
[0039]
Specifically, the sound track can be made different for each utterance, and the above-mentioned starting point and the like can be used as a candidate for the control timing of the editing process together with the gesture model matching.
For example, when the start time of the audio track is set as the cut point, the video may be deleted and edited with the cut point specified by the matching between the body feature C and the gesture model J as the cut end point. When the end time of is used as the cut point, the video may be deleted and edited with the cut point specified by the matching of the body feature C and the gesture model J as the cut start point.
[0040]
As a method of detecting the continuity of the audio track, a method based on a silent section as shown in FIG. 13 can be adopted.
When detecting the continuity of the audio data, the volume level is measured, and when the volume level falls below a predetermined threshold, the time is detected as the silent state time Δt. For example, in a certain video data, if a silence state time Δt is detected immediately after the audio track Audio a and then the audio track Audio b is detected, if the silence state time Δt is within a predetermined time (for example, 5 seconds), The audio tracks Audio a and Audio b may be handled as continuous audio tracks, and when they exceed a predetermined time, they may be handled as different audio tracks.
[0041]
Further, in the present invention, when sound data generated by a subject such as audio included in video data is more actively used for editing processing, for example, in the apparatus configuration shown in FIG. 2 is to extract the audio data included in the video together with the time information in the video, and the gesture model description unit 3 describes an audio waveform model of a predetermined phrase (for example, “cut”) as shown in FIG. The matching unit 4 determines the consistency between the audio data extracted from the video and the audio waveform model. Alternatively, the cut point extracting unit 5 may determine the content and range of the editing process, output a control signal to the video editing unit 6, and perform the corresponding editing process on the video input from the video input unit 1. There.
[0042]
Further, in the above description, the editing process is performed on the video that is input and stored based on the physical motion and the sound output operation of the subject in the video. Based on the operation and the sound output operation, for example, video input control for switching a video camera constituting the video input unit 1 to another video camera may be performed.
[0043]
In this case, for example, as a device configuration as shown in FIG. 15, the control point extracting unit 35 uses the image frame position specified by the matching unit 4 in the same manner as above as the start or end position of the control, Based on this, the video control unit 36 may output a control signal to the video input unit 1 for changing the mode of video input.
In FIG. 15, the same components as those shown in FIG. 1 are denoted by the same reference numerals. FIG. 15 shows an example of the matching process based on the gesture model. However, the matching process based on the sound output from the subject can be applied in the same manner as described above.
[0044]
By controlling the video input in this way, the speaker who is the subject can switch between multiple video cameras based on his / her gestures and remarks and produce videos at different angles without giving instructions to the cameraman. Shooting, fading out a video camera or performing mosaic processing on an input video so that confidential materials are not clearly seen in the video, and switching an input path of a video input unit 1 to input a substitute video , Etc. can be performed.
[0045]
Note that, in the above description, the sound output operation has been described using the sound emitted by the subject as an example. Various types of sounds, such as sounds generated by the user, can be used for editing and controlling video input.
In the above description, the editing process has been described by taking the deletion of the video portion as an example. However, the present invention may also reduce the resolution of the video portion, mosaic the video portion, Various types of editing processing, such as adding an annotation to a document, can be employed.
[0046]
【The invention's effect】
As described above, according to the present invention, a characteristic behavior mode of a subject is extracted from a video, and the control process of the video is performed using the extracted behavior as a start point and an end point. Control processing such as deletion can be realized by natural human actions that do not hinder continuity such as conversation by the subject person.
[Brief description of the drawings]
FIG. 1 is a diagram showing a configuration of a video processing device according to an embodiment of the present invention.
FIG. 2 is a diagram illustrating a configuration of video data.
FIG. 3 is a diagram illustrating gesture extraction according to an example of the present invention.
FIG. 4 is a diagram illustrating part line segment data (feature amount) according to an example of the present invention.
FIG. 5 is a diagram illustrating an example of data output by a body characteristic amount extraction unit according to an example of the present invention.
FIG. 6 is a diagram illustrating a gesture model according to an example of the present invention.
FIG. 7 is a diagram showing an example of data of cut point information according to an example of the present invention.
FIG. 8 is a diagram showing a processing procedure of body feature extraction according to an example of the present invention.
FIG. 9 is a diagram illustrating an example of data in an internal memory in a body characteristic amount extraction process according to an example of the present invention.
FIG. 10 is a diagram illustrating opening and closing of two fingers according to an example of the present invention.
FIG. 11 is a diagram illustrating a video data editing portion based on a cut point according to an example of the present invention.
FIG. 12 is a diagram illustrating cut points due to a combination of a gesture model and voice processing according to an example of the present invention.
FIG. 13 is a diagram illustrating continuity detection of audio information according to an example of the present invention.
FIG. 14 is a diagram illustrating a voice information model according to an example of the present invention.
FIG. 15 is a diagram showing a configuration of a video processing device according to another embodiment of the present invention.
[Explanation of symbols]
1: video input unit 2: body feature extraction unit
3: Gesture model description part 4: Matching part
5: cut point extraction unit, 6: video editing unit,
7: video storage unit, 9: image frame,
35: control point extraction unit, 36: video control unit,
C: feature quantity, J: gesture model,
L1 to L7: part line segment data, M: subject,

Claims (12)

An image control device that performs a predetermined control process on an image,
Video input means for inputting video,
Image control processing means for specifying a characteristic behavior mode of the subject from the input video, and outputting a control signal for adding a corresponding control process to the video based on the predetermined characteristic behavior mode,
A video processing device comprising: The video processing device according to claim 1,
The image control processing means includes: a characteristic amount extracting means for specifying a characteristic behavior mode of the subject from the input video by a physical motion; a model storage means holding a behavior mode model by the physical motion; Matching means for determining consistency between the behavior mode specified by the means and the behavior mode model held in the model storage means, and outputting the control signal based on a determination result by the matching means. A video processing device characterized by the above-mentioned. The video processing device according to claim 2,
The video processing apparatus according to claim 1, wherein the behavior mode model stored in the model storage unit has a description format having a time width representing a time change of the body motion. The video processing device according to claim 1,
The image input from the image input means includes audio information,
The image control processing means includes: a feature amount extracting means for specifying a characteristic action mode by an audio output operation by the subject from the input video; a model storage means holding an action mode model by the audio output operation; Matching means for determining consistency between the behavior mode specified by the quantity extraction means and the behavior mode model held in the model storage means, and outputting the control signal based on the determination result by the matching means A video processing device characterized by performing: The video processing device according to any one of claims 1 to 4,
A video processing apparatus comprising: a video editing unit that adds an editing process corresponding to a control signal output from the video control processing unit to a video input and stored from the video input unit. The video processing device according to any one of claims 1 to 4,
A video processing apparatus comprising: a video input control unit that applies control corresponding to a control signal output from the video control processing unit to a video input from the video input unit. A video control method for applying a predetermined control process to a video,
A video processing method comprising: identifying a characteristic behavior mode of a subject from an input video; and performing a corresponding control process on the video based on the specified predetermined characteristic behavior mode. A video control method for applying a predetermined control process to a video,
From the input video, identify the characteristic behavior mode of the subject's body motion,
Determine the consistency between the specified behavior mode and the behavior mode model prepared in advance,
A video processing method, wherein a corresponding control process is added to a video based on the determination result. A video control method for applying a predetermined control process to a video, identifying a characteristic behavior mode by a sound output operation by a subject from a video including input audio information,
Determine the consistency between the specified behavior mode and the behavior mode model prepared in advance,
A video processing method, wherein a corresponding control process is added to a video based on the determination result. A video control method for applying a predetermined control process to a video,
Identify the characteristic behavior of the subject from the input video,
A video processing method, wherein an editing process corresponding to the specified predetermined characteristic behavior mode is accumulated in addition to the input image. A video control method for applying a predetermined control process to a video,
Identify the characteristic behavior of the subject from the input video,
A video processing method, wherein control corresponding to the specified predetermined characteristic behavior mode is added to the video input. A program for causing a computer to execute video control for applying a predetermined control process to an input video,
A function of identifying a characteristic behavior mode of the subject from the input video,
A function of determining consistency between the specified behavior mode and a behavior mode model prepared in advance,
A program for realizing, to a computer, a function of outputting a control signal for adding a corresponding control process to an image based on the determination result.

Images