JP2013115527A - Video conference system and video conference method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a mechanism for cutting the face of a person uttering a voice, out of a camera image and displaying the face superposed on a material image.SOLUTION: In a transmission terminal 1, first image input means 100 receives input of image information, and face recognition means 101 recognizes a face. Second image input means 103 receives input of background information, and voice input means 106 grasps which direction the voice has reached from. In a reception terminal 2, orientation information acquisition means 207 grasps which direction the voice has reached from, cut-out information acquisition means designates which position in the image information the face exists. Image cut out means 204 cuts out a face image existing in the reaching direction of the voice, image superposition means 205 superposes the face image on the background information, and display means 206 outputs the resulting image to a display device 5.

Description

  The present invention relates to a video conference system including a plurality of information terminals, and more particularly to an information terminal that transmits video, audio, and data to other information terminals, and receives and reproduces the information.

  In recent years, with the widespread use of the high-speed Internet, more and more users use a video conference function that performs two-way communication by connecting to a TV or a camera. In particular, in a video conference system, there is an increasing need to share materials between both parties and view the material image and the image of the other camera at the same time.

  In the realization of a system for simultaneously viewing the document video and the video of the other camera, for example, the above system is realized by dividing the screen into several parts and displaying the material video on one screen and the other camera's video on the other screen. However, in this case, there is a problem that it is difficult to read the characters because the display size of the material is small.

  As a method for solving this problem, by using a technique such as Patent Document 1, the area of the face of the other party photographed by the other camera is displayed in a small size with the picture-in-picture function, and instead the display size of the document is reduced. Can be bigger. However, this method has a problem that it is difficult to see the material when a plurality of people participate in the conference at the same time.

JP 2001-177812 JP

  In a video conferencing system, a mechanism is realized in which the face of a person who has made a sound is cut out from the camera video and superimposed on the document video.

  In order to solve the above-described object, for example, the configuration described in the claims is adopted.

  It is possible to realize a mechanism in which the face of a person who has made a sound is cut out from the camera video and superimposed on the document video. Problems, configurations, and effects other than those described above will be clarified by the following description of embodiments.

2 is an example (1) of a block diagram of a system in Embodiment 1. FIG. 3 is an example (2) of a block diagram of the system in Embodiment 1. FIG. It is an example of explanatory drawing of the stream 400 and the transmission information 410. FIG. It is an example of explanatory drawing of cutout information 424 and directivity information 425. 10 is an example (1) of an operation flow diagram of a system in Embodiment 2. FIG. FIG. 10B is an example (2) of an operation flowchart of the system in the second embodiment.

  The above problem is solved by the following means.

  The video conference system according to the present application includes, for example, a transmission terminal that inputs and transmits video information, audio information, and background information, and a reception terminal that receives and reproduces the information.

  The transmission terminal includes first video input means for inputting video information from the camera, second video input means for inputting data and acquiring it as background information, audio input means for inputting audio information, and first video input means. Face recognition means for recognizing a face from the input video, cutout information generation means for generating cutout information that is information for cutting out the face video, and generating direction information by detecting from which direction the voice comes Directing information generating means, MUX means for multiplexing the directivity information, clipping information, audio information, background information, and video information, and communication means for performing communication.

  The receiving terminal also acquires communication means for receiving information, DEMUX means for separating information, cutout information acquisition means for acquiring cutout information, directivity information acquisition means for acquiring directivity information, and background information. Second video acquisition means, video selection means for selecting a face based on the cut-out information, video cut-out means for cutting out the face video from the cut-out information and the video information, and a video that superimposes the face video and the background information It comprises superimposing means, display means for displaying such information, and sound output means for outputting sound.

  The receiving terminal may have each function of the transmitting terminal and have the function of the transmitting terminal. Similarly, the transmitting terminal may have the functions of the receiving terminal owned by each means of the receiving terminal.

  The above-described object is achieved by the following processing means that cuts out the face of a person who has produced a sound from the camera video and superimposes it on the material video for display.

  In the transmission terminal, video information is input from the camera by the first video input means, and background information is input by the second video input means. Voice information is input from a microphone by voice input means. The face recognition means recognizes the position of the face in the video information from the video information, and the cut-out information generation means generates cut-out information indicating the position of the face in the video information. The directivity information generation means analyzes the voice information input by the voice input means to grasp from which direction the voice has arrived, and generates directivity information representing the voice arrival direction. The MUX means multiplexes the video information, audio information, background information, cutout information, and directivity information to generate a stream, and the communication means transmits the stream to the receiving terminal via the communication line.

  In the receiving terminal, the stream is received by the communication means, and the stream is decomposed by the DEMUX means. Background information is acquired by the second video acquisition means, cut-out information is acquired by the cut-out information acquisition means, and directivity information is acquired by the directivity information acquisition means. The video selection means analyzes the directivity information to select a face video located in the direction of voice arrival, and the video clipping means cuts out the face video from the video information. The video superimposing means superimposes the background information and the face video into a video, displays the video by the display means, and outputs the sound by the audio output means.

  By such a function of the processing means, a mechanism is realized in which the face of the person who made the sound is cut out from the camera image and superimposed on the document image for display.

  The above object can also be realized by the following means.

  In the transmission terminal, video information is input from the camera by the first video input means, and background information is input by the second video input means. Voice information is input from a microphone by voice input means. The video information, audio information, and background information are multiplexed by the MUX means to generate a stream, and the stream is transmitted to the receiving terminal via the communication line by the communication means.

  In the receiving terminal, the stream is received by the communication means, and the stream is decomposed by the DEMUX means. The background information is acquired by the second video acquisition means. The face recognition unit recognizes the portion corresponding to the face from the video information, and the cutout information generation unit generates cutout information indicating the portion corresponding to the face. Voice information is acquired by the voice acquisition means, and the direction information generation means recognizes from which direction the voice has arrived and generates the direction information.

  The video selection means analyzes the directivity information to select a face video located in the direction of voice arrival, and the video clipping means cuts out the face video from the video information. The video superimposing means superimposes the background information and the face video into a video, displays the video by the display means, and outputs the sound by the audio output means.

  With the function of such means, a mechanism is realized in which the face of the person who made the sound is cut out from the camera image and superimposed on the document image for display.

  Embodiments of the present invention will be described below with reference to the drawings.

  Hereinafter, Example 1 will be described with reference to FIG.

  FIG. 1 is an example of a block diagram showing a configuration of a system according to an embodiment of the present invention. This system outputs a transmission terminal 1, a reception terminal 2, the Internet 3, an information device 4 that generates background information 423, a display device 5 that displays video information 424, a microphone device 8 that acquires audio information 422, and audio information 422. The speaker device 10 is configured. A person 6 and a person 9 are persons who operate the transmission terminal 1 and are captured by the camera of the transmission terminal 1, and a person 7 is a person who operates the reception terminal 2.

  This video conference system is composed of a pair of a transmission terminal 1 and a reception terminal 2. The transmission terminal 1 and the reception terminal 2 are terminal devices that communicate with each other via the Internet 3, for example, a fixed information terminal such as a set-top box or a personal computer, a mobile communication terminal such as a PDA or a smartphone, or It is a communication device dedicated to video conferencing. The transmission terminal 1 and the reception terminal 2 are equipped with a communication device, and can communicate with each other by a program that operates on a CPU. The information device 4 may be any information device as long as it generates a video image, such as a fixed computer such as a personal computer or a camera. The display device 5 is a display device such as a television or a video projection device such as a projector, and may be any information device that displays video information.

An example of the configuration of the transmission terminal 1 and the reception terminal 2 will be described with reference to the block diagram shown in FIG. The transmission terminal 1 and the reception terminal 2 may be implemented with hardware as each processing means corresponding to each block in FIG. 1 or as software.
When each processing means is implemented as hardware, the hardware to be implemented may be processed by a plurality of processing means by one hardware, or conversely, the processing performed by one processing means may be performed by a plurality of hardware. May go.
Further, when each processing means is implemented as software, the transmission terminal 1 and the reception means 2 are equipped with a CPU, a storage device, a memory, and the like, and the CPU expands and executes the program stored in the storage device in the memory. By doing so, the processing performed by each processing means can be performed. Here, with regard to the above software, one software may perform processing performed by a plurality of processing means, and conversely, a plurality of software may perform processing performed by one processing means.

  In the following description, for simplification of explanation, the CPU provided in each of the transmission terminal 1 and the reception terminal 2 executes the video conference program, and the processing corresponding to the block diagram shown in FIG. 1 by executing the video conference program The description will be made assuming that the function corresponding to the means can be said. In the following description, each function expressed by the CPU executing the video conference program is described as being executed by each processing unit shown in the block diagram of FIG.

  The transmission terminal 1 is connected to a device that outputs video such as a camera device, and inputs first video input means 100 that inputs video information 421; second video input means 103 that acquires background information 423 from the information device 4; and a microphone Recognition by the voice recognition means 101 for acquiring voice information 422 from the apparatus, face recognition means 101 for recognizing the face of the person 6 or 9 from the video information 421 input by the first video input means 100, and recognition by the face recognition means 101 Direction information for recognizing the voice arrival direction from the voice information 422 input by the voice input means 106 and converting it to the direction information 425 The generation unit 107, the video information 421, the background information 423, the audio information 422, the directivity information 425, and the cutout information 424 are multiplexed and the stream 400 is multiplexed. The MUX means 104 for converting, a communication unit 105 for transmitting the stream 400 to the receiving terminal 2 via the Internet 3.

  The receiving terminal 2 includes a communication unit 200 that receives the stream 400 via the Internet 3, a DEMUX unit 201 that decomposes the stream 400, a cut-out information acquisition unit 202 that acquires the cut-out information 424 from the DEMUX unit 201, and a DEMUX Second video acquisition means 203 for acquiring background information 423 from means 201, directivity information acquisition means 207 for acquiring directivity information 425 from DEMUX means 201, and video selection for selecting the face of person 6 or person 9 from directivity information 425 Image 208, image cutout means 204 for cutting out the face image selected by the image selection means 208 from the cutout information 424 and the image information 421, and an image to be output by superimposing the face image and the background information 423 Superimposing means 205, display means 206 for displaying output video on display device 5, sound Composed from the audio output means 213 for outputting a.

  Next, the stream 400 and transmission information 410 used in the transmission terminal 1 and the reception terminal 2 will be described with reference to FIG.

  The stream 400 is composed of one or a plurality of transmission information 410. The transmitting terminal 1 can transmit information such as video and audio to the receiving terminal 2 by transmitting the stream 400 listing the transmission information 410 in time series to the receiving terminal 2.

  The transmission information 410 is information in which one or more information among the video information 421, the audio information 422, the background information 423, the cut-out information 424, and the directivity information 425 is stored, and only one of the information is stored. Alternatively, a plurality of types may be multiplexed in the transmission information 410 at the same time.

  The video information 421 is data output from an information device that outputs video, such as a camera device, and is data that contains video of the face of the person 6 or person 9 photographed by the camera. The video information 421 includes time information 510, and is information that can be displayed on the display device 5 as moving image information by the receiving device 2 by aligning and outputting the time information 510.

  The audio information 422 is data output from the microphone device 8 and is information on the sound produced by the person 6 and the person 9 photographed by the camera. The audio information 422 also has directivity information indicating from which direction the audio has reached, such as stereo audio. Also, the audio information 422 includes time information 510 and is information that can be output by the speaker device 10 after being aligned with the time information 510.

  The background information 423 is a video output from an information device such as a personal computer, and may be a moving image or a still image. The background information 423 includes time information 510 and is information that can be output in alignment with the time information 510.

  Next, the cutout information 424 and the directivity information 425 included in the transmission information 410 will be described with reference to FIG. The cutout information 424 is information indicating at which coordinate of the video information 421 the user's face included in the video information 421 is located. The directivity information 425 is information indicating from which direction the sound detected by the microphone 8 is emitted.

  The cut-out information 424 includes time information 510, video identification information 511, origin information 513, size 514, and direction information 515.

  The time information 510 is time information for determining when to output information, and is, for example, absolute time such as GMT or JST, or elapsed time information in which one period at a certain frequency is one count. By using this time information 510, the video information 421, the audio information 422, the background information 423, the cutout information 424, and the directivity information 425 included in the transmission information 410 can be simultaneously output from the receiving terminal 2 at a fixed time. It is.

  The video identification information 511 is an identifier uniquely assigned to each video information 421 and is information for specifying the video information 421.

  The origin information 513 is information representing an origin for determining which coordinate position to cut out from the video information 421 identified by the video identification information 511, and is information including coordinate positions in the X-axis direction and the Y-axis direction. .

  The size 514 is range information indicating how much information is cut out from the origin information 513 in the X-axis direction and the Y-axis direction, and is represented by vertical and horizontal dot sizes, for example.

  The azimuth information 515 is information indicating from which direction the face image indicated by the cutout information 424 has arrived in the three-dimensional space. For example, the azimuth information 515 may be an angle obtained by rotating 360 degrees clockwise from north as 0. Orientation information such as east, west, south, and north may be used.

  The directivity information 425 includes time information 521 and direction information 523.

  The time information 521 is time information that determines when the information is output, and is, for example, absolute time such as GMT or JST, or elapsed time information in which one period at a certain frequency is one count.

  The azimuth information 523 is information corresponding to the azimuth information 515 of the cut-out information 424. For example, the azimuth information is an angle obtained by rotating 360 degrees clockwise with north as 0, or azimuth information such as east, west, south, and north. In short, any information may be used as long as the information indicates from which direction the voice comes.

  Next, an operation for realizing a mechanism in which the face of a person who has produced a sound is cut out from the camera image and superimposed on the document image will be described with reference to the flowchart of FIG.

  When the user instructs the start of the video conference between the transmission terminal 1 and the reception terminal 2, the process 1000 is executed at the transmission terminal 1 and the process 1100 is executed at the reception terminal 2. Hereinafter, it demonstrates according to a flowchart.

  In the transmission terminal 1 and the reception terminal 2, each CPU reads out the video conference program from the storage device, develops it on the memory, and executes the video conference program to develop the video conference function (1000, 1100).

The second video input means 103 of the transmission terminal 1 receives the background information 423 from an external or internal program and outputs it to the MUX means 104 (1001).
Also, the first video input unit 100 acquires the video information 421 from an imaging device such as an external or internal camera device and outputs the video information 421 to the face recognition unit 101 and the MUX unit 104 (1002).

  Next, the face recognition unit 101 performs image analysis on the video information 421, detects where the face is, acquires the coordinate value, and outputs it to the cut-out information generation unit 102 (1003). This face detection is done by analyzing the image in pixel units and detecting features such as eyes, nose, mouth, and contours, or by matching against a database of face images. Any method may be used. In short, it is only necessary to determine at which coordinate of the video information 421 the face is located.

  Next, the cutout information generation unit 102 receives the coordinate information acquired by the face recognition unit 101 in step 1003, generates cutout information 424 based on the coordinate information, and outputs it to the MUX unit 104 (1004).

  The cut-out information generation unit 102 stores the current time in the time information 510, the identification information of the video information 421 in the video identification information 511, and the information acquired from the coordinate information acquired in step 1003 in the origin information 513 and the size 514, respectively. To do. Further, the cut-out information generation unit 102 calculates the direction in which the face position in the video information 421 is located with the direction in which the camera device is facing as the origin, and the cut-out information 424 is obtained as the direction information 515. Save to.

Next, the voice input unit 106 acquires the voice information 422 from the microphone device 8 (1005).
At this time, the voice input means 106 detects from which direction the voice has arrived using a plurality of microphones, and generates directivity information 425 (1006). The voice input means 106 stores the current time in the time information 521 as explained in FIG. Further, the azimuth is calculated by using the direction in which the camera device is facing as the origin as the voice arrival direction, and stored in the direction information 523.
The cut-out information generation unit 102 and the voice input unit 106 perform the cut-out information 424 and the directivity information 425 generated, respectively.

  Next, the MUX unit 104 creates a stream 400 by multiplexing the video information 421, the audio information 422, the background information 423, the cutout information 424, and the directivity information 425, and receives it via the communication unit and the Internet 3. The stream 400 is transmitted to the apparatus 2 (1007).

The communication means 200 of the receiving terminal 2 receives the stream 400 transmitted by the transmitting terminal 1 in Step 1007 (1101), and outputs the received soot dream 400 to the DEMUX means 201.
The DEMUX means 201 decomposes the multiplexed stream 400 (1102).
Next, the video cutout unit 204 acquires the video information 421 from the DEMUX unit 201 (1103), the second video acquisition unit 203 acquires the background information 423 (1104), and the cutout information acquisition unit 202 outputs the cutout information 424. Is acquired (1105).

  Next, the voice information 422 is acquired from the DEMUX means 201 (1106), and the directivity information acquisition means 207 acquires the directivity information 425 (1107). The sound output means 213 outputs the sound information 422 as sound using the speaker device 10.

  Here, the video selection unit 208 acquires the cutout information 424 in which the heading information 523 of the directivity information 425 matches the heading information 515 of the cutout information 424, and obtains the video identification information 511, the origin information 513, and the size 514. The acquired face is selected (1108).

Next, the video cutout unit 204 cuts out the video information based on the origin information 513 and the size 514 with respect to the video information 421 associated with the video identification information 511 to obtain a face video (1109).
Next, the video superimposing means 205 overwrites the face video on the background information 423 and superimposes the video to create new video information (1110).
The video superimposing means 205 outputs the created video information to the display device 5 (1111).

  With the above operation, it is possible to realize a mechanism in which the face of a person who has made a sound is cut out from the camera video and superimposed on the document video.

  Further, in step 1109 of the receiving terminal 2, the video information 421 is cut out in order by using the origin information 513 and the size 514 of the plurality of pieces of cutout information 424 instead of using the direction information 523 of the directivity information 425. By doing so, it is possible to realize a mechanism for displaying the faces of all the participants one by one on the background information 423 in order.

  Further, in step 1110 of the receiving terminal 2, all face images generated in step 1109 are overlaid on the background information 423 so that the faces of all participants are displayed on the background information 423 at the same time. Can also be realized.

Hereinafter, Example 2 will be described with reference to FIG.

  FIG. 5 is an example of a block diagram illustrating the configuration of the system according to the second embodiment. Similar to the system described in FIG. 1, the system acquires the transmission terminal 1, the reception terminal 2, the Internet 3, the information device 4 that generates the background information 423, the display device 5 that displays the video information 424, and the audio information 422. The microphone device 8 includes a speaker device 10 that outputs audio information 422.

  An example of the configuration of the transmission terminal 1 and the reception terminal 2 will be described with reference to the block diagram shown in FIG.

  The transmission terminal 1 is connected to a device that outputs video such as a camera device, and inputs first video input means 100 that inputs video information 421; second video input means 103 that acquires background information 423 from the information device 4; and a microphone Audio input means 106 for obtaining audio information 422 from the device, MUX means 104 for multiplexing the video information 421, background information 423, audio information 422 and converting it into a stream 400, and a terminal for receiving the stream 400 via the Internet 3 2 and a communication means 105 for transmitting to the network.

  The receiving terminal 2 includes a communication unit 200 that receives the stream 400 via the Internet 3, a DEMUX unit 201 that decomposes the stream 400, a second video acquisition unit 203 that acquires background information 423 from the stream 400, and the stream 400. The video information 421 is acquired, the face recognition unit 209 that recognizes where the face is, the cut information generation unit 210 that generates the cut information 424 indicating where the face is, and the audio information 422 from the stream 400. The voice acquisition unit 211, the voice information 422 is analyzed, the direction information generation unit 212 that generates the direction information 425 indicating which direction the voice has reached, and the face of the person 6 or the person 9 is selected from the direction information 425. To the video selection means 208 from the video selection means 208 to be cut out, the cutout information 424 and the video information. A video cutout unit 204 that cuts out the video of the selected face, a video superimposing unit 205 that superimposes the face video and background information into an output video, and a display unit 206 that displays the output video on the display device 5. Composed.

  Next, an implementation example of a mechanism for cutting out the face of a person who uttered sound from a camera video and superimposing it on a material video will be described with reference to the flowchart of FIG.

  When the user instructs the start of the video conference between the transmission terminal 1 and the reception terminal 2, the process 1000 is executed at the transmission terminal 1 and the process 1100 is executed at the reception terminal 2. Hereinafter, it demonstrates according to a flowchart.

  In the transmission terminal 1 and the reception terminal 2, each CPU reads out the video conference program from the storage device, develops it on the memory, and executes the video conference program to develop the video conference function (1000, 1100).

The second video input means 103 of the transmission terminal 1 receives the background information 423 from an external or internal program and outputs it to the MUX means 104 (1001).
Also, the first video input unit 100 acquires the video information 421 from an imaging device such as an external or internal camera device, outputs it to the face recognition unit 101 and the MUX unit 104, and stores it (1002).

Next, the voice input unit 106 acquires the voice information 422 from the microphone device 8 (1005).
The MUX means 104 creates a stream 400 by multiplexing the video information 421, the audio information 422, and the background information 423, and transmits the stream 400 to the receiving device 2 via the communication means and the Internet 3 (1007).

The communication means 200 of the receiving terminal 2 receives the stream 400 transmitted by the transmitting terminal 1 in Step 1007 (1101), and outputs the received soot dream 400 to the DEMUX means 201.
The DEMUX means 201 decomposes the multiplexed stream 400 (1102).
Next, the video cutout unit 204 acquires the video information 421 from the DEMUX unit 201 (1103), and the second video acquisition unit 203 acquires the background information 423 (1104).

  Next, the face recognition unit 209 performs image analysis on the video information 421, detects where the face is, acquires the coordinate value, and outputs it to the cut-out information generation unit 210 (1113). This face detection is done by analyzing the image in pixel units and detecting features such as eyes, nose, mouth, and contours, or by matching against a database of face images to determine the face. Any method may be used. In short, it is only necessary to determine at which coordinate of the video information 421 the face is located.

  Next, the cutout information generation unit 210 receives the coordinate information acquired by the face recognition unit 209 in step 1113, generates cutout information 424 based on the coordinate information, and outputs it to the MUX unit 104 (1114). The cut-out information generation unit 210 stores the current time in the time information 510, the identification information of the video information 421 in the video identification information 511, and the information acquired from the coordinate information acquired in step 1113 in the origin information 513 and the size 514, respectively. To do. Further, the cut-out information generation unit 102 calculates in which direction the face position in the video information 421 is located and stores it in the cut-out information 424 as the direction information 515.

Next, the voice acquisition unit 211 acquires the voice information 422 from the DEMUX unit 201 (1106).
The voice acquisition unit 211 analyzes the voice to detect from which direction the voice has arrived, and generates directivity information 425 (1115).

  Here, the video selection unit 208 acquires the cutout information 424 in which the heading information 523 of the directivity information 425 matches the heading information 515 of the cutout information 424, and obtains the video identification information 511, the origin information 513, and the size 514. The acquired face is selected (1108).

Next, the video cutout unit 204 cuts out the video information based on the origin information 513 and the size 514 with respect to the video information 421 associated with the video identification information 511 to obtain a face video (1109).
Next, the video superimposing means 205 overwrites the face video on the background information 423 and superimposes the video to create new video information (1110).
The video superimposing means 205 outputs the created video information to the display device 5 (1111).

  With the above operation, it is possible to realize a mechanism in which the face of a person who has made a sound is cut out from the camera video and superimposed on the document video.

  Further, in step 1109 of the receiving terminal 2, the video information 421 is cut out in order by using the origin information 513 and the size 514 of the plurality of pieces of cutout information 424 instead of using the direction information 523 of the directivity information 425. By doing so, it is possible to realize a mechanism for displaying the faces of all the participants on the background information 423 one by one in order.

  In step 1110 of the receiving terminal 2, the plurality of face images generated in step 1109 are aligned and overwritten on the background information 423, thereby simultaneously displaying the faces of the plurality of participants on the background information 423. The mechanism can be realized.

  In the above-described embodiment, the video cutout unit 204 included in the receiving terminal 2 is configured to acquire a face video from the video information based on the cutout information 424 and the directivity information 425. In addition, the transmission terminal 1 may be configured to include a video cutout unit. Thereby, the cut-out information acquisition unit 202, the directivity information acquisition unit 207, and the video selection unit 208 can be omitted from the configuration of the receiving device 2. In this case, the video information transmitted from the transmission device 1 to the reception device 2 via the communication unit 105 is a face image that has already been subjected to the clipping process by the video clipping unit. The amount of information transmitted can be reduced.

  In addition, this invention is not limited to an above-described Example, Various modifications are included. For example, the above-described embodiments have been described in detail for easy understanding of the present invention, and are not necessarily limited to those having all the configurations described. Further, a part of the configuration of one embodiment can be replaced with the configuration of another embodiment, and the configuration of another embodiment can be added to the configuration of one embodiment. Further, it is possible to add, delete, and replace other configurations for a part of the configuration of each embodiment.

  Each of the above-described configurations, functions, processing units, processing means, and the like may be realized by hardware by designing a part or all of them with, for example, an integrated circuit. Each of the above-described configurations, functions, and the like may be realized by software by interpreting and executing a program that realizes each function by the processor. Information such as programs, tables, and files for realizing each function is stored in a memory, a hard disk, a recording device such as an SSD (Solid State Drive), or a recording medium such as an IC card, an SD card, or a DVD (Digital Versatile Disk). Can be put.

  Further, the control lines and information lines indicate what is considered necessary for the explanation, and not all the control lines and information lines on the product are necessarily shown. Actually, it may be considered that almost all the components are connected to each other.

  The above embodiment is effective in a video conference system that realizes a picture-on-picture in which a document video and a camera video are displayed in a superimposed manner in a communication system connected by radio and wire.

DESCRIPTION OF SYMBOLS 1 ... Transmission terminal 2 ... Reception terminal 400 ... Stream 410 ... Transmission information 424 ... Extraction information 425 ... Direction information 1000 ... Processing flow 1100 which implement | achieves operation | movement of the transmission terminal 1 ... Processing flow which implement | achieves operation | movement of the reception terminal 2

Claims (12)

A video conference system comprising a transmission terminal that transmits video and a reception terminal that receives video transmitted from the transmission terminal, wherein the transmission terminal includes:
First video input means for receiving input of the first video;
Second video input means for receiving input of the second video;
Face recognition means for recognizing a human face included in the first video;
Cutting information generating means for generating cutting information which is information indicating the coordinates of the face;
A first communication means for transmitting the first video, the second video and the cutout information;
The receiving terminal is
Second communication means for receiving the first video, the second video, and the cutout information;
Video cutting means for generating the face as a third video based on the first video and the clipping information;
Video superimposing means for superposing the third video and the second video to generate a fourth video;
Output means for outputting the fourth video,
A video conference system characterized by that. The video conference system according to claim 1,
The transmitting terminal is
Voice input means for inputting voice;
Directional information generating means for generating directional information indicating the direction of voice arrival,
Said 1st communication means is provided with the function to transmit said 1st image | video, said 2nd image | video, said cut-out information, and said directional information, The video conference system characterized by the above-mentioned. The video conference system according to claim 2,
The second communication means receives the first video, the second video, the cutout information, and the directional information,
The receiving terminal is
Based on the directivity information, from a plurality of third video created by the video clipping means, comprising video selection means for selecting a video to be superimposed on the second video,
The video superimposing unit generates the fourth video by superimposing the third video selected by the video selection unit and the second video,
A video conference system characterized by that. A video conference system comprising a transmission terminal that transmits video and a reception terminal that receives video transmitted from the transmission terminal, wherein the transmission terminal includes:
First video input means for receiving input of the first video;
Second video input means for receiving input of the second video;
A first communication means for transmitting the first video and the second video,
The receiving terminal is
Second communication means for receiving the first video and the second video;
Face recognition means for recognizing a human face included in the first video;
Cutting information generating means for generating cutting information which is information indicating the coordinates of the face;
Video cutting means for generating the face as a third video based on the first video and the clipping information;
Video superimposing means for superposing the third video and the second video to generate a fourth video;
Output means for outputting the fourth video,
A video conference system characterized by that. The video conference system according to claim 4,
The transmitting terminal includes voice input means for inputting voice,
The first communication means transmits the first video, the second video, and the audio;
A video conference system characterized by The video conference system according to claim 5,
The second communication means receives the first video, the second video, and the audio,
The receiving terminal is
Directional information generating means for generating directional information indicating the arrival direction of the voice;
Video selection means for selecting a video to be superimposed on the second video from a plurality of third videos created by the video clipping means based on the orientation information;
The video superimposing unit generates the fourth video by superimposing the third video selected by the video selection unit and the second video,
A video conference system characterized by that. A video conference method used in a video conference system comprising: a transmission terminal that transmits video; and a reception terminal that receives video transmitted from the transmission terminal, wherein the transmission terminal includes:
Receiving the input of the first video;
Receiving a second video input;
Recognizing a human face included in the first video;
Generating cutout information that is information indicating the coordinates of the face;
Transmitting the first video, the second video, and the cut-out information;
In the receiving device,
Receiving the first video, the second video and the clipping information;
Generating the face as a third video based on the first video and the clipping information;
Superposing the third video and the second video to generate a fourth video;
And a step of outputting the fourth video. The video conference method according to claim 7, wherein in the transmission device,
Receiving voice input;
Generating directivity information indicating a direction of arrival of the voice, and
In the step of transmitting the first video, the second video, and the cutout information, the directional information is transmitted.
A video conferencing method characterized by the above. 9. The video conference method according to claim 8, wherein in the receiving device,
Obtaining the directional information;
Selecting a video to be superimposed on the second video from a plurality of third videos based on the orientation information;
Generating a third video from the cut-out information and the directional information;
A video conferencing method characterized by: A video conference method used in a video conference system comprising: a transmission terminal that transmits video; and a reception terminal that receives video transmitted from the transmission terminal, wherein the transmission terminal includes:
Receiving the input of the first video;
Receiving a second video input;
Transmitting the first video and the second video, and
In the receiving terminal,
Receiving the first video and the second video;
Recognizing a human face included in the first video;
Generating cutout information that is information indicating the coordinates of the face;
Generating the face as a third video based on the first video and the clipping information;
Superposing the third video and the second video to generate a fourth video;
A video conferencing method comprising: a step of outputting a fourth video. The video conference method according to claim 10, wherein in the transmission terminal,
Receiving voice input;
And transmitting the voice. A video conference method comprising: 12. The video conference method according to claim 11, wherein the receiving apparatus includes:
Receiving the voice,
Generating directional information indicating a direction of arrival of the voice;
Selecting a video to be superimposed on the second video from the plurality of third videos based on the directivity information, and
In the step of generating the fourth image by superimposing the third image and the second image, the fourth image is generated by superimposing the selected third image and the second image. To
A video conferencing method characterized by the above.

Images