JP2013093741A - Communication device, communication system, and communication method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To surely grasp what kind of display is performed on a display screen of a communication partner.SOLUTION: A communication device for performing video communication through a network comprises: network transmission/reception means for performing transmission/reception of data with the network; decoding means for outputting a decoded video signal by decoding an encoded video signal received through the network transmission/reception means from a communication partner; screen configuration means for configuring a screen by using one or more video signals including the decoded video signal; and video encoding means for outputting an encoded video signal by encoding an input video. The video encoding means receives the screen configured by the screen configuration means as a first video signal, and encodes the first video signal to output the encoded video signal.

Description

  The present invention relates to a communication device, a communication system, and a communication method.

  There exists Unexamined-Japanese-Patent No. 2010-14162 (patent document 1) as background art of the said technical field. In this gazette, in order to grasp what kind of display is performed on the display screen of the partner site, the layout information of the partner screen is received and the layout of the partner side screen is reproduced on the own site side based on that information Is displayed.

JP 2010-141662 A

  In the above-mentioned patent document 1, only the screen layout information of the other party is received, and the screen layout of the other party is reproduced again on the own site side. At that time, the individual image data itself used to construct the screen does not use the image data transmitted from the partner side, but reproduces the screen layout using the image data existing on the local site side. . Therefore, for example, when a situation such as the video on the local site does not normally reach the other party due to some cause such as a communication failure, the image data used by the other party and the image existing on the local site side There is a discrepancy in the data, but this cannot be confirmed. As a result, there is a problem that the display screen content of the partner side and the display screen content of the partner screen reproduced at the local site side are inconsistent.

In order to solve the above problems, for example, the configuration described in the claims is adopted.
The present application includes a plurality of means for solving the above-described problems. For example, in a communication apparatus that performs video communication via a network, network transmission / reception means that transmits / receives data to / from the network. A screen is configured using decoding means for decoding the encoded video signal received from the communication partner through the network transmission / reception means and outputting the decoded video signal, and one or more video signals including the decoded video signal And a video encoding unit that encodes the input video and outputs an encoded video signal, wherein the video encoding unit converts the screen configured by the screen configuration unit to the first video. It is input as a signal, encodes the first video signal, and outputs an encoded video signal.

  According to the above means, it is possible to grasp what kind of display is being performed on the display screen of the partner site. Furthermore, even when the video on the local site side does not normally reach the other party due to some cause such as a communication failure, it is possible to reliably grasp the image actually displayed on the other party.

It is an example of the block diagram of a communication apparatus. It is an example of the block diagram of a communication system. It is an example of the image data of the base A. It is an example of the display screen of the base B. It is an example of the block diagram of a communication apparatus. It is an example of the display screen of the base B.

  Hereinafter, examples will be described with reference to the drawings.

  FIG. 2 shows an example of a communication system that transmits and receives media data such as video signals and audio signals in the first embodiment. Here, the media is a format for expressing information, and includes video information, audio signals, character information, computer electronic files, and the like.

  In FIG. 2, there are two bases, base A and base B. In base A, a transmitting / receiving device 11 and a home gateway (hereinafter referred to as HGW) 12 are installed, and these are connected to each other via a home network 13. The HGW 12 is connected to the public network 3. An address A1 is assigned to the transmission / reception apparatus 11, an address A2 is assigned to the HGW 12, and an address A3 is assigned to the public network side.

  Similarly, the base station B is provided with a transmission / reception device 21 and an HGW 22, which are connected to each other via a home network 23. The HGW 22 is connected to the public network 3. The transmission / reception device 21 is assigned an address B1, the HGW 22 is assigned an address B2 on the public network side, and an address B3 is assigned on the home network side.

  Further, a SIP server 4 is connected to the public network 3.

  The configuration of the transmission / reception device 11 will be described with reference to FIG. FIG. 1 is an example of a configuration diagram of the transmission / reception device 11. The transmission / reception device 11 includes a network transmission / reception unit 111, a media decoding unit 112, a display layout unit 113, a speaker 114, a control unit 115, a UI processing unit 116, an imaging processing unit 117, a microphone 118, a media encoding unit 119, SIP. A communication unit 1110 is provided.

  The network transmission / reception unit 111 is connected to the home network 13 and performs transmission / reception processing of media data such as messages and video / audio based on instructions from the control unit 115. Media data received by the network transmission / reception unit 111 is sent to the media decoding unit 112.

  The media decoding unit 112 receives media data from the network transmission / reception unit 111 and performs video and audio decoding processing based on instructions from the control unit 115. The video decoded by the media decoding unit 112 is sent to the display layout unit 113. The media decoding unit 112 can decode a plurality of video data. That is, when a plurality of pieces of video media data are received from the network transmission / reception unit 111, each is decoded, and the decoded result is sent to the display layout unit 113, respectively. In FIG. 1, for convenience, the main video and the sub video are illustrated.

  The audio decoded by the media decoding unit 112 is output via the speaker 114.

  The display layout unit 113 uses the plurality of video data received from the media decoding unit 112 to perform layout processing for composing and superimposing video to configure a display screen. The display screen data output from the display layout unit 113 is output and displayed on an external display device (not shown) via a display output terminal.

The control unit 115 manages state transitions necessary for performing communication processing, and controls operations of the network transmission / reception unit 111, the media decoding unit 112, the UI processing unit 116, and the media encoding unit 119.
In addition, the control unit 115 creates a transmission message to an external device and transmits the message via the network transmission / reception unit 111. The control unit 115 receives and interprets a received message from an external device via the network transmission / reception unit 111, and performs the above control based on the content.

  The UI processing unit 116 performs user interface processing. That is, an input operation such as transmission, disconnection, or reception operation from the user is accepted, and the contents are notified to the control unit 116. Further, based on an instruction from the control unit 115, an incoming call notification or the like is output to the user via the display layout unit 113 and / or the speaker 114.

  Note that the transmission / reception device 11 does not necessarily include the speaker 114, and may be configured to output audio data to an external speaker. Hereinafter, the transmission / reception device 21 will be described as having the same configuration as the transmission / reception device 11, but it is not necessary that both have the same configuration.

  In the following description, the media decoding unit 112, the display layout unit 113, the UI processing unit 116, and the media encoding unit 119 are described as hardware. However, these functions can also be realized by software by causing the control unit 115 to develop and execute programs having functions corresponding to the above in a memory (not shown). In the following, for simplicity of explanation, the media decoding unit 112, the display layout unit 113, the UI processing unit 116, and the media encoding unit 119 operate, including the case where the control circuit 115 executes a program corresponding to each function. A description will be given so that each process is executed as a subject.

  Similarly, the transmission / reception device 11 does not necessarily have to display an image via an external display device, and may be configured to include a display device.

Next, operations related to transmission of media data will be described.
The imaging processing unit 117 includes a lens and an imaging device therein, captures an image, and sends the contents to the media encoding unit 119. The microphone 118 inputs voice and sends the content to the media encoding unit 119.

  The media encoding unit 119 receives video and audio from the imaging processing unit 117 and the microphone 118, respectively, and encodes the video and audio into media data based on an instruction from the control unit 115.

  Further, an external video source (not shown) such as a personal computer (hereinafter referred to as PC) screen can be input to the media encoding unit 119 via an external video input terminal. When an external video source is input, the external video source is encoded into media data based on an instruction from the control unit 115.

  The encoded media data is sent to the network transmission / reception unit 111. Thereafter, the data is transmitted to an external network via the network transmission / reception unit 111.

  The SIP communication unit 1110 registers the SIP-URI of the transmission / reception apparatus 11 with respect to the SIP server 4 via the HGW 12. In addition to the above SIP-URI registration, the SIP communication unit 1110 communicates with other SIP devices on the public network using SIP. This SIP communication includes establishment of a bandwidth guarantee type session.

  The transmission / reception device 21 has the same configuration as the transmission / reception device 11 described above, and a description thereof will be omitted.

  Next, video transmission at the site A will be described.

  In the transmission / reception apparatus 11 at the site A, the image of the user Andy at the site A is captured as described above by the built-in imaging processing unit 117. The captured video is shown in FIG. Further, a PC is connected to the external input terminal of the transmission / reception device 11, and an output video (FIG. 3B) from the PC is input.

  These videos are each encoded by the media encoding unit 119, and the encoded media data is sent to the transmitting / receiving device at the base B via the network transmitting / receiving unit 111.

Next, processing at the base B will be described.
The transmission / reception device 21 at the site B receives the media data transmitted from the transmission / reception device 11 at the site A by the network transmission / reception unit 111. Among these, video media data includes a plurality of media data as described above.
The received media data is sent to the media decoding unit 112, where video and audio decoding processing is performed. Of the decoded videos, the video of the user Ady at the site A is sent to the display layout unit 113 as a main video, and the PC image at the site A is sent as a sub-video.

  The display layout unit 113 uses the above video data received from the media decoding unit 112 to perform layout processing for composing and superimposing video to configure a display screen. The example of the display screen data comprised in FIG. 4 is shown. (A) is the main video alone, (b) is the sub-video alone, and (c) and (d) are examples of the composite screen of the main video and the sub-video. Regarding the selection of these layouts, a user instruction is transmitted to the display layout unit 113 via the UI processing unit 116 and is selected based on the instruction.

  The display screen data configured by the display layout unit 113 is sent to the media encoding unit 119. The media encoding unit 119 encodes the received display screen data into media data. In addition, the video imaged by the imaging processing unit 117 at the site B and the audio input from the microphone 118 are similarly encoded into media data by the media encoding unit 119. These encoded media data are sent to the transmission / reception apparatus 11 at the site A via the network transmission / reception unit 111.

  Next, reception of video at site A will be described.

  The transmission / reception device 11 at the site A receives the media data transmitted from the transmission / reception device 21 at the site B by the network transmission / reception unit 111. The received media data is sent to the media decoding unit 112, where video and audio decoding processing is performed. The decoded video includes the video imaged by the imaging processing unit 117 of the site B and the display screen data of the site B. The control unit 115 outputs the video imaged by the imaging processing unit 117 of the site B and the display screen data of the site B to the display device via the layout display unit 113, and displays the image on the display device. At this time, if the layout display unit 113 of the site A configures the display screen data so that the display screen data at the site B is smaller than the video imaged at the site B, the user image display at the site B is displayed. It is possible to check the video of the user at the site A without disturbing.

  On the other hand, when the user at the site A desires to see the display screen at the site B in more detail, the display screen of the site B displayed at the site A is displayed, for example, in accordance with an instruction from the user at the site A. You may make it expand on a screen and hide the image | video of the user of the base B. FIG. Thereby, the user at the site A can check at what resolution the PC screen transmitted from the site A is displayed at the site B.

  As described above, the content of the display screen of the site B can be confirmed on the site A side. Further, the display screen data of the site B is created using the video actually received at the site B. Therefore, even if there is a problem in the video transmission from the base A to the base B for some reason, the display content of the base B including the fault screen can be confirmed and understood on the base A side. is there.

  In the above description, when encoding the display screen data into the media data, the media encoding unit 119 at the site B may encode the display screen data after reducing the resolution or the frame rate as necessary. . By encoding with the resolution or frame rate of the display screen data lower than that of the video captured by the imaging processing unit 117 of the base B that is the main video, the bandwidth of the network line and the encoding capability of the media encoding unit 119 are increased. Even in a limited case, it is possible to prioritize the video quality on the main video side and to check the display screen.

  In the above embodiment, the transmitting / receiving device 21 at the site B encodes the data of the entire display screen configured by the display layout unit 113 by the media encoding unit 119, and transmits / receives at the site A via the network transmitting / receiving unit 111. However, the media data sent from the transmitting / receiving device 11 at the site A may be replaced with the data of the entire display screen. Thereby, the user of the base A can confirm that the data transmitted to the base B has actually reached the base B. At this time, the transmission / reception device 21 at the site B can temporarily decode the media data received from the transmission / reception device 11 at the site A and not re-encode it. In this case, the processing load on the transmission / reception device 21 at the site B can be reduced. This is especially true when the transmission / reception device 21 at the site B displays the main video (or sub-video) from the site A on the full screen without performing layout processing. This is effective because the video signal becomes the display screen itself at the site B.

  In the present embodiment, the transmission / reception device 11 at the site A has been described as transmitting the PC image at the site A to the site B as a sub-video. However, the transmission / reception device at the site A stores a file displayed as a PC image. The file may be transmitted to the site B as a file. In this case, the transmission / reception device 21 at the site B may interpret the received file as a video and superimpose it on the main video as display data at the site B.

Next, Example 2 will be described. Description of the same parts as those in the first embodiment will be omitted.
The difference from the first embodiment is that the display screen on the site B side is transmitted based on a request from the site A side.

  The transmission / reception device 11 on the site A side receives a user instruction requesting transmission of a display screen to the site B in the UI processing unit 116. The UI processing unit 116 transmits the instruction to the control unit 115. When the control unit 115 receives the instruction, the control unit 115 transmits a display screen transmission request to the transmission / reception device 21 on the site B side via the network transmission / reception unit 111.

  In the transmission / reception apparatus 12 on the site B side, the network transmission / reception unit 111 receives the display screen transmission request from the site A side and transmits it to the control unit 115. Upon receiving the display screen transmission request, the control unit 115 controls the media encoding unit 119 based on the request, encodes the display screen data into media data, and transmits the media data to the transmitting / receiving device 12 on the site A side. To send to.

  As described above, whether or not to transmit the display screen on the site B side to the site A side can be controlled according to the request from the site A side.

  Further, when the transmitting / receiving device 21 on the site B side receives an instruction to invalidate transmission of the display screen from the user on the site B side through the UI processing unit 116, the control unit 115 that has received the instruction In spite of receiving the display screen transmission request from A, the media encoding unit 119 may be controlled not to transmit the display screen data to the site A. In that case, in consideration of the intention of the user at the base B, the display screen can be prevented from being transmitted, and privacy can be protected.

  In this case, the control unit 115 may transmit a notification that the display screen transmission request is invalidated to the base A via the network transmission / reception unit 111. The transmitting / receiving device 11 on the site A side that has received the notification may display a message corresponding to the notification to notify the user on the site A side. In this case, the user at the site A can know the reason why the screen is not displayed even though the transmission request for the display screen at the site B is instructed, and the convenience of the user can be improved. is there.

  Next, Example 3 will be described. Description of the same parts as those in the first embodiment will be omitted.

  In FIG. 5, the example of a block diagram of the transmission / reception apparatus 11 in a present Example is shown. The difference from FIG. 1 is that the output of the imaging processing unit 117 is input to the display layout unit 113. Accordingly, the display layout unit 113 can perform layout processing for configuring a display screen using local camera video on the site B side as needed in addition to video sent from the site A side. . That is, in addition to the example of the display screen data shown in FIG. 4, it is possible to create a display screen using local camera video as shown in FIG. The display screen data output from the display layout unit 113 is output to an external display device (not shown) via a display output terminal and displayed to the user on the site B side. Thereby, the user on the site B side can confirm how he / she is photographed, and the user's convenience can be improved.

  Furthermore, the display layout unit 113 outputs the display screen shown in FIG. 6A to the display output terminal, and simultaneously generates screen data shown in FIG. 6B excluding only the local camera video from the display output terminal. The latter may be sent to the media encoding unit 119, and the video may be encoded into media data and transmitted to the site A side.

  In this case, the video content overlaps with the camera video of the base B (video shot by the imaging processing unit 117) encoded by the media encoding unit 119 and transmitted to the base A side separately from the display screen data. Can be avoided, and it is possible to improve the intelligibility for the user.

  In addition, this invention is not limited to an above-described Example, Various modifications are included. For example, the above-described embodiments have been described in detail for easy understanding of the present invention, and are not necessarily limited to those having all the configurations described. In addition, a part of the configuration of a certain embodiment can be replaced with the configuration of another embodiment, and the configuration of another embodiment can be added to the configuration of a certain embodiment. Further, it is possible to add, delete, and replace other configurations for a part of the configuration of each embodiment.

  Each of the above-described configurations, functions, processing units, processing means, and the like may be realized by hardware by designing a part or all of them with, for example, an integrated circuit. Each of the above-described configurations, functions, and the like may be realized by software by interpreting and executing a program that realizes each function by the processor. Information such as programs, tables, and files for realizing each function can be stored in a recording device such as a memory, a hard disk, an SSD (Solid State Drive), or a recording medium such as an IC card, an SD card, or a DVD.

  Further, the control lines and information lines indicate what is considered necessary for the explanation, and not all the control lines and information lines on the product are necessarily shown. Actually, it may be considered that almost all the components are connected to each other.

1 ... Base A
2 ... Base B
3 ... External network (public network)
4 ... SIP servers 11, 12 ... transmitting / receiving devices 13, 23 ... home network

Claims (11)

In a communication device that performs video communication via a network,
Network transmitting / receiving means for transmitting / receiving data to / from the network;
Decoding means for decoding the encoded video signal received from the communication partner through the network transmission / reception means and outputting the decoded video signal;
Screen configuration means for configuring a screen using one or more video signals including the decoded video signal;
Video encoding means for encoding an input video signal and outputting an encoded video signal;
With
The video encoding means receives a screen configured by the screen configuration means as a first video signal, encodes the first video signal, and outputs an encoded video signal. apparatus. The communication device according to claim 1,
In addition to the network transmission / reception means, a local video input means for inputting local video is provided.
The video encoding means receives the local video input from the video input means as a second video signal in addition to the first video signal, encodes the second video signal, and generates an encoded video. Output signal,
The communication apparatus characterized in that the resolution of the first video signal is the same or smaller than the resolution of the second video signal. The communication device according to claim 1,
In addition to the network transmission / reception means, a local video input means for inputting local video is provided.
The video encoding means receives the local video input from the video input means as a second video signal in addition to the first video signal, encodes the second video signal, and generates an encoded video. A communication apparatus that outputs a signal and has a frame rate for encoding the first video signal equal to or lower than a frame rate for encoding the second video signal. The communication device according to claim 1,
Control means for receiving a transmission request for a configuration screen from the communication partner through the network transmission / reception means;
The said control means controls the said video encoding means to permit the output of the said encoding video signal, when the transmission request of the said structure screen is received from the said communicating party. The communication device according to claim 4, wherein
Comprising an operation input means for accepting an operation from a user;
The control means includes
When an instruction to invalidate the transmission of the configuration screen is given by the user via the operation input means, the encoded video signal is output regardless of whether the transmission request of the configuration screen is received from the communication partner. A communication apparatus for controlling the video encoding means so as not to permit it. The communication device according to claim 5, wherein
The control means includes
When the user is instructed to invalidate the transmission of the configuration screen via the operation input means and receives the transmission request for the configuration screen from the communication partner, the invalidation notification is transmitted to the communication partner. A communication device. The communication device according to claim 1,
Display output for screen display,
In addition to the network transmission / reception means, a local video input means for inputting local video is provided.
The screen composition means
A first configuration screen that receives the local video signal and configures a screen including the local video signal;
Output a second configuration screen that does not include the local video signal,
The video encoding means receives the second configuration screen configured by the screen configuration means as a first video signal, encodes the first video signal, and outputs an encoded video signal.
A communication apparatus that outputs the first configuration screen from the display output. The communication device according to claim 1,
The video encoding means receives a file input, encodes the input file, and includes the encoded file in the encoded video signal. In a communication system that performs video communication between a first communication device and a second communication device via a network,
The first communication device is
Network transmitting / receiving means for transmitting / receiving data to / from the network;
Decoding means for decoding the encoded video signal received from the second communication device through the network transmission / reception means and outputting a decoded video signal;
Screen configuration means for configuring a screen using one or more video signals including the decoded video signal;
Video encoding means for encoding an input video signal and outputting an encoded video signal;
With
The video encoding means receives a screen configured by the screen configuration means as a first video signal, encodes the first video signal, and outputs an encoded video signal. Device,
The second communication device is
A communication device that transmits an encoded video signal to the first communication device, and receives, decodes, and displays the encoded video signal transmitted from the first communication device. A communication system characterized by being. A second communication device in a communication system that performs video communication between a first communication device and a second communication device via a network,
The first communication device is
Network transmitting / receiving means for transmitting / receiving data to / from the network;
Decoding means for decoding the encoded video signal received from the second communication device through the network transmission / reception means and outputting a decoded video signal;
Screen configuration means for configuring a screen using one or more video signals including the decoded video signal;
Video encoding means for encoding an input video signal and outputting an encoded video signal;
With
The video encoding means receives a screen configured by the screen configuration means as a first video signal, encodes the first video signal, and outputs an encoded video signal. Device,
The second communication device is
A communication device that transmits an encoded video signal to the first communication device, and receives, decodes, and displays the encoded video signal transmitted from the first communication device. There is a communication device. In a communication method for performing video communication via a network,
A network receiving step for receiving data from the network;
A decoding step of decoding the encoded video signal received from the communication partner and outputting the decoded video signal;
A screen configuration step of configuring a screen using one or more video signals including the decoded video signal;
A video encoding step of encoding an input video signal and outputting an encoded video signal;
A network transmission step for transmitting data to the network;
With
In the video encoding step, the screen configured in the screen configuration step is input as a first video signal, the first video signal is encoded, and an encoded video signal is output. Method.

Images