JP5058042B2 - Image communication system and image communication method - Google Patents

Description

  The present invention relates to an image communication system and an image communication method for displaying received images on each other's screen by transmitting and receiving captured images both on the first and second image communication apparatuses.

  With an increase in data communication bandwidth, so-called videophone communication, in which image communication is performed simultaneously with telephone communication, can be easily realized.

  A videophone communication device (hereinafter simply abbreviated as “communication device”) that realizes a videophone communication includes an imaging unit for capturing an image of a caller, a communication unit for transmitting and receiving audio data and image data, It includes display means for displaying images, a handset for inputting and outputting audio, and the like.

  The image picked up by the image pickup means is transmitted as image data to the counterpart communication device and displayed on a part of the display means for confirmation of the caller.

  Such a communication system is used, for example, for video conferences and presentations connecting remote locations.

  During meetings and presentations, it may be possible to ask for explanations or ask questions by specifying what is displayed in the image sent from the other party. In that case, specify the object verbally. There is a need to. If the object is clear, even verbal explanations may be sufficiently communicated, but there is also an object that is unclear and it is difficult to specify the object accurately orally There is. If it only takes a long time to specify an object, or if the other party mistakenly recognizes an object that is different from the object, the meeting and presentation will be smooth without communication. It will not proceed to.

  In the image input / output system described in Patent Literature 1, when position information indicating a specific location on a document image is input on the presentation side using the position information input unit of the display tablet of the own device, the position information is transmitted via the transmission unit. The partner apparatus displays a cursor at the corresponding position of the document image displayed on the document image display unit of the display tablet based on the position information sent to the partner apparatus.

  In the information input device described in Patent Document 2, the camera device that recognizes the position of the pointer and the display device each have a polarization filter that transmits different polarized light, and transmits information by transmitting camera parameters and the like to the other side. Display the other party's video to match the displayed screen.

JP-A-8-9075 JP 2005-117285 A

  In the invention described in Patent Document 1, a specific position of a document image can be specified, but the presentation side can only specify the position of its own document and cannot be specified from the side receiving the presentation. In the invention described in Patent Document 2, the amount of data to be transmitted is large in order to transmit the designated position to the other party, and a hardware configuration is added such that it is necessary to attach different polarization filters between the camera device and the display device. There is a need to.

  An object of the present invention is to provide an image communication system and an image communication method that allow a partner to clearly grasp an object with a simple configuration.

The present invention provides an image communication system in which both the first and second image communication devices transmit and receive a captured video to display the received video on each other's screen.
The first image communication device includes:
First communication means for receiving a video sent from the second image communication device and transmitting the video to the second image communication device;
First imaging means for imaging a first main video;
First display means for displaying a second main video sent from the second image communication device;
A first image position specifying means for specifying any range of the second main image Ru transmitted from the second image communication apparatus,
First video extraction means for extracting a video displayed in a range designated by the first video position designation means as a first sub-video ,
The first communication means transmits the first main image the first image pickup means has captured the second image communication apparatus, when the first image extraction means for extracting a first sub-picture, the first sub-extracted Sending video to the second image communication device ;
When the first communication means receives the second sub-video sent from the second image communication device, the first display means displays the received second sub-video,
The second image communication device includes:
Second communication means for receiving video sent from the first image communication device and transmitting video to the first image communication device;
A second imaging means for imaging the second main video;
Second display means for displaying a first main video sent from the first image communication device;
Second video position specifying means for specifying an arbitrary range of the first main video sent from the first image communication device;
Second video extraction means for extracting video displayed in a range designated by the second video position designation means as second sub-video;
The second communication means transmits the second main video imaged by the second imaging means to the first image communication device, and when the second video extraction means extracts the second sub-video, the extracted second sub-video is extracted. Sending video to the first image communication device;
When the first communication means receives the first sub-picture sent from the first image communication apparatus, the second display means displays the received first sub-picture .

Further, the present invention provides an image communication system for displaying received images on each other's screen by transmitting and receiving the captured images by both the first and second image communication apparatuses.
The first image communication device includes:
First communication means for receiving a video sent from the second image communication device and transmitting the video to the second image communication device;
First imaging means for imaging a first main video;
First display means for displaying a second main video sent from the second image communication device;
A first image position specifying means for specifying any range of the second main image Ru transmitted from the second image communication apparatus,
First range extraction means for extracting range information indicating the range designated by the first video position designation means as first range information ;
The first communication means, wherein the first main video the first imaging means has captured transmitted to the second image communication apparatus, wherein the first range extracting means for extracting a first range information, extracted first range Sending information to the second image communication device ;
When the first display means receives the second range information sent from the second image communication device, the first display means generates a second sub-video from the first main video transmitted to the second image communication device based on the received second range information. Extract and display the video,
The second image communication device includes:
Second communication means for receiving video sent from the first image communication device and transmitting video to the first image communication device;
A second imaging means for imaging the second main video;
Second display means for displaying a first main video sent from the first image communication device;
Second video position specifying means for specifying an arbitrary range of the first main video sent from the first image communication device;
Second range extraction means for extracting range information indicating the range designated by the second video position designation means as second range information;
The second communication unit transmits the second main video imaged by the second imaging unit to the first image communication device, and the second range extraction unit extracts the second range information when the second range extraction unit extracts the second range information. Sending information to the first image communication device;
When the second display means receives the first range information sent from the first image communication device, based on the received first range information, the second display means generates a first sub-video from the second main video transmitted to the first image communication device. An image communication system characterized by extracting and displaying a video .

Further, the present invention provides an image communication system for displaying received images on each other's screen by transmitting and receiving the captured images by both the first and second image communication apparatuses.
The first image communication device includes:
First communication means for receiving a video sent from the second image communication device and transmitting the video to the second image communication device;
First imaging means for imaging a first main video;
First display means for displaying a second main video sent from the second image communication device;
A first image position specifying means for specifying any range of the second main image Ru transmitted from the second image communication apparatus,
First video extraction means for extracting a video displayed in a range designated by the first video position designation means as a first sub-video;
When the first video image extracting means for extracting a first sub-picture, generating a first first synthesized image by synthesizing the sub-picture extracted by the first main image and the first image extracting means taken by the first image pickup means First synthesizing means ,
The first communication means, when said first combining means generates a first composite image, the first synthesized image is transmitted to the second image communication apparatus, instead of the first main image the first image pickup means has captured ,
When the first communication means receives the second composite video sent from the second image communication device, the first display means displays the received second composite video,
The second image communication device includes:
Second communication means for receiving video sent from the first image communication device and transmitting video to the first image communication device;
A second imaging means for imaging the second main video;
Second display means for displaying a first main video sent from the first image communication device;
Second video position specifying means for specifying an arbitrary range of the first main video sent from the first image communication device;
Second video extraction means for extracting video displayed in a range designated by the second video position designation means as second sub-video;
When the second video extraction unit extracts the second sub-video, the second main video captured by the second imaging unit and the second sub-video extracted by the second video extraction unit are combined to generate a second composite image. Second synthesizing means,
The second communication means, when the second combining means for generating a second synthesized image, the second synthesized image is transmitted to the first image communication apparatus, instead of the second main image the second image pickup means has captured ,
The second display means is an image communication system that displays the received first composite video when the second communication means receives the first composite video sent from the first image communication device .

Further, the present invention provides an image communication method for displaying received images on each other's screen by both the first and second image communication devices transmitting and receiving captured images.
In the first image communication device,
A first imaging step of imaging a first main video;
A first main transmission step of transmitting the first main video to the second image communication device;
A first main display step of receiving and displaying a second main video sent from the second image communication device;
A first image position specifying step of specifying an arbitrary range of the second main image Ru transmitted from the second image communication apparatus,
A first video extraction step of extracting the video displayed in the range designated in the first video position designation step as a first sub-video;
When the first sub-image is extracted in the first image extraction step, a first sub-transmission step of transmitting the first sub-images that are extracted in the second image communication apparatus,
When receiving the second sub-video sent from the second image communication device, a first sub-display step for displaying the received second sub-video is executed.
In the second image communication device,
A second imaging step of imaging the second main video;
A second main transmission step of transmitting the second main video to the first image communication device;
A second main display step of receiving and displaying a first main video sent from the first image communication device;
A second video position specifying step of specifying an arbitrary range of the first main video sent from the first image communication device;
A second video extraction step of extracting the video displayed in the range designated in the second video position designation step as a second sub-video;
A second sub-transmission step of transmitting the extracted second sub-video to the first image communication device when the second sub-video is extracted in the second video extraction step;
When the first sub-video sent from the first image communication device is received, the second sub- display step of displaying the received first sub-video is executed.

Further, the present invention provides an image communication method for displaying received images on each other's screen by both the first and second image communication devices transmitting and receiving captured images.
In the first image communication device,
A first imaging step of imaging a first main video;
A first main transmission step of transmitting the first main video to the second image communication device;
A first main display step of receiving and displaying a second main video sent from the second image communication device;
A first image position specifying step of specifying an arbitrary range of the second main image Ru transmitted from the second image communication apparatus,
A first range extracting step of extracting range information indicating the range specified in the first video position specifying step as first range information ;
When the first range information is extracted in the first range extraction step, a first range transmission step of transmitting the first range information extracted in said second image communication apparatus,
When the second range information sent from the second image communication device is received, the second sub-video is extracted from the first main video transmitted to the second image communication device and displayed based on the received second range information. Performing the first sub-display process ;
In the second image communication device,
A second imaging step of imaging the second main video;
A second main transmission step of transmitting the second main video to the first image communication device;
A second main display step of receiving and displaying a first main video sent from the first image communication device;
A second image position specifying step of specifying an arbitrary range of the first main image Ru transmitted from the first image communication apparatus,
A second range extracting step of extracting range information indicating the range specified in the second video position specifying step as second range information ;
When the second range information is extracted in the second range extraction step, a second range transmission step of transmitting a second range information extracted in the first image communication apparatus,
When the first range information sent from the first image communication device is received, the first sub-video is extracted and displayed from the second main video transmitted to the first image communication device based on the received first range information. An image communication method characterized by executing a second sub display step .

Further, the present invention provides an image communication method for displaying received images on each other's screen by both the first and second image communication devices transmitting and receiving captured images.
In the first image communication device,
A first imaging step of imaging a first main video;
A first main transmission step of transmitting the first main video to the second image communication device;
A first main display step of receiving and displaying a second main video sent from the second image communication device;
A first image position specifying step of specifying an arbitrary range of the second main image Ru transmitted from the second image communication apparatus,
A first video extraction step of extracting the video displayed in the range designated in the first video position designation step as a first sub-video;
When the first sub-image is extracted in the first image extraction step, a first synthesis step of generating a first synthesized image by synthesizing the first sub-picture is extracted from the first main video,
A first composite transmission step of transmitting a first composite image to the second image communication device instead of the first main video when the first composite image is generated in the first composite step;
When the second composite video sent from the second image communication device is received, a first composite display step for displaying the received second composite video is performed.
In the second image communication device,
A second imaging step of imaging the second main video;
A second main transmission step of transmitting the second main video to the first image communication device;
A second main display step of receiving and displaying a first main video sent from the first image communication device;
A second image position specifying step of specifying an arbitrary range of the first main image Ru transmitted from the first image communication apparatus,
A second video extraction step of extracting the video displayed in the range designated in the second video position designation step as a second sub-video;
When the second sub-picture is extracted by the second image extraction step, a second synthesis step of generating a second synthesized image by synthesizing the second sub-picture is extracted from the second main video,
A second composite transmission step of transmitting a second composite image to the first image communication device instead of the second main video when the second composite image is generated in the second composite step;
When the first composite video sent from the first image communication device is received, a second composite display step of displaying the received first composite video is executed.

According to the present invention, the first image communication device and the second image communication device have the same configuration. Hereinafter, a case where the first sub-video is displayed from the first image communication apparatus to the second image communication apparatus will be described. The same applies to the case where the second sub-video is displayed from the second image communication device to the first image communication device.
In the first image communication device, when the first video position designating unit designates an arbitrary range of the second main video sent from the second image communication device, the first video extraction unit performs the first video position. The video displayed in the range specified by the specifying means is extracted as the first sub-video. The first communication means transmits the first sub-video extracted by the first video extraction means to the second image communication device.

In the second image communication apparatus, when the second communication means receives the first sub-picture sent from the first image communication apparatus, the second display means receives the first received by the second communication means. Display sub-picture.

  The operator of the first image communication apparatus only needs to specify the range in which the object to be notified to the other party is displayed, and the operator of the first image communication apparatus only needs to see the displayed sub-video. .

Therefore, the object can be clearly grasped by the other party with a simple configuration.
According to the present invention, in the first image communication device, when the first video position specifying means specifies an arbitrary range of the second main video sent from the second image communication device, the first communication means The first main video imaged by the first imaging means and range information indicating the range designated by the first video position designation means are transmitted to the second image communication device.

When the second communication means receives the first main video and range information sent from the first image communication apparatus, the second extraction means transmits the second image communication apparatus to the first image communication apparatus. A sub-picture is extracted from the second main video based on the range information received by the second communication means, and the second display means is extracted by the second extraction means received by the second communication means. Displayed sub-picture.

  The operator of the first image communication apparatus only needs to specify the range in which the object to be notified to the other party is displayed, and the operator of the first image communication apparatus only needs to see the displayed sub-video. .

  Therefore, the object can be clearly grasped by the other party with a simple configuration. Furthermore, since it is only necessary to transmit range information such as coordinates instead of the image data itself, the amount of data to be transmitted can be reduced.

According to the present invention, in the first image communication device, when the first video position designating unit designates an arbitrary range of the second main video sent from the second image communication device, the first video extraction unit. Extract the video displayed in the range specified by the first video position specifying means as the first sub-video. The first synthesizing unit generates a first synthesized image by synthesizing the first main video captured by the first imaging unit and the first sub-video extracted by the first video extracting unit, and the first communication unit includes: The first synthesized image synthesized by the first synthesizing means is transmitted to the second image communication apparatus.

In the second image communication device, when the second communication means receives the first composite image sent from the first image communication device, the second display means receives the second communication means. The first composite image is displayed.

  The operator of the first image communication apparatus only needs to specify the range in which the object to be notified to the other party is displayed, and the operator of the first image communication apparatus only needs to see the displayed sub-video. .

  Therefore, the object can be clearly grasped by the other party with a simple configuration. Furthermore, since the second image communication device only needs to display the received image, even a simpler configuration, such as a conventional image communication device, can display the sub-picture.

According to the invention, in the first image communication device, when an arbitrary range of the second main video sent from the second image communication device is designated in the first video position designation step, the first video extraction step Then, the video displayed in the range designated in the first video position designation step is extracted as the first sub-video. In the first transmission step, the first main video imaged in advance and the first sub-video extracted in the first video extraction step are transmitted to the second image communication device.

The second image communication apparatus, the second reception step, upon receiving the first sub-picture transmitted from said first image communication apparatus, in the second display step, first received by the second reception step Display sub-picture.

  The operator of the first image communication apparatus only needs to specify the range in which the object to be notified to the other party is displayed, and the operator of the first image communication apparatus only needs to see the displayed sub-video. .

Therefore, the object can be clearly grasped by the other party with a simple configuration.
According to the present invention, in the first image communication device, when an arbitrary range of the second main video sent from the second image communication device is specified in the first video position specifying step, the first transmission step is performed. The first main video imaged in advance and the range information indicating the range specified in the first video position specifying step are transmitted to the second image communication device.

In the second image communication device, when receiving the first main video and the range information sent from the first image communication device in the second reception step, the second image communication device transmits the first main image and range information to the first image communication device in the second extraction step. from the second main image, based on the range information received by the second reception step, to extract the second sub-picture, in the second display step, the first main video and the received by the second reception step displaying a second sub-picture extracted in the second extraction step.

  The operator of the first image communication apparatus only needs to specify the range in which the object to be notified to the other party is displayed, and the operator of the first image communication apparatus only needs to see the displayed sub-video. .

  Therefore, the object can be clearly grasped by the other party with a simple configuration. Furthermore, since it is only necessary to transmit range information such as coordinates instead of the image data itself, the amount of data to be transmitted can be reduced.

According to the invention, in the first image communication device, when an arbitrary range of the second main video sent from the second image communication device is designated in the first video position designation step, the first video extraction step Then, the video displayed in the range designated in the first video position designation step is extracted as the first sub-video. In the first synthesis step, a first main image captured in advance and the first sub-video extracted in the first video extraction step are combined to generate a first composite image, and in the first transmission step, the first sub-image is generated . The first combined image combined in the combining step is transmitted to the second image communication device.

In the second image communication apparatus, the second reception step, upon receiving the first synthesized image sent from the first image communication apparatus, in the second display step, received by the second reception step wherein The first composite image is displayed.

  The operator of the first image communication apparatus only needs to specify the range in which the object to be notified to the other party is displayed, and the operator of the first image communication apparatus only needs to see the displayed sub-video. .

  Therefore, the object can be clearly grasped by the other party with a simple configuration. Furthermore, since the second image communication device only needs to display the received image, even a simpler configuration, such as a conventional image communication device, can display the sub-picture.

FIG. 1 is a schematic diagram showing an outline of an image communication system 1 according to an embodiment of the present invention.
The image communication system 1 of the present invention is composed of two image communication devices, a first image communication device 100 and a second image communication device 200, and images captured by the camera 101 of the first image communication device 100 are image communication. Is transmitted to the second image communication apparatus 200, and the received image is displayed on the second display 202 of the second image communication apparatus. In parallel with this, the video imaged by the camera 201 of the second image communication device 200 is transmitted to the first image communication device 100 by image communication, and the received image is displayed on the first display 102 of the first image communication device. To do.

  Further, the first image communication apparatus 100 is provided with the first touch panel 103 overlaid on the first display 102, and can designate a specific area for the image displayed on the first display 102. Yes. Specifically, it is assumed that the operator of the first image communication apparatus 100 wants to specify the object 300 in the image captured by the second camera 201.

  The operator of the first image communication apparatus 100 is the object 330 via the first touch panel 103 with the touch pen 104 while the image transmitted from the second image communication apparatus 200 is displayed on the first display 102. Is input to the second image communication apparatus 200, the information about the image of the portion of the object 300 displayed in the input specified area 105 is input.

  The second image communication apparatus 200 displays the image displayed in the designated area 105 on the second display 202 of its own device based on the received information. At this time, the captured image sent from the first image communication apparatus 100 and the image on which the object 300 is displayed are displayed superimposed on the second display 202.

  The operator of the second image communication apparatus 200 can clearly and quickly grasp the object 300 designated by the operator of the first image communication apparatus 100 by looking at the image displayed on the second display 202. Can do.

  In order to further designate a part of the designated area 105, if a designated part in the designated area is designated with a pointer (cursor, arrow, etc.) via the first touch panel 103, the information of the designated part is transferred to the second image communication. The image is transmitted to the apparatus 200, and the second image communication apparatus 200 adds the pointer of the designated portion to the image in the designated area and displays it.

  In addition, when the object 300 displayed on the first display 102 is small or when it is desired to clarify the designated portion, the instruction for enlarging the image in the designated area 105 is input, and the enlargement instruction is then displayed. The image is transmitted to the image communication apparatus 200, and the image of the designated area 105 is enlarged and displayed on the second display 202 in the second image communication apparatus 200.

  Note that the second image communication apparatus 200 is also provided with a second touch panel 203, and a designated area can be input as in the first image communication apparatus 100. That is, the first image communication device 100 and the second image communication device 200 have the same configuration, and even if the device on the input side of the designated area is the first image communication device 100, the second image communication device The apparatus 200 may be used.

FIG. 2 is a block diagram illustrating a configuration of the image communication system 1.
In the present embodiment, the image communication system 1 includes two image communication devices, a first image communication device 100 and a second image communication device 200.

  A first image communication device (hereinafter abbreviated as “first device”) 100 includes a first imaging unit 10 and a first image processing unit 11, and the first image processing unit 11 further includes first video processing. Unit 12, first communication unit 13, first storage unit 14, first pointer control unit 15, first coordinate detection unit 16, first display unit 17, and first tablet 18.

  The first video processing unit 12 includes a shared video generation unit 12a, an image enlargement processing unit 12b, a PIP processing unit 12c, and a video composition unit 12d.

  A second image communication device (hereinafter abbreviated as “second device”) 200 includes a second imaging unit 20 and a second image processing unit 21, and the second image processing unit 21 further includes second video processing. Unit 22, second communication unit 23, second storage unit 24, second pointer control unit 25, second coordinate detection unit 26, second display unit 27, and second tablet 28.

  The second video processing unit 22 includes a shared video generation unit 22a, an image enlargement processing unit 22b, a PIP processing unit 22c, and a video composition unit 22d.

  Since the first device 100 and the second device 200 have the same configuration as described above, the configuration of the first device 100 will be described below. Also, as shown in FIG. 1, the side that designates the designated area 105 is the first device 100, and the side that displays the designated image is the second device 200.

  The first imaging unit 10 is realized by, for example, a CCD (Charge Coupled Device) camera, for example, and is a first camera 101 that receives reflected light from a subject and outputs the received light amount as an analog video signal.

  The first image processing unit 11 converts the video signal output from the first imaging unit 10 into digital video data, performs necessary image processing, and transmits the digital image data to the second device 200. In addition, necessary image processing is also performed on the image in the designated area input by the first tablet 18 and transmitted to the second device 200.

  The shared video generation unit 12a acquires and stores the shared video coordinate information 30 as information related to the shared video that is the sub-video. As shown in FIG. 1, the shared video is an image displayed in the designated area 105 when the designated area 105 is input, and is an image captured and sent by the second device 200 ( This corresponds to a part of the main video).

  The image enlargement processing unit 12b performs enlargement processing of the shared video or the main video, and stores the inputted enlargement size (enlargement magnification, dimensions after enlargement, etc.) 31.

  The PIP processing unit 12c superimposes and displays the shared video on the main video by picture-in-picture (PIP) processing.

  The shared video to be displayed superimposed on the main video is different between the device that designates the designated area and the device that receives the designated shared video.

  In the first device 100 that designates the designated area 105, the first display unit superimposes the main video based on the image data received from the second device 200 and the shared video in the designated area 105 designated by the own device. 17 is displayed.

  In the second device 200 that receives the designated shared video, the main video based on the image data received from the first device 100 and the shared video in the designated area 105 designated by the first device 100 are superimposed. Displayed on the second display unit 27.

  The PIP processing unit 12c performs PIP processing, that is, PIP ON / OFF indicating whether to display the shared video regardless of whether the specified region is input and the shared video is displayed or the specified shared video is displayed. Store information. When the shared video is displayed on the first display unit 17, PIP is ON, and when not displayed, the PIP is OFF. The PIP position information 33 and the PIP size information 34 are information indicating the position coordinates at which the shared video is displayed on the first display unit 17 and the size thereof when performing PIP display. These pieces of information are configured so that change values can be input from the first tablet 18 so that the operator of the apparatus can arbitrarily change the information, and the stored information is updated when the information is input. If not changed, PIP processing is performed based on an initial value given in advance.

  In the PIP process, the video composition unit 12d does not superimpose the shared video on the main video, but creates one image data by synthesizing the main video and the shared video, and the synthesized image data is stored in the second device 200. Send to.

  The first communication unit 13 is configured to be capable of image communication, telephone communication, and data communication with the second device 200, and can perform communication via a public telephone communication network, communication via a LAN (Local Area Network), and the like. It is not limited to infrastructure.

  The first communication unit 13 refers to information related to communication of the connection destination communication device as the connection destination information 35, in this embodiment the second device 200, for example, a telephone number, an IP address, a communication protocol to be used, and the like. Communication with the apparatus 200 is performed.

  The first storage unit 14 is a main video storage unit 14 a that stores the main video transmitted from the second device 200, and the video and main video that are captured by the first imaging unit 10 and transmitted to the second device 200. A PIP video 36 that is a video to be superimposed on the main video when the PIP processing unit 12c performs PIP processing is also stored. The

  The first pointer control unit 15 controls the position of the pointer input by the first tablet 18. The first coordinate detection unit 16 detects the coordinates input by the first tablet 18, and passes the detected coordinate information to the first pointer control unit 15 and the first video processing unit 12 as necessary. The first display unit 17 is the first display 102 that mainly displays the main video sent from the second device 200 and the shared video in the designated area.

  The first tablet 18 is composed of an input device such as a first touch panel 103 and a pen 104, and an operator uses the pen 104 to trace over the first display 102 provided with the first touch panel 103 overlaid. Since the coordinates of the corresponding part are input, an intuitive operation is possible.

  Below, the image communication process by the image communication system 1 is demonstrated in detail using a flowchart.

  When the shared video is displayed on the second device 200, information related to the shared video is transmitted from the first device 100 to the second device 200, and the second device 200 displays the shared video on the second display unit based on the related information. 27.

  As for information related to the shared video, the first mode in which image data in the designated area 105 is transmitted as information, the second mode in which the coordinates of the designated area are transmitted as information, the shared video in the designated area 105, and the first imaging unit. There is a third mode in which the captured main video is synthesized and the synthesized video is transmitted as information.

  FIG. 3 is a flowchart showing the image communication processing of the first form. FIG. 3A (a) shows the flow of the first device 100 on the side that designates the shared video, and FIG. 3B (b) shows the flow of the second device 200 on the side that displays the designated shared video.

  In the first device 100, in step A1, the first communication unit 13 receives image data serving as the main video from the second device 200, which is the counterpart communication device, and displays the main video on the first display unit 17 in step A2. Let

  In step A3, it is determined whether or not the designated area 105 has been input via the first tablet 18. If input, the process proceeds to step A4. If not input, the process returns to step A1 to repeat reception and display of the main video.

  In step A4, the coordinates of the input area are extracted based on the detection result of the first coordinate detector 16, and stored in the shared video generator 12a. In step A5, based on the stored shared video coordinate information 30, the image corresponding to the designated area 105 of the main video received from the second device 200 is used as the shared video, and the image data of the designated area portion is stored. Is stored as related information.

  In step A6, the shared video is trimmed so that it can be easily distinguished when displayed superimposed on the main video, and the display size is read in step A7.

  In step A8, the shared image is PIP-displayed at the display size and display position read by the first display unit 17 by the PIP processing unit 12c. In the first device 100, the shared video that the operator has specified can be confirmed by looking at the image PIP-displayed on the first display unit 17.

  In step A9, when the size of the shared video is changed by a predetermined input procedure, the size information 34 of the shared video to be transmitted to the second device 200 is updated in step A10. In step A11, when a pointer is added to the shared video by a predetermined input procedure, in step A12, a pointer image is added to the shared video to be transmitted to the second device 200 and stored in the transmission video storage unit 14b. .

  Note that the size can be changed by dragging the edge of the shared video currently displayed via the first tablet 18, for example. The addition of the pointer can also be performed by performing a pointing operation on any of the shared images currently displayed via the first tablet 18.

  In step A <b> 13, a shared video display command is transmitted to the second device 200 in order to display the shared video on the second display unit 27 of the second device 200. The shared video display command is a command for activating the PIP processing unit 22c in the second device 200 to display the transmitted shared video in PIP.

  In step A14, the partial image data to be the shared video stored in the transmission video storage unit 14b is transmitted to the second device 200.

  Although not shown, the first device 100 captures the main video with the first imaging unit 10 and transmits it to the second device 200. The transmission timing of the main video may be transmitted simultaneously in the shared video transmission step, or may be transmitted in different steps.

  In the second device 200, the second imaging unit 20 captures the main video in Step B1, and transmits the image data serving as the main video to the first device 100 in Step B2.

  In step B3, it is determined whether a shared video display command has been received. If it has been received, the process proceeds to step B4. If not received, the process returns to step B1 to repeat the shooting and transmission of the main video.

  In step B4, partial image data to be shared video is received. In step B5, the PIP processing unit 22 is activated, and the display position is read in step B5 and the display size is read in step B6.

  In step B <b> 7, the PIP processing unit 22 superimposes the shared video on the main video sent from the first device 100 and causes the second display unit 27 to perform PIP display. In step B8, when the display position of the shared video is changed by a predetermined input procedure, the display position of the shared video displayed in step B9 is changed.

  When the size of the shared video is changed by a predetermined input procedure in step B10, the size of the shared video displayed in step B11 is changed.

  The display position can be changed, for example, by dragging a portion other than the edge of the shared video currently displayed via the second tablet 28. The size can be changed by dragging the edge of the shared video currently displayed via the second tablet 28, for example.

  In step B12, when an instruction to end the display of the shared video is input, the shared video is deleted and the process ends. If no end instruction is input, the process returns to step B7 to continue the display.

  FIG. 4 is a flowchart showing the image communication processing of the second form. 4A (a) shows the flow of the first device 100 on the side that specifies the shared video, and FIG. 4B (b) shows the flow of the second device 200 on the side that displays the specified shared video.

  In the first device 100, in step C1, the first communication unit 13 receives image data as a main video from the second device 200, which is the counterpart communication device, and displays the main video on the first display unit 17 in step C2. Let

  In Step C3, it is determined whether or not the designated area 105 has been input via the first tablet 18, and if it is input, the process proceeds to Step C4, and if not input, the process returns to Step C1 to repeat the reception and display of the main video.

  In step C4, based on the detection result of the first coordinate detection unit 16, the coordinates of the input region are extracted and stored in the shared video generation unit 12a. In step C5, based on the stored shared video coordinate information 30, a portion of the main video received from the second device 200 corresponding to the designated area 105 is set as a shared video, and the coordinate information of the designated area is set to the second coordinate information. Stored as related information to be transmitted to the apparatus 200.

  In step C6, the shared video is trimmed so that it can be easily distinguished from the main video, and the display size is read in step C7.

  In step C8, the PIP processing unit 12c causes the shared image to be PIP-displayed at the display size and display position read into the first display unit 17. In the first device 100, the shared video that the operator has specified can be confirmed by looking at the image PIP-displayed on the first display unit 17.

  In step C9, when the size of the shared video is changed by a predetermined input procedure, the size information 34 of the shared video to be transmitted to the second device 200 is updated in step C10. In step C11, when a pointer is added to the shared video by a predetermined input procedure, in step C12, a pointer image is added to the shared video to be transmitted to the second device 200 and stored in the transmission video storage unit 14b. .

  In step C <b> 13, a shared video display command is transmitted to the second device 200 in order to display the shared video on the second display unit 27 of the second device 200. The shared video display command is a command for causing the PIP processing unit 22c to be activated in the second device 200 and causing the shared video to be PIP-displayed based on the coordinate information of the transmitted shared video.

  In step C14, coordinate information indicating the designated area to be the shared video stored in the transmission video storage unit 14b is transmitted to the second device 200.

  Following the coordinate information, in step C15, information indicating the display size of the shared video is transmitted to the second device 200, and in step C16, information indicating the pointer position added to the shared video is transmitted to the second device 200.

  Although not shown, the first device 100 captures the main video with the first imaging unit 10 and transmits it to the second device 200. The transmission timing of the main video may be transmitted simultaneously in the shared video transmission step, or may be transmitted in different steps.

  In the second device 200, the second imaging unit 20 captures the main video in Step D1, and transmits the image data serving as the main video to the first device 100 in Step D2.

  In step D3, it is determined whether or not a shared video display command has been received. If it has been received, the process proceeds to step D4. If not received, the process returns to step D1 to repeat the shooting and transmission of the main video.

  In step D4, coordinate information indicating a portion that becomes a shared video is received. Subsequently, in step D5, display size information of the shared video is received, and in step D6, information indicating a pointer position is received.

  In step D7, coordinate information is applied to the captured video transmitted to the first display unit 100, and the image of the corresponding part is extracted as a shared video. In step D8, the display size of the shared video is changed based on the received display size.

  In step D9, it is determined whether pointer display additional information has been received. If it has been received, a pointer is added to the shared video in step D10.

  In step D <b> 11, the PIP processing unit 22 c superimposes the extracted shared video on the main video sent from the first device 100 and displays the PIP on the second display unit 27. In step D12, when the display position of the shared video is changed by a predetermined input procedure, the display position of the shared video displayed in step D13 is changed.

  When the size of the shared video is changed by a predetermined input procedure in step D14, the size of the shared video displayed in step D15 is changed.

  In step D16, when an instruction to end the display of the shared video is input, the shared video is deleted and the process ends. If no end instruction is input, the process returns to step D11 to continue the display.

  FIG. 5 is a flowchart showing image communication processing of the third form. FIG. 5A (a) shows the flow of the first device 100 on the side that specifies the shared video, and FIG. 5B (b) shows the flow of the second device 200 on the side that displays the specified shared video.

  In the first device 100, in step E1, the first communication unit 13 receives image data serving as a main video from the second device 200, which is the counterpart communication device, and displays the main video on the first display unit 17 in step E2. Let

  In step E3, it is determined whether or not the designated area 105 has been input via the first tablet 18. If input, the process proceeds to step E4. If not input, the process returns to step E1 to repeat the reception and display of the main video.

  In step E4, based on the detection result of the first coordinate detector 16, the coordinates of the input area are extracted and stored in the shared video generator 12a. In step E5, based on the stored shared video coordinate information 30, a portion of the main video received from the second device 200 is taken as a shared video image corresponding to the designated area 105, and image data of the designated area portion is stored. Remember.

  In step E6, the shared video is trimmed so that it can be easily distinguished when displayed superimposed on the main video, and in step E7, the display size is read.

  In step E8, the PIP processing unit 12c causes the shared image to be PIP-displayed at the display size and display position read into the first display unit 17. In the first device 100, the shared video that the operator has specified can be confirmed by looking at the image PIP-displayed on the first display unit 17.

  In step E9, when the size of the shared video is changed by a predetermined input procedure, the size information 34 of the shared video to be transmitted to the second device 200 is updated in step E10. In step E11, when a pointer is added to the shared video by a predetermined input procedure, a pointer image is added to the shared video and stored in the transmission video storage unit 14b in step E12.

  In step E13, the first imaging unit 10 captures the main video, and in step E14, the image data serving as the main video and the partial image data serving as the shared video are synthesized. In step E15, the composite image data is transmitted to the second device 200 as related information.

  In the present embodiment, the shooting of the main video by the first device 100 has been described as step E13 so that it is clear that the video is being shot. However, the shooting timing is not limited to this, and the shared video is recorded in step E14. Just take a picture before compositing.

  In the second device 200, the second imaging unit 20 captures the main video in step F1, and in step F2, the image data serving as the main video is transmitted to the first device 100.

  In step F3, composite image data that is a composite video obtained by combining the main video and the shared video is received from the first apparatus 100, and the composite image is displayed in step F4.

As described above, by specifying an area on the first apparatus 100 side and displaying an image of the specified area on the second apparatus 200 on the other side, the operator of the second apparatus 200 can display the first apparatus 1.
The object targeted by the 00 operator can be easily and clearly grasped.

  In the above description, in the first embodiment, once the designated area is designated, the corresponding image data is transmitted to the other party. However, in the first device 100, an instruction to cancel the designation of the shared video is input. Until this time, every time the main video is sent from the second communication device 100, it is also possible to generate image data to be a shared video and continue to send it to the second device 200. In the second mode, it is also possible to continue to generate and display image data to be shared video in the second communication device 100 until a coordinate change or an instruction to cancel the designation of shared video is input. is there.

It is a schematic diagram which shows the outline of the image communication system 1 which is one Embodiment of this invention. 1 is a block diagram showing a configuration of an image communication system 1. FIG. It is a flowchart which shows the image communication process of a 1st form. It is a flowchart which shows the image communication process of a 1st form. It is a flowchart which shows the image communication process of a 2nd form. It is a flowchart which shows the image communication process of a 2nd form. It is a flowchart which shows the image communication process of a 3rd form. It is a flowchart which shows the image communication process of a 3rd form.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image communication system 100 1st image communication apparatus 101,201 Camera 102 1st display 103 1st touch panel 104 Pen 105 Designated area 200 2nd image communication apparatus 202 2nd display 203 2nd touch panel 300 Target object

Claims (6)

In the image communication system in which both the first and second image communication devices transmit and receive the captured video to display the received video on each other's screen,
The first image communication device includes:
First communication means for receiving a video sent from the second image communication device and transmitting the video to the second image communication device;
First imaging means for imaging a first main video;
First display means for displaying a second main video sent from the second image communication device;
A first image position specifying means for specifying any range of the second main image Ru transmitted from the second image communication apparatus,
First video extraction means for extracting a video displayed in a range designated by the first video position designation means as a first sub-video ,
The first communication means transmits the first main image the first image pickup means has captured the second image communication apparatus, when the first image extraction means for extracting a first sub-picture, the first sub-extracted Sending video to the second image communication device ;
When the first communication means receives the second sub-video sent from the second image communication device, the first display means displays the received second sub-video,
The second image communication device includes:
Second communication means for receiving video sent from the first image communication device and transmitting video to the first image communication device;
A second imaging means for imaging the second main video;
Second display means for displaying a first main video sent from the first image communication device;
Second video position specifying means for specifying an arbitrary range of the first main video sent from the first image communication device;
Second video extraction means for extracting video displayed in a range designated by the second video position designation means as second sub-video;
The second communication means transmits the second main video imaged by the second imaging means to the first image communication device, and when the second video extraction means extracts the second sub-video, the extracted second sub-video is extracted. Sending video to the first image communication device;
The second display means displays the received first sub-video when the first communication means receives the first sub-video sent from the first image communication device . In the image communication system in which both the first and second image communication devices transmit and receive the captured video to display the received video on each other's screen,
The first image communication device includes:
First communication means for receiving a video sent from the second image communication device and transmitting the video to the second image communication device;
First imaging means for imaging a first main video;
First display means for displaying a second main video sent from the second image communication device;
A first image position specifying means for specifying any range of the second main image Ru transmitted from the second image communication apparatus,
First range extraction means for extracting range information indicating the range designated by the first video position designation means as first range information ;
The first communication means, wherein the first main video the first imaging means has captured transmitted to the second image communication apparatus, wherein the first range extracting means for extracting a first range information, extracted first range Sending information to the second image communication device ;
When the first display means receives the second range information sent from the second image communication device, the first display means generates a second sub-video from the first main video transmitted to the second image communication device based on the received second range information. Extract and display the video,
The second image communication device includes:
Second communication means for receiving video sent from the first image communication device and transmitting video to the first image communication device;
A second imaging means for imaging the second main video;
Second display means for displaying a first main video sent from the first image communication device;
Second video position specifying means for specifying an arbitrary range of the first main video sent from the first image communication device;
Second range extraction means for extracting range information indicating the range designated by the second video position designation means as second range information;
The second communication unit transmits the second main video imaged by the second imaging unit to the first image communication device, and the second range extraction unit extracts the second range information when the second range extraction unit extracts the second range information. Sending information to the first image communication device;
When the second display means receives the first range information sent from the first image communication device, based on the received first range information, the second display means generates a first sub-video from the second main video transmitted to the first image communication device. An image communication system, wherein video is extracted and displayed . In the image communication system in which both the first and second image communication devices transmit and receive the captured video to display the received video on each other's screen,
The first image communication device includes:
First communication means for receiving a video sent from the second image communication device and transmitting the video to the second image communication device;
First imaging means for imaging a first main video;
First display means for displaying a second main video sent from the second image communication device;
A first image position specifying means for specifying any range of the second main image Ru transmitted from the second image communication apparatus,
First video extraction means for extracting a video displayed in a range designated by the first video position designation means as a first sub-video;
When the first video image extracting means for extracting a first sub-picture, generating a first first synthesized image by synthesizing the sub-picture extracted by the first main image and the first image extracting means taken by the first image pickup means First synthesizing means ,
The first communication means, when said first combining means generates a first composite image, the first synthesized image is transmitted to the second image communication apparatus, instead of the first main image the first image pickup means has captured ,
When the first communication means receives the second composite video sent from the second image communication device, the first display means displays the received second composite video,
The second image communication device includes:
Second communication means for receiving video sent from the first image communication device and transmitting video to the first image communication device;
A second imaging means for imaging the second main video;
Second display means for displaying a first main video sent from the first image communication device;
Second video position specifying means for specifying an arbitrary range of the first main video sent from the first image communication device;
Second video extraction means for extracting video displayed in a range designated by the second video position designation means as second sub-video;
When the second video extraction unit extracts the second sub-video, the second main video captured by the second imaging unit and the second sub-video extracted by the second video extraction unit are combined to generate a second composite image. Second synthesizing means,
The second communication means, when the second combining means for generating a second synthesized image, the second synthesized image is transmitted to the first image communication apparatus, instead of the second main image the second image pickup means has captured ,
It said second display means, when said first composite video image sent from the first image communication apparatus a second communication means for receiving an image communication system and displaying the first combined video received. In the image communication method of displaying the received video on each other's screen by both the first and second image communication devices transmitting and receiving the captured video,
In the first image communication device,
A first imaging step of imaging a first main video;
A first main transmission step of transmitting the first main video to the second image communication device;
A first main display step of receiving and displaying a second main video sent from the second image communication device;
A first image position specifying step of specifying an arbitrary range of the second main image Ru transmitted from the second image communication apparatus,
A first video extraction step of extracting the video displayed in the range designated in the first video position designation step as a first sub-video;
When the first sub-image is extracted in the first image extraction step, a first sub-transmission step of transmitting the first sub-images that are extracted in the second image communication apparatus,
When receiving the second sub-video sent from the second image communication device, a first sub-display step for displaying the received second sub-video is executed.
In the second image communication device,
A second imaging step of imaging the second main video;
A second main transmission step of transmitting the second main video to the first image communication device;
A second main display step of receiving and displaying a first main video sent from the first image communication device;
A second video position specifying step of specifying an arbitrary range of the first main video sent from the first image communication device;
A second video extraction step of extracting the video displayed in the range designated in the second video position designation step as a second sub-video;
A second sub-transmission step of transmitting the extracted second sub-video to the first image communication device when the second sub-video is extracted in the second video extraction step;
Upon receiving the first sub-picture to be sent from the first image communication apparatus, an image communication method and executes a second sub-display step of displaying the first sub-video received. In the image communication method of displaying the received video on each other's screen by both the first and second image communication devices transmitting and receiving the captured video,
In the first image communication device,
A first imaging step of imaging a first main video;
A first main transmission step of transmitting the first main video to the second image communication device;
A first main display step of receiving and displaying a second main video sent from the second image communication device;
A first image position specifying step of specifying an arbitrary range of the second main image Ru transmitted from the second image communication apparatus,
A first range extracting step of extracting range information indicating the range specified in the first video position specifying step as first range information ;
When the first range information is extracted in the first range extraction step, a first range transmission step of transmitting the first range information extracted in said second image communication apparatus,
When the second range information sent from the second image communication device is received, the second sub-video is extracted from the first main video transmitted to the second image communication device and displayed based on the received second range information. Performing the first sub-display process ;
In the second image communication device,
A second imaging step of imaging the second main video;
A second main transmission step of transmitting the second main video to the first image communication device;
A second main display step of receiving and displaying a first main video sent from the first image communication device;
A second image position specifying step of specifying an arbitrary range of the first main image Ru transmitted from the first image communication apparatus,
A second range extracting step of extracting range information indicating the range specified in the second video position specifying step as second range information ;
When the second range information is extracted in the second range extraction step, a second range transmission step of transmitting a second range information extracted in the first image communication apparatus,
When the first range information sent from the first image communication device is received, the first sub-video is extracted and displayed from the second main video transmitted to the first image communication device based on the received first range information. And a second sub-display step . In the image communication method of displaying the received video on each other's screen by both the first and second image communication devices transmitting and receiving the captured video,
In the first image communication device,
A first imaging step of imaging a first main video;
A first main transmission step of transmitting the first main video to the second image communication device;
A first main display step of receiving and displaying a second main video sent from the second image communication device;
A first image position specifying step of specifying an arbitrary range of the second main image Ru transmitted from the second image communication apparatus,
A first video extraction step of extracting the video displayed in the range designated in the first video position designation step as a first sub-video;
When the first sub-image is extracted in the first image extraction step, a first synthesis step of generating a first synthesized image by synthesizing the first sub-picture is extracted from the first main video,
A first composite transmission step of transmitting a first composite image to the second image communication device instead of the first main video when the first composite image is generated in the first composite step;
When the second composite video sent from the second image communication device is received, a first composite display step for displaying the received second composite video is performed.
In the second image communication device,
A second imaging step of imaging the second main video;
A second main transmission step of transmitting the second main video to the first image communication device;
A second main display step of receiving and displaying a first main video sent from the first image communication device;
A second image position specifying step of specifying an arbitrary range of the first main image Ru transmitted from the first image communication apparatus,
A second video extraction step of extracting the video displayed in the range designated in the second video position designation step as a second sub-video;
When the second sub-picture is extracted by the second image extraction step, a second synthesis step of generating a second synthesized image by synthesizing the second sub-picture is extracted from the second main video,
A second composite transmission step of transmitting a second composite image to the first image communication device instead of the second main video when the second composite image is generated in the second composite step;
An image communication method comprising: executing a second composite display step of displaying the received first composite video when receiving the first composite video sent from the first image communication device .

Images