JP2001053947A - Image information transmitting device and system therefor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To attain the remote control of a screen to freely shift a visual point in a range that is permitted by the opposite party while keeping the privacy of both its own and opposite party sides by selecting an optional transmission range in an optional transmission permission range included in an image area that is selected out of an image fetched by an image acquisition means and transmitting the selected transmission range as an image. SOLUTION: A selection position controller 14 limits the images of a full screen which are photographed by an image input device 10 via an input image selection device 11 after image pickup selection range information is generated. A user inputs a transmission permission range that can be opened by the user himself to the opposite party of communication via the controller 14 to an input image that has its output enable range limited by the device 11 and has its reduced area. The controller 14 sets a transmittable range via a transmission permission image selection device 12 and controls the transmission permission range. Then the images included in the transmission permission range are selected by the opposite party of communication via a transmitting image selection device 13 or dynamically selected.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an information processing apparatus having a communication function for taking and forming an image of a self-side and an image of a counterpart. The present invention relates to an image information transmission device and an image information transmission system that enable switching between a partner image and a partner image so that natural communication can be achieved while ensuring privacy.

[0002]

2. Description of the Related Art As a system for communicating with a remote partner using not only voice but also video,
There are a TV (video) conference system and a video phone system. These systems include not only speakers and microphones, but also a TV camera and a monitor, and can send and receive images via a communication line, allowing them to send images of themselves and nearby objects to a distant partner. The video of the other party is displayed on the monitor device while being sent, and the dialogue can be advanced while watching the figure of the other party.

[0003] As described above, the TV conference system and the videophone system can communicate with each other while watching the other party from a distance, even if they are far away. as,
It is a convenient system that allows you to communicate.

However, such a system generally has a mechanism in which a television camera is fixed at a fixed position, and an image within a range captured at the camera position is sent to the other party for display. The recipient cannot see what he wants to see. Therefore, even if there is a request to see where the other party is looking, or to see the speaker up when all the people are meeting, or to look at all the attendees,
Only images in the fixed field of view of the fixed camera could be seen.

A technique for solving this problem is described in "Image Processing Apparatus with Videophone", Japanese Patent Laid-Open No. 09-200714.
The technology disclosed in Japanese Unexamined Patent Publication (Kokai) No. H10-216, has been developed. This technology is briefly described as follows.

That is, when transmitting a photographed image of one's own side to the other party by videophone, the photographed image is effectively processed by, for example, hiding a part that the other party does not want to see or combining an image different from the actual one. And send it.

More specifically, this system has a normal mode and an image processing mode. When a videophone line is connected to the image processing mode, the CPU applies a mosaic pattern to a part of the self-portrait photographed by the image photographing unit. The captured image is processed by performing the superimposition process, and the processed image is transmitted to the other party.

However, in the case of this system, it is a technique for maintaining the security of the self-side image, and does not allow the other party to perform panning to see another desired place.

Also, "" Videophone "", JP-A-06-70
No. 315 "also relates to a video telephone system, which is a technology relating to a video telephone system capable of enlarging and reducing a received image and using a special device. This is a videophone technology capable of displaying image data in an enlarged or reduced size without displaying the image data.

However, this system is characterized in that image data is enlarged or reduced without using a special device, and it is not possible to pan a desired portion or secure security.

[0011]

In any case, according to the prior art, when trying to communicate without meeting with each other, for example, in the case of a videophone or the like, remote control of an image is performed on the transmission side. I was not able to reflect my privacy awareness well.

In particular, regarding camera images, there are only two states of sending and not sending, and it is not possible to finely and dynamically set an allowable range that may change depending on the other party.

In a normal face-to-face dialogue state, interlocutors can see each other's favorite places when they like, so that it is possible to freely operate the image on the operated side by themselves in communication. This is important. It is also important that the privacy of the operated side of the image to be transmitted can be protected at the same time.

For example, when a videophone call is suddenly received in a state where a private time has not considered a meeting with another person, the lens of the camera is closed or the imaging is stopped. You have to answer the call or ignore the incoming call, and you cannot dispel the mental resistance of the communication using images at the time of initial introduction.

An object of the present invention is to provide remote control of the screen so that the viewpoint can be freely changed within the range permitted by the other party while protecting privacy by respecting the wills of the self side and the other party. It is an object of the present invention to provide an image information transmitting apparatus and an image information transmitting system that realize a technology for enabling the image information transmitting apparatus.

[0016]

SUMMARY OF THE INVENTION According to the present invention, speakers can see each other's desired images, as in communication in which they meet at the same place and face-to-face with each other. In order to be able to protect the privacy, it is configured as follows.

[1] According to the present invention, an arbitrary transmission permitted range in the image area is selected from the image captured by the image acquisition means, and further, an arbitrary transmission range within the selected transmission permitted range is selected. Thus, an arbitrary transmission range within which transmission is actually performed within the selected transmission permission range is transmitted as an image.

As a result, the degree of freedom when installing the image acquisition means (camera) is expanded, and it is not necessary to re-install the camera when changing the screen to be transmitted or to use a motor to drive.

[2] The present invention is also characterized in that an arbitrary range in the image captured by the image acquiring means is selected, and an arbitrary range in the selected arbitrary image is selected as a transmission permitted range. .

As a result, the degree of freedom in installing the image acquisition means (camera) further increases than in the configuration of [1].

[3] Further, in the configuration of the item [1], there is provided a means provided on the receiver side for determining a transmission range to be actually transmitted and provided to the transmission range selection means. I do.

Thus, the receiver can select his / her favorite received image while respecting the sender's privacy.

[4] In transmitting an image, the size of a frame image is set to the size of an image in an arbitrary transmission range to be actually transmitted. High resolution is achieved by dividing an arbitrary transmission range to be selected and actually transmitted in units of the selected size, and allocating a transmission time per one frame image to a transmission time of the divided image to transmit. , And the divided image is reassembled on the other side.

Thus, it is possible to perform communication of an image having a higher resolution than the standard according to the standard without changing the standard. Conceptually, it is regarded as a reblock that covers a unit of the DCT region.

[5] Further, in the configuration of the item [1], a means for dynamically changing a transmission image size during communication is provided.

As a result, it is possible to dynamically change the image size.

[6] In the configuration of the item [1], the image data line is common to the switching between the self-side image and the other-side image, and the same control line is used to control the switching between the self-side image and the other-side image. It is characterized in that it is configured to be performed by using.

As a result, the number of necessary signal lines can be reduced.

[7] In the configuration of the item [1], the image data line is common to the switching between the self-side image and the partner image, and the switching control between the self-side image and the partner image is controlled by the same data line. Is sent as a command by using the command.

Thus, the number of necessary signal lines can be further reduced.

[8] In the configuration of [1], a storage device (storage means) is provided to hold an image prepared in advance, and an image in the transmission permitted range is an image from a camera (image acquisition means). And one of the images from the storage device is selected.

Thus, not only images from the camera (image acquisition means) but also images prepared in advance can be freely transmitted.

[9] Further, the present invention relates to a system in which the image capturing means captures images of each other, sends them to the other party, and displays them on the image display means provided on each side to be used for dialogue. Used at a fixed point as each imaging means, so as to use an imaging means having a wide imaging area, and a means for setting a range to allow transmission from the image area obtained by this imaging means, By providing a means for obtaining an image area of a predetermined size smaller than the image area in the image area within the allowable range set as an actual transmission image area, to obtain an image of the transmission image area,
The image obtained in this way is obtained by means for obtaining from the image area within the acceptable range in the image by the imaging means, and the obtained image is transmitted to the other party, and the transmitted image is obtained. An image display means for receiving an image and displaying it as an image, and generating a corresponding command by performing an operation of instructing movement so that the displayed image becomes an image of another area in the communication permission area. Transmission means.

[0034] According to such a configuration, the image capturing means captures images of each other, sends them to the other party, displays them on the image display means provided on each side, and uses them for dialogue. The imaging means having a wide imaging area is fixed at a fixed point, and a range in which transmission in the image area obtained by the imaging means is permitted is set, and an image area within the allowable range and a predetermined size smaller than this is set. When the image of the area is set as an actual transmission image area and the image of that area is sent to the other party, and the received image can be viewed on the monitor screen (image display means), By sending the command, if the content of the command is a desired designated position movement within the communication permission area, the transmission image area can be moved to the desired position and transmitted to the other party. So as to. Thereby, the communicator moves the transmission image area to the area of the place to be viewed, so as to move the actual line of sight in the communication permission image permitted by the other party. Areas that the other party does not want to show to others are completely restricted, while images of other areas can be freely viewed by remote control. In other words, it is possible to obtain an image information transmission system in which a part that the user does not want to show is not shown, and the other area can be freely observed by the other party.

[10] Further, in the system of [9], a storage means for holding an image prepared in advance and a selection means for selecting an image of the storage means or an image of the transmission image area are provided. The image selected by the means is transmitted to the other party.

[0036] According to this configuration, even when a scene in which communication with the other party is not assumed in advance during the private time is encountered, it is possible to flexibly respond by using the prepared image. Even in a situation such as "I don't want to capture the current state", it is possible to respond to communication without any trouble.

Therefore, according to the system of the present invention, it is possible to reduce the mental resistance at the time of the initial introduction of “communication using an image” which cannot be wiped out as a sensory rudder.

[11] In the configuration of the item [6],
Providing means for generating at least one of pointer information indicating a desired position of the transmission image or pointer information indicating a desired position of the partner image, so that the generated pointer information is transmitted by the transmission means,
A pointer image having a desired shape is displayed on the display screen of the image display means in accordance with the received pointer information.

In this way, since the position of the viewpoint can be understood, it is possible to communicate by grasping the point of interest of the opponent and the trend of the admiration of the opponent like a normal face-to-face conversation.

[12] In the configuration of the item [11], the means for generating pointer information causes the difference between the pointer positions and the information calculated from the difference information to be output as pointer information for transmission. It is characterized by the following.

Thus, it is possible to transmit the pointer information while minimizing the code amount of the pointer information.

[13] Further, there is provided a first selecting means for selecting an arbitrary captured image transmission permitted range within the image area from an image captured by the image obtaining means, and a storing means. A second selection unit that selects a plurality of images prepared in advance and a transmission-permitted image from the image acquisition unit, or selects a plurality of images and divides the layers to overlap the image to obtain a transmission-permitted range image; There is provided a third selecting means for selecting an arbitrary transmission image to be actually transmitted within the transmission permitted range image, and a means for transmitting the transmission image selected by the third selecting means.

Thus, an image within the transmission permitted range can be selected and transmitted from the image obtained by the image obtaining unit and the plurality of images prepared in advance in the storage unit.

[14] Further, in a system for performing communication between the first device and the second device, when a transmission image shape of a communication partner does not resemble its own display image shape,
It is characterized by having a scaling function for scaling so as to always include the shape of the partner image. In addition, when the transmission image shape of the communication partner does not resemble the display image shape of the other party, the display shape of the other party is added by adding a prepared image to the own transmission image in accordance with the display image shape of the other party. It is characterized by having a function to make it similar to.

Thus, even when the transmitted and received image shape does not resemble the display image shape of the display, the image can be displayed without a sense of incongruity.

[0046]

Embodiments of the present invention will be described below with reference to the drawings. According to the present invention, when communicating with each other while transmitting their own images while viewing the images, the other area can be freely observed by the other area by hiding the area of the image that the user does not want to show to others. In this way, a technique is provided that allows the user to freely pan an image desired to be viewed within the range permitted by the other party while respecting the will of the self side and the other party and protecting privacy.

Further, in order to eliminate the inconvenience of having to set the position of the camera manually in advance by setting the position of the self-portrait in the conventional TV conference system, in the present invention, the transmission permission is set. Provided is a technique capable of reducing the fine adjustment of the camera by using a camera capable of photographing a self-portrait photographing range wider than an image. In addition, this eliminates the driving portion of the camera unit and enables quick panning and the like.

As a particularly important technique for ensuring privacy and enabling panning by the other party's operation, the present invention selects a transmission permission range that is wider than the image transmission image. The operated side cuts out and transmits a desired image from the screen within the permitted range. Thereby, it is possible to select an image in consideration of privacy. Also, since the actual transmission image is smaller than the communication permission range, the communication capacity can be reduced.

Further, the transmission permission range can be changed by the other party, dynamically changed, or only the image prepared in advance can be permitted, thereby enabling flexible communication. I do.

Hereinafter, the details will be described. FIG. 1 shows an embodiment of the system of the present invention. The configuration shown in this figure shows the components for one terminal. In actual use, at least two terminals are prepared and connected to each other via a transmission line. At a minimum, one system is composed of two terminals, and the terminals send and monitor images on their own terminal side. FIG. 2 shows an example of image input in the present system.

In FIG. 1, reference numeral 10 denotes an image input device;
Is an input image selection device, 12 is a transmission permitted image selection device, 1
3 is a transmission image selection device, 14 is a selected position control device, 15
Is a storage device, 20 is a transmission image control device, 21 is a partner image control device, 22 is a transmission information synthesis device, 23 is a reception information separation device, 30 is a transmission pointer control device, 31 is a reception pointer control device, and 40 is an image display. Device, 50 is a transmission / reception device, and 51 is a transmission path. Of these, the image input device 10 is a television camera fixed to a predetermined position in advance with respect to a subject, and The image signal is obtained by fixedly capturing an image of a desired range including a fixed point from a fixed point, and outputs a full-screen image 110 (see 110 in FIG. 2) captured by the camera as a signal. This image input device 10 is used as an imaging lens,
For example, a wide-angle lens is used, so that an image having a wide field of view can be captured although the camera position is fixed and the lens is fixed.

Further, the selected position control device 14 outputs a full-screen image 110 captured by the camera (see 110 in FIG. 2).
On the other hand, imaging selection range information as information for generating position information of an image range of a predetermined allowable maximum frame and restricting output of an image region exceeding the range is generated, and an image outside this range is generated. Input image selection device 1 so as not to output
1 is a device for controlling the device 1.

As a result, it is possible to change the camera installation position and perform high-speed panning, etc., here.
This eliminates the need for manually resetting the camera position or driving the camera with a motor.

The selected position control device 14 determines that a range within an input image 111 (see 111 in FIG. 2) described later may be disclosed to another person within the range. This is also a device for controlling the transmission-permitted image selection device 12 so as not to output an image outside this range.

To determine the transmission permission range 112,
Using a pointing device such as a cross key, a joystick, and a mouse, for example, a frame is displayed on a display on which an image is displayed, and the position of the frame is moved and adjusted by operating the pointing device to allow the user to perform the operation. The selection position control device 14 can determine an appropriate area in order to secure the privacy of the user, and an image outside this range is not output to the selection position control device 14. In this way, the transmission-permitted image selecting device 12 may be controlled.

The input image selecting device 11 is a device for restricting an image to be output only for an image in an area within the range indicated by the information of the imaging selection range input from the selected position control device 14. , The obtained image is restricted to an image in an area within the imaging selection range with respect to the full-screen image 110 captured by the camera, and the restricted image is transmitted to the transmission-permitted image selection device 1.
2 image input image 111 (see FIG. 2).

Further, the transmission-permitted image selecting device 12
Transmission permission range 11 input from selected position control device 14
2 is a device that controls the sending image range of the input image 111 so that the range of the sending image is the image of the area of the transmission permission range 112 based on the information of the transmission image control device 20. When an instruction to extract a specified position image corresponding to a command described later as the transmission image 113 is received, an image of the image area corresponding to the specified position in the image in the range of the input image 111 is extracted as the transmission image 113. It also has functions.

The transmission image selection device 13 transmits the transmission permitted range 112 output via the transmission permission image selection device 12.
, Or an image selected from the permitted range image or the public image stored in the storage device 15 as a transmission image. The storage device 15 transmits the transmission image output from the transmission image selection device 13. The image 113 can be stored as an allowable range image or a public image, such as an image of the user in a state where the dress is prepared in advance in preparation for unexpected communication such as private time, a scene to be shown to the other party, etc. It can be used for purposes such as storing prepared images.

The transmission image control device 20 receives command-related information transmitted from the other party out of the information separated by the reception information separation device 23,
If the transmission image is to be selected and the position to be the transmission image within the transmission permission range 112 is specified, the image at the specified position is transmitted when the transmission is permitted. It has a function of controlling the transmission image selection device 13 so as to extract the image 113.

The partner image control device 21 is an operation device for inputting information such as the direction of the partner image desired to be viewed by the partner device (the partner terminal) and the public image number information. For example, a position operating means (for example, a joystick, a cross cursor key, a cursor key, a pointing device, etc.) operated by a user of the present system to perform screen panning, a command key, a numeric keypad, and the like are provided. This is a device that converts the input content into an instruction (command) corresponding to the content and outputs it to the transmission information synthesizing device 22.

The reception information separation device 23 is connected to the transmission line 51
The transmission pointer control device 30 separates and extracts the received information from the image information, the command relationship, and the pointer information relationship and extracts the image information, the command relationship, and the pointer information relationship. It generates pointer information for indicating, for example, by optically detecting the sight line direction of the pupil, obtaining the position information, and expressing the position information on the monitor screen with bright spot information corresponding to the position information. The eye tracker, which is a well-known gaze tracking device, is used as detection means, and the eye tracker is attached to a speaker so that gaze detection information obtained from the eye tracker is sent as gaze control information, or a transmission image is monitored. When the pointing device is pointed on the monitor screen, its position information is obtained, and the obtained line-of-sight detection information is used as line-of-sight control information. Such as by using a device or to send Te are considered.

The reception pointer control device 31 uses the line-of-sight control information obtained from the partner device to place an arbitrary-shaped pointer image at a position corresponding to the line-of-sight position in the image. This is a device that forms an image.

The image display device 40 operates the input image selection device 11, operates the setting information and the transmission-permitted image selection device 12, and outputs the setting information, the output image of the transmission image selection device 13 and the reception information separation device 23. This is a display for displaying image information on a screen.

Further, the transmission information synthesizing device 22 transmits the image selected by the transmission image selecting device 13 to the partner image control device 2.
1 and the transmission pointer information output from the transmission pointer control device 30 to generate transmission information.
2 is transmitted to the transmission line 51,
This is a device that receives transmission information from the other party transmitted through the transmission path 51 and provides the received information to the reception information separation device 23.

Next, the operation of the present system having such a configuration will be described.

The full-screen image 110 captured and input by the image input device 10 is limited by the input image selection device 11 by the information of the imaging selection range input from the selection position control device 14, and the input image 111 ( 2 in FIG.
11).

When changing the imaging region, the selected position control device 14 changes the imaging selection range. by this,
The part can be changed without moving the position of the camera (image input device 10) and without changing the direction of the camera lens. This is the image input device 10
For example, a wide-angle lens such as a fish-eye lens is used as the imaging lens, so that an image having a wide field of view can be obtained.

In contrast to a full screen image 110 which is an image having a wide field of view obtained from the image input device 10, the input image 111 whose area has been reduced by the restriction on the range that can be output by the input image selection device 11. (See 111 in FIG. 2)
In response to this, a transmission permission range 112 that the user himself / herself may disclose to others is input from the selection position control device 14, and the transmission permission image selection device 12 sets and controls the transmission permission range 112 accordingly. . As a result, the intention of the sender can be reflected, and privacy can be protected. The image within the transmission permission range 112 obtained in this way is selected by the transmission image selection device 13 by the communication partner or dynamically. The extracted image is the transmission image 113 (see FIG. 2).

However, the transmission image actually transmitted is not limited to the image from the transmission-permitted image selection device 12 but may be the image from the storage device 15. Which one to use
This is selected by the transmission image selection device 13.

As described above, the transmission-permitted image selecting device 1
2 or an arbitrary image selected from the permitted range image from the storage device 15 is transmitted to the transmission information synthesizing device 22 as a transmission image.

Incidentally, in the case of the present system, the transmission image can be permitted to be operated by the communication partner as described later, and the control is performed by the transmission image control device 20. Here, the operation content from the communication partner is sent in a command format. The transmitted command is transmitted by being embedded in image data from the other party, as will be described later in detail.

For example, when the present system is applied to a TV phone, an image browser, or the like, a communication partner such as the TV phone, the image browser, or the like operates the partner image control device 21 in its own system to issue a command. Put out.

That is, the opponent image control device 21 is for inputting a direction to be viewed by the opponent and a public image number. When an instruction (command) for a public image number or the like is issued, this information is
After being combined with other signals (for example, the transmission image selected by the transmission image selection device 13 and the pointer information) in 2, the transmission / reception device 50 outputs the signal to the transmission path 51.

On the other hand, the receiving side receives the transmission information transmitted from the other side via the transmission line 51 and gives the received information to the reception information separating device 23 to separate the information. By this separation processing, the command-related information and the image-related information included in the information transmitted from the other party are separated. Among the separated information, the received information separating device 23 gives image-related information to the image display device 40 to display it as an image, and also displays the command-related information as the transmission image control device 2.
By giving the value to 0, a transmission image is selected and a position as a transmission image within the transmission permission range 112 is designated.

The transmission image control device 20 controls the transmission image selection device 13 so as to extract the image at the designated position as the transmission image 113 if the transmission is permitted. .

As a result, for example, if the command content is “designating a position to be a transmission image in the transmission permission range 112”, the command content within the transmission permission range 112 (the area indicated by the outline 112 in FIG. 2) Panning to position, ie
The screen can be moved, and this image is sent to the other party via the transmission information synthesizing device 22 and the transmission / reception device 50.
Can be received and observed on the image display device 40 of the own system.

Similarly, if the command content is a panning of the public image of the other party, and if the request content is permitted, the public image of the other party is set within the transmission permission range 112. Can be changed. At this time, the image that is actually being transmitted can be sent to the transmission information synthesizing device 22 and the transmitted image can be confirmed on the image display device 40.

As a result, an image area smaller than the communication permission image having a large image area is sent to the other party as an actual transmission image area, and the other party can view the received image on the monitor screen. In this case, by sending a command from the other party, it is possible to move the transmission image area to an arbitrary position within the communication permission area and send it to the other party, and the communicator is permitted by the other party It is possible to freely view a desired place while respecting the security matters of the other party, like moving an actual gaze in the communication permission image. In other words, it is possible to obtain a system that does not show a part that the user does not want to see and allows the other party to freely observe the other area.

In the case of the present system, the storage device 15 stores an inconvenient image prepared in advance in case of an unexpected situation, and the output of the image input device 10 according to the situation. Instead of the image, the stored image alone or an image obtained by combining the stored image, the own captured image, and the background image in the memory can be sent to the other party. This can be realized by causing the transmission image control device 20 to control the transmission image selection device 13 to select the image.

By doing so, for example, even in the case of encountering a scene where communication with the other party was not assumed in advance in private time, it is possible to flexibly respond by using the prepared image. become,
`` I can't see a person when I wake up, '' `` the hairstyle is messed up, '' `` I don't want to meet people with this outfit, '' `` I don't want to be seen with this face, '' `` It is possible to respond to the communication without any trouble even in a situation that is inconvenient for the user himself, such as "I don't like the situation."

It is also possible to provide a means for selecting an image at the initial stage and a transmission allowable range of the image based on the additional information of the communication partner, so that the transmission allowable range at the initial stage can be automatically set. In this way, it is possible to save the trouble of setting the initial transmission permission range, and further, it is possible to deal with the acquaintance and the salesman separately,
It will also be possible to protect privacy rationally.

From the above, according to the system of the present invention, it is generally possible to reduce the mental resistance at the time of initial introduction of “communication using an image” which cannot be wiped out as a sensory rudder. .

By applying this system, any high-resolution video can be easily transferred.

FIG. 3 shows an example of transfer of a high-resolution image. Although FIG. 3 shows a case of a rectangle, the present invention can of course be applied to a case of a triangle or a regular hexagonal meter. To transfer a high-resolution image when the communication conditions are the same, the size of the image to be sent at a time may be reduced. That is, scan 1 frame screen 1
Rather than completing in one pass, the image is divided into multiple parts, and the time required for transmission of the original one frame is allocated in units of the division, and the original image is assigned the number of pixels of one frame to the divided image by transmission. Image transmission at high resolution, and one frame may be completed gradually.

For this purpose, a plurality of transmission images 113 smaller in size than the image occupying the transmission permission range 112 are cut out, and the cut-out order is transmitted to the communication partner. By doing this, if the communication conditions are the same, the time required to complete a screen of one frame will depend on the number of cutouts, but high-resolution communication can be easily performed.

[0086] Such high-resolution communication is very effective for communication devices having a learning phase, since communication can be performed using standards except for parameters relating to image assembly.

In FIG. 3, a combined transmission image 114 in which a plurality of communication images 113 within a standard range are transmitted from the transmission-permitted image 112 is transmitted. Unit communication image 1
13 regarding the transmission order, the horizontal priority, the vertical priority,
Various methods such as zigzag scanning may be applied, and a method suitable for the purpose of communication and the method of use may be adopted. When the combination transmission image 114 is the same as the transmission permission image 112, all the image ranges permitted by the communication partner can be transmitted.

FIG. 4 shows an example of mounting of the image display device 40.

The image transfer method of the self-side image 200 and the other-side image 201 to the image display device 40 is such that when YUV 16 [bit] words are transferred with respect to the data line,
There are 16 self-side image data lines 210, 16 partner image data lines 211, and image control lines 220 (for example, three lines of a vertical synchronization signal, a horizontal synchronization signal, and a field line).

Therefore, in the case of this example, a considerable mounting area is required for mounting. Therefore, this image control line 2
In the case of an image display device that does not use a field signal, such as a liquid crystal display, the number of required control lines is reduced by using field lines as information lines for the self-side image and the partner side image. Decrease.

FIG. 5 shows a method in which a command key is assigned to a specific value of data by further proceeding the concept of FIG.
If a form in which the meaning of the control line is expressed by a value subsequent thereto is adopted, a further effect of reducing the number of signal lines can be expected, which is advantageous in mounting. The same data can be sent to the data line by sending the effective data bit number of the image in units of 8 bits, multiplying the frequency a plurality of times, and sending a plurality of data.

FIG. 6 shows an example of a command in the case of the method shown in FIG.

Here, for example, when the transfer in the “YUV: 42216 [bit] format” which is one of the standards for the color information expression format is considered, the color components Cb (chroma blue), Y (yellow), Cr ( Chroma red), four 8 [bits] in the form of Cb, Y, Cr, Y
Assuming that 8 [bit] transfers are sequentially performed as a set of data, the values of Y, Cr, and Cb are all “0”.
0h "and" FFh "(where h indicates hexadecimal notation), and a set of the four data beginning with" FFh "is interpreted as a command. I do.

By doing so, for example, “FF” in FIG.
It can be seen that the first byte of the 4-byte data starting from "h" is a command data group starting from "FFh", and the remaining three bytes can freely add information. [Byte] = 3 × 8 [bit] ”and 2
There are 4 [bits], and therefore, by using these 24 [bits], it is possible to transmit "2 to the 24th power" control states.

However, regarding the remaining three bytes, if it is assumed that the fourth byte (XYh) represents the sync field information, the remaining 16 [bits], ie, the third and second bytes, are used. “2 × 8 [bit] = 16 [bi
t] ”is available, and control information of the“ 2 × 16 ”type can be exchanged for the remaining 16 bits. Therefore, the total of the third byte and the second byte can be exchanged. Utilizing 16 [bits] ″, it is possible to display an image data number in an input image from a camera or an image stored in the storage device 15 or transmit a request in combination with a command or the like.

In the example of FIG. 6, the third byte (the content is "Z
Zh ″), it is determined whether the image is from the camera or an image in the storage device 15.

The transmission pointer of an arbitrary shape is used to indicate the line of sight of the user, the position of the topic in the transmission image, and the like. The pointer information input from the transmission pointer control device 30 is communicated by the same means as described above. Then, the image display device 40 is arbitrarily formed by the reception pointer control device 31 on the other side.
Image on the screen.

Various methods of synthesizing pointer information are conceivable. For example, as shown in FIG. 7, a tag having the same level as an image and a sound is used to communicate the movement amounts of the X, Y, and Z axes. Is realized by:

As described above, the transmission pointer control device 30 is provided with a function of generating at least the pointer information indicating the desired position of the transmission image and the pointer information indicating the desired position of the partner image. After being synthesized by the device 22, the data is transmitted via the transmission / reception device 50, and the pointer information in the transmitted information is separated by the transmission / reception information separation device 23. The device 31 is configured to display a pointer image of a desired shape on the display screen of the image display device 40.

In this way, since the position of the viewpoint and the like can be understood, it is possible to communicate by grasping the attention point of the opponent and the trend of the admiration of the opponent as in a normal face-to-face conversation. .

If the transmission pointer control device 30 that generates pointer information outputs the difference between the pointer positions and the information calculated from the difference information as transmission pointer information, the code amount of the pointer information is reduced. Pointer information can be transmitted with a minimum.

FIG. 8 shows that the image of the other party is its own display screen shape 1
15 shows an example of a display method in a case where a similar shape to 15 is not made.
Display area 1 in image display device 40 of own system
15 and the transmission image 113 from the partner system are not similar, the display image is distorted as it is,
Although there is a concern that the display image may be distorted, the image display device 4
Since there is a display controller that controls display control at 0, the scaling function of the display controller element is used,
The display is controlled so that the transmission image 113 is always included in the display image of the user.

By doing so, the display is not different from the recognition of the transmission image of the other party, and the inconvenience of communication of the other party is reduced. The same applies when the display image shape of the user becomes 116.

In particular, by using this scaling function, if the image shape of the communication partner does not resemble the shape of the display image on the self side, scaling is performed so that the partner image matches the screen shape. It is possible to display an image on the display device without any trouble even if the screen shape is different.

Although various embodiments have been described above, the present invention is, in short, to take images of one another with the image pickup means and send them to the other party, and display them on the image display means provided on each side for interactive communication. In the system that can be used, an image pickup unit having a wide image pickup area is fixedly used at each fixed point as each image pickup unit, and a range in which transmission is permitted from the image region obtained by this image pickup unit is set. When an image area within a range and an image area of a predetermined size smaller than this is sent to the other party as an actual transmission image area, and the other party can view the received image on the monitor screen, By sending a command from the side, the transmission image area can be moved to an arbitrary position within the communication permission area and sent to the other party, thereby enabling communication. By moving the transmission image area to the area you want to see like moving the actual line of sight in the communication permission image permitted by the other party, the other Areas that are not desired are completely restricted, while images of other areas can be freely viewed by remote control. In other words, it is possible to obtain an image information transmission system that does not show a portion that the user does not want to see and that allows the other party to freely observe the other range.

The present invention is a technique for compensating for the drawback that in a conventional TV conference system or videophone system, it is not possible to pan a partner image according to one's own intention only by looking at a screen sent from the partner. In addition to being able to regulate the viewable area of the image and being able to pan freely within the range of the regulation, we will respect the will of each side and the other side and maintain privacy while maintaining In addition, it is possible to provide a technique that can freely pan a screen within a range permitted by the other party.

Conventionally, it has been necessary to manually adjust the position of the image pickup device (camera) with respect to the photographing range of the self-side image in the TV conference system. By using an imaging device capable of photographing the self-portrait photographing range, it is possible to reduce the fine adjustment of the imaging device, thereby realizing quick panning and the like while eliminating a mechanical drive part of the imaging device portion. Features can also be obtained.

In particular, in the system of the present invention, by selecting a transmission permission range wider than the image transmission image, the operated side cuts out a desired image from the screen and transmits it by panning. Can be widened, and an image can be selected in consideration of privacy. Also, since the other party can freely track the image of the self side, the self side can freely move as long as it is within the transmission permission range, and therefore, it is conscious whether or not it is captured by the camera as in the conventional case This eliminates the hassle of communication, and thus allows the communication to take the initiative.

Further, since the actual transmission image is narrower than the communication permission range, the communication capacity can be reduced.

Also, since the transmission permission range can be changed by the other party, dynamically changed, or only images prepared in advance can be permitted.
This allows for flexible communication.

Therefore, according to the present invention, the speakers can see each other's desired images, and the privacy of the sender can be protected, as in the communication of meeting at the same place and exchanging faces with each other. It is possible to realize the provision of a communication system that enables the communication.

The present invention is not limited to the above-described embodiment, but can be implemented with various modifications.

[0113]

As described in detail above, according to the present invention,
It is possible to provide an image communication system in which, in the image of the other party, an area that the other party does not want to show to others is not shown, and the other range can be freely observed by the other party.

Therefore, according to the present invention, interlocutors can see each other's desired images, as in communication in which they meet at the same place and face-to-face with each other, and privacy at the sender side can be improved. Establish communication system technology that can be secured.

Further, by combining the image with a previously prepared image, for example, even in a scene where communication with the other party is not assumed in advance in the private time, the prepared image can be combined. This makes it possible to respond flexibly, reduces the mental resistance at the time of the initial introduction of communication using images, and allows interlocutors to communicate with each other without having a mechanical drive in the image input unit. Since the user can see the desired image and realizes the function to protect the privacy of the sender, the system can be simplified and the price can be reduced. It is possible to transfer an arbitrary high-resolution video, and so on.

[Brief description of the drawings]

FIG. 1 is a block diagram showing an embodiment of the system of the present invention.

FIG. 2 is a diagram for explaining the present invention, and is a diagram for explaining an example of an image range in the embodiment of the present invention.

FIG. 3 is a diagram for explaining the present invention, and is a diagram for explaining high-resolution image transfer in the embodiment of the present invention.

FIG. 4 is a diagram for explaining the present invention, showing an example of mounting of an image display device according to the present invention.

FIG. 5 is a diagram for explaining the present invention, showing another example of mounting of the image display device in the present invention.

FIG. 6 is a diagram for explaining the present invention, showing an example of a data format containing a command used in the system of the present invention.

FIG. 7 is a diagram for explaining the present invention, showing an example of a data format including pointer information used in the system of the present invention.

FIG. 8 is a diagram showing an example of an image display.

[Description of Signs] 10: Image input device, 11: Input image selection device, 12: Transmission permitted image selection device, 13: Transmission image selection device, 14: Selected position control device, 15: Storage device, 20: Transmission image control Apparatus, 21: partner image control apparatus, 22: transmission information composition apparatus, 23: reception information separation apparatus, 30: transmission pointer control apparatus, 31: reception pointer control apparatus, 40: image display apparatus, 50: transmission / reception apparatus, 51 ... Transmission path, 110 ... Lens input image, 111 ... Input image, 112 ... Transmission permitted image, 113 ... Transmission image, 114 ... Combined transmission image 200 ... Self side image 201 ... Other side image 210 ... Self side image data line Reference numeral 211: partner image data line 220: image control line 230: data bus 115: display image shape 116: display image shape (2)

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Junichi Yoshizawa 2-9-9 Suehirocho, Ome City, Tokyo Inside the Toshiba Ome Plant (72) Inventor Sou Osawa 2-9-9 Suehirocho, Ome City, Tokyo Stock Company 5C064 AA01 AA02 AC01 AC02 AC07 AC11 AC13 AC22 AD06 5C075 AA90 CA03 CD13 5C076 AA40 5K101 KK04 KK07 NN06 NN14

Claims (15)

[Claims] 1. An image acquiring means for acquiring an image, a first selecting means for selecting an arbitrary transmission permitted range in the image area from the image acquired by the image acquiring means, and Second selecting means for selecting an arbitrary transmission image which is within the transmission permission range selected and which is actually transmitted, and means for transmitting the transmission image selected by the second selecting means. An image information transmission device characterized by the above-mentioned. 2. An image acquiring means for acquiring an image, a first selecting means for selecting an arbitrary range in the image acquired by the image acquiring means, and an image acquiring means for selecting an arbitrary area in the image selected by the first selecting means. Second selecting means for selecting a desired range as a transmission permitted range; and third selecting means for selecting an arbitrary transmission image to be actually transmitted within the transmission permitted range selected by the second selecting means. And a means for transmitting the transmission image selected by the third selection means. 3. A method according to claim 1, further comprising the step of: selecting an image from a plurality of images prepared in advance from the storage unit, or selecting a plurality of images, dividing the layers and superimposing the images to obtain a transmission-permitted range image. 2 selecting means, a second selecting means for selecting an arbitrary transmission image actually transmitted within the transmission permission range obtained thereby, and a transmission image selected by the second selecting means. Transmitting means for transmitting the image information. 4. An image acquiring means for acquiring an image, a first selecting means for selecting an arbitrary captured image transmission permitted range in the image area from the image acquired by the image acquiring means, and a storage means. A second selection of selecting a plurality of images prepared in advance from the storage unit and a transmission-permitted image from the image acquisition unit, or selecting a plurality of images and dividing the layers to superimpose the images to obtain a transmission-permitted range image Means, a third selection means for selecting an arbitrary transmission image to be actually transmitted within the transmission permitted range image, and means for transmitting the transmission image selected by the third selection means. An image information transmission device, comprising: 5. A transmission image range to be actually transmitted is divided into a plurality of transmission image ranges according to a predetermined transmittable image size, and the divided images are transmitted in an arbitrary order. 5. The image information transmission device according to any one of claims 4 to 4. 6. A transmission image range to be actually transmitted is divided into a plurality of transmission image ranges according to a predetermined transmittable image size, and the divided images are transmitted in an arbitrary order. 5. The image information transmitting apparatus according to claim 1, further comprising means for dynamically changing a transmission image range size. 7. An image acquiring means for taking in an image, a first selecting means for selecting an arbitrary permitted image transmission range within the image area from the image taken by the image acquiring means, and a storage means. A second method of selecting a plurality of images prepared in advance from the storage unit and a transmission-permitted image from the image acquisition unit, or selecting a plurality of images and dividing the layers to superimpose the images to obtain a transmission-permitted range image. Selecting means; third selecting means for selecting an arbitrary transmission image to be actually transmitted within the transmission permitted range image; and means for transmitting the transmission image selected by the third selecting means. A communication is performed by using a first device provided and a second device configured using at least one unit of the first device, and only the second selection unit or only the third selection unit Or both For the selecting means, the transmitted image is an image of an adjacent area portion within the transmission permitted range, or is a transmission permitted image in another storage means, or an image from an image acquiring means for capturing an image. Further, an image information transmission device and an image information transmission system that can be dynamically operated from a partner device. 8. The image information transmitting apparatus according to claim 1, further comprising a selection unit that selects an initial transmission permission range based on additional information of a communication partner. Image information transmission system. 9. A system for communicating between said first device and said second device, wherein a common image data line is used for image switching between a self image and a partner image in each device, and the self image and the partner image are switched. 8. The image information transmitting apparatus according to claim 1, wherein the switching control is performed using the same control line. 10. A system for performing communication between said first device and said second device, wherein a common image data line is used for image switching between a self image and a partner image in each device, and the self image and the partner image are switched. 8. The switching control of (1) is also transmitted as a command using the same data line.
An image information transmission device according to the item. 11. A system for performing communication between said first device and said second device, wherein when the transmission image shape of a communication partner does not resemble its own display image shape, the shape of the partner image must be changed. The image information transmitting apparatus according to claim 1, further comprising a scaling function for scaling to include the image information. 12. A system for performing communication between said first device and said second device, wherein when a shape of a transmission image of a communication partner does not resemble a shape of a display image of the communication partner, the display image shape of the partner is changed. 8. The apparatus according to claim 1, further comprising a function of adding a prepared image to a transmission image of the user to make the display image similar to the display shape of the other party. The image information transmission device according to the above. 13. A system for communicating between a first image information transmitting device and a second image information transmitting device, wherein a pointer tag is provided in transmission image data, and transmission is performed using the same communication path as the transmission image. An image information transmitting apparatus and an image information transmitting system for transmitting a pointer indicating a part of an image, a pointer indicating a part of a partner image, or both pointer information. 14. A system for communicating between a first image information transmitting device and a second image information transmitting device, wherein a pointer indicating a part of a transmission image, a pointer indicating a part of a partner image, or An image information transmission device characterized by displaying both pointers. 15. The image information transmitting apparatus and the image information transmitting system according to claim 13, wherein a difference between pointer positions and information calculated from the difference information are transmitted in transmitting the pointer information. .