JP2002051315A - Data transmitting method and data transmitter, and data transmitting system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a data transmitting method and a data transmitter, and a data transmitting system in which the image quality of a transmitted image and its own image can be held at the substantially same level. SOLUTION: In the data transmitting system where image data is transmitted while being contained in an image frame of preset size at the time of communication among a plurality of terminals 50-1, 50-2, an image including a target image to be transmitted is picked up, a target image to be transmitted from the picked up image is regulated to fully fill the image frame and then the regulated image data in the image frame is transmitted while being compressed.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for transmitting data including an image when performing communication, streaming, and the like between a plurality of terminals, a portable telephone, and a portable telephone.
The present invention relates to a data transmission device such as a DA, a TV telephone terminal, and a PC terminal, and a data transmission system.

[0002]

2. Description of the Related Art As a communication service for performing communication, streaming, and the like between a plurality of terminals, visual communication as a data transmission system by a mobile terminal (MT) such as a mobile phone as shown in FIG. Service is being performed.

[0003] The visual communication service system 1 includes a mobile terminal (hereinafter simply referred to as a terminal).
By exchanging the voice and the image and exchanging the signals through the transmission path 3 between the terminal 2-1 and the terminal 2-2, it is possible to enjoy a conversation while seeing the face of the other party.

The terminals 2-1 and 2-2 are shown in FIG.
As shown in the figure, a main body case 21 and an operation unit 22 provided below the main body case 21 including a numeric keypad and the like for inputting a telephone number for a call function and inputting a predetermined operation instruction, It has a display monitor 23 provided above the operation unit 22, a camera 24 provided above the display monitor 23 (above the main body case 21), and an antenna 25.

[0005] The transmission path 3 is a wireless base station (MBS: Mobile
Base Station) 31-1, 31-2, Mobile Switching Center (M
SC: Mobile Switching Center) 32-1, 32-
2. Home location register (HLR: Home Locat)
Gateway Mobile Switching Center (GMSC: Ga)
teway Mobile Switching Center) 33-1,33-
2 and a network 35.

[0006] The radio base stations 31-1 and 31-2 are connected to the terminal 2.
-1, 2-2. These base stations 31-1,
The signal exchanged at 31-2 is sent to the mobile switching centers 32-1 and 32-2 located at a higher level, and further transmitted via the higher-level gateway mobile switching stations 33-1 and 33-2 to the backbone line. It is connected to the net 35.

[0007]

However, when transmitting an image to such a network 35, the amount of information that can flow through the network 35 is limited by the bandwidth of the network 35.
Therefore, for example, a user who is having a conversation with the terminal 2-1 can monitor his / her own image by the camera 24 with high image quality, as shown by an image 23a in FIG. Is compressed by the terminal 2-1.
When transmitted to the terminal 2-2, the terminal 2-2
The image quality of the video sent from 1 is extremely deteriorated like the image indicated by 23b. Similarly, the image transmitted from the terminal 2-2 to the terminal 2-1 is extremely deteriorated at the terminal 2-1.

As described above, in the conventional image transmission system, the transmitted image is extremely deteriorated, the difference between the transmitted image and the monitored image is too large, and at the same time, there is a sense of discomfort. You will feel stress without grasping the other person's expression.

Further, in the terminals 2-1 and 2-2, since the camera 24 is located above the monitor display monitor 23 of the terminal, the user looks at the display monitor 23 of the terminal, and My eyes are pointing down,
A disadvantage arises in that it is not possible to establish a conversation with a glance.

The present invention has been made in view of the above circumstances, and a first object of the present invention is to provide a data transmission method and a data transmission apparatus capable of maintaining the quality of a transmitted image and that of an own image at substantially the same level. Another object of the present invention is to provide a data transmission system.

A second object of the present invention is to provide a data transmission method and apparatus capable of realizing communication in a state where the eyes are more closely aligned, and a data transmission system.

[0012]

In order to achieve the above object, the present invention relates to a data transmission method for transmitting at least image data in an image frame of a predetermined size, comprising: An image including an image is captured, the target image to be transmitted from the captured image is adjusted so as to substantially fill the image frame, and the image data in the adjusted image frame is compressed and transmitted.

The present invention also relates to a data transmission method for transmitting at least image data, wherein an image including a target image to be transmitted is captured, and a target image to be transmitted is determined from a captured image by a predetermined predetermined method. The image data is adjusted so as to have a size, and the image data in the adjusted image frame is compressed and transmitted.

Further, according to the method of the present invention, when adjusting the target image to fill the image frame or when adjusting the target image to a predetermined size, the image of the characteristic portion of the image to be transmitted is adjusted. , And the image is tracked by cutting out an image around the best matching part.

In the method of the present invention, when adjusting the target image to fill the image frame or when adjusting the target image to a predetermined size, the image of the characteristic portion of the transmission image is used. Then, the distance of the characteristic portion is calculated, and thereby, enlargement, reduction, and tracking of the image are performed.

Further, according to the method of the present invention, when detecting a characteristic portion of an image, the state of the image is locked in a desired image state while monitoring the state in which the transmitted image is captured, and the image of the locked image is locked. The vicinity of the center is defined as a characteristic portion of the image.

In the method of the present invention, when a characteristic portion of an image is detected, a state in which the transmitted image is captured is displayed, a predetermined portion is designated on the display screen, and the characteristic of the image is centered on the designated point. Part.

In the method of the present invention, when the target image is locked by utilizing the characteristic portion of the image, the image is captured at a frame rate equal to or higher than that of the transmitted image, and the tracking operation is performed. Is performed, and a reference image serving as a reference for tracking is updated at a rate equal to or higher than the transmission frame rate.

Further, according to the method of the present invention, the state in which the transmitted image is captured is displayed, the range to be transmitted is specified on the display screen, and the range for transmitting the image is determined.

The present invention is a data transmission method for transmitting at least image data while displaying at least image data of a user on a display screen among a plurality of terminals and displaying received image data on the display screen. And capturing an image including a target image of the user to be transmitted from a substantially central portion of the display screen.

According to the present invention, at least a plurality of terminals display at least image data of a user on a display screen, display received image data from the other party on the display screen, and display a content which is a common topic. A data transmission method for transmitting at least image data while displaying on a display screen, wherein an image including a target image of a user to be transmitted is captured from a substantially central portion of the display screen, and a display screen, which is an imaging portion, is captured. With the center at the center, the image data is displayed on one side of the upper or lower side of the display screen, and the content is displayed on the other side.

That is, in the method of the present invention, on the display screen, image data is displayed above the content or image data is displayed below the content.

Also, the present invention is a data transmission apparatus for transmitting at least image data in an image frame of a preset size, wherein the imaging means captures an image including a target image to be transmitted, A first circuit for adjusting a target image to be transmitted from an image captured by the image capturing means to substantially fill the image frame, and a second circuit for compressing and transmitting image data in the adjusted image frame And

According to the present invention, there is provided a data transmitting apparatus for transmitting at least image data, wherein an image pickup means for picking up an image including a target image to be transmitted, and a target image to be transmitted based on the picked-up image are determined in advance. And a second circuit for compressing and transmitting image data within the adjusted image frame.

In the device of the present invention, the first circuit transmits the signal when adjusting the target image to fill the image frame or when adjusting the target image to a predetermined size. Image matching is performed by using the image of the characteristic portion of the power image, and an image around the best matching portion is cut out to perform image tracking.

In the apparatus according to the present invention, the first circuit adjusts the target image to fill the image frame, or adjusts the target image to a predetermined size. By using the image of the characteristic portion, the distance of the characteristic portion is calculated, and accordingly, enlargement, reduction, and tracking of the image are performed.

In the apparatus of the present invention, a display means for displaying image data, and an image displayed on the display means,
A lock unit capable of locking the image in a desired image state, wherein the first circuit detects the vicinity of the center of the image when locked by the lock unit when detecting a characteristic portion of the image. The characteristic part of.

Further, in the apparatus of the present invention, it is possible to display image data and include a display means including a pointer capable of designating a predetermined location, and to designate a predetermined location of an image displayed on the display means with the pointer. Designation means, and
When detecting the characteristic part of the image, the first circuit sets the characteristic part of the image around the point specified by the specifying means.

Further, in the device of the present invention, when the first circuit locks the target image using the characteristic portion of the image, the first circuit has the same or higher frame rate as the transmitted image. , An image is captured, a tracking operation is performed, and a reference image serving as a reference for tracking is updated at a rate equal to or higher than the transmission frame rate.

Further, in the apparatus of the present invention, it is possible to display image data and include a display means including a pointer capable of designating a predetermined location, and to designate a predetermined location of an image displayed on the display means with the pointer. Designation means, and
The first circuit determines a range specified by the specifying unit as a range for transmitting an image.

The present invention is also a data transmission system for transmitting at least image data while displaying at least image data of a user on a display screen among a plurality of terminals and displaying received image data on the display screen. An image capturing unit that captures an image including a target image to be transmitted, a first circuit that adjusts the target image to be transmitted from the image captured by the image capturing unit so that the target image substantially fills an image frame, A second circuit for compressing and transmitting the image data in the adjusted image frame.

Further, the present invention is a data transmission system for transmitting at least image data while displaying at least a user's image data on a display screen between a plurality of terminals and displaying the received image data on the display screen. An image capturing unit that captures an image including a target image to be transmitted, a first circuit that adjusts the target image to be transmitted from the captured image to have a predetermined size, A second circuit for compressing and transmitting the image data in the adjusted image frame.

Further, the present invention is a data transmission system for transmitting at least image data while displaying at least image data of a user on a display screen among a plurality of terminals and displaying received image data on the display screen. An image pickup means for picking up an image including a target image of a user to be transmitted is arranged substantially at the center of the display screen.

According to the present invention, at least a plurality of terminals display at least image data of a user on a display screen, display received image data from the other party on the display screen, and display a content which is a common topic. What is claimed is: 1. A data transmission system for transmitting at least image data while displaying it on a display screen, comprising: an imaging unit disposed at a substantially central portion of the display screen for capturing an image including a target image of a user to be transmitted; And display means for displaying the image data on one side of the upper or lower side of the display screen with the center substantially at the center of the display screen being the center, and displaying the content on the other side.

According to the present invention, the face image is tracked in accordance with the enlargement, reduction, and movement so that, for example, the user's face image as the target image fills the image frame. Then, only the face image is cut out, and this image is compressed and transmitted.

Further, according to the present invention, for example, tracking is performed in accordance with the compression, reduction and movement so that the size of the image of the user who has started transmission is always the same. As a result, a person image of the same size is extracted, and this image is compressed and transmitted.

Further, while the user looks at the monitor screen,
By configuring the terminal to lock (keep) the state of the image in the terminal in a desired state, the terminal performs tracking, enlargement, reduction, and clipping of the image in accordance with the user's instruction. Further, in order to continue this locked state, at the time of lock start, for example, characteristic image portions (eg, eyes, nose, mouth, ears, and facial contour organ portions) are extracted from this image, This portion is used as a reference to enlarge, reduce, and track the image. For example, the distance between both eyes is detected, and from this distance,
The enlargement / reduction values and the cutout position of the image are detected.

Further, all the images input from the camera as the image pickup means are output to the user's monitor image, and the user cuts out this range by designating the capture range of the image. The image is reduced, enlarged, and tracked so as to have the same size as. In addition, by using a pointer that can be operated by the user on the monitor screen, the user can specify a characteristic image at the time of lock start, thereby reducing or enlarging the image.
Tracking is performed. Furthermore, in order to improve the tracking accuracy, an image is captured at a frame rate equal to or higher than the transmitted image, a tracking operation is performed, and a rate equal to or higher than the transmission frame rate is performed. Then, a reference image serving as a reference for tracking is updated.

Further, an image pickup means is embedded in the display screen. Thereby, the eyes of the images transmitted to each other can be matched.

Further, according to the present invention, for example, a face image and a topic content are arranged vertically, and an image pickup means is arranged in the vicinity of the face image. You can tell that you are
If you look at each other's faces on the monitor, you can realize virtual reality communication that your eyes are aligned.

[0041]

FIG. 1 is a system configuration diagram showing an embodiment of a visual communication service system as a data transmission system according to the present invention.

The visual communication service system 100 transmits a signal between a plurality of terminals, for example, a mobile terminal (hereinafter simply referred to as a terminal) 50-1 and a terminal 50-2 by compressing voice and images and transmitting each other. By exchanging through the road 60, it is possible to enjoy a conversation while looking at the other person's face.

FIG. 2 is an external view showing an example of the configuration of the terminals 50-1 and 50-2. As shown in FIG. 2, the terminals 50-1 and 50-2 are used to input a main body case 51 and a telephone number provided below the main body case 51 for a call function, An operation unit 52 including ten keys for inputting operation instructions, a display monitor 53 as a monitor provided above the operation unit 52, and a display monitor 53
A camera module 54 having a lens portion disposed in a substantially central region of the camera module, an antenna 55, and a GU.
And a switch 56 as a lock means or a designation means having a function as I. In FIG. 2, a microphone unit and a speaker unit are omitted, and a specific circuit configuration of a terminal as a data transmission device will be described later in detail.

As described above, in the display monitor 53, the lens portion of the camera module 54 is disposed in a substantially central area thereof, and as shown in FIG. A predetermined content image by an application service provider (ASP) is displayed. In the example of FIG. 2, the face image is displayed on the upper side and the content image is displayed on the lower side.
It is also possible to display the content image on the upper side and the face image on the lower side.

The transmission line 60 is a wireless base station (MBS: Mobi
le Base Station) 61-1 and 61-2, Mobile Switching Center (MSC) 62-1 and 62
-2, Home Location Register (HLR: Home Loc)
Gateway Mobile Switching Center (GMSC:
Gateway Mobile Switching Center) 63-1, 63-
2, and the network 64.

The radio base stations 61-1 and 61-2 are connected to the terminal 5
Communication with 0-1 and 50-2 is performed. These base stations 61-
Signals exchanged at 1, 61-2 are sent to the mobile switching centers 62-1 and 62-2 located at a higher level, and further,
It is connected to the backbone line network 64 via the higher-order gateway mobile switching offices 63-1 and 63-2. Further, a gateway 64 (GW) is connected to the line network 64, and an application service provider (ASP) 66 that enables content distribution is connected to the gateway 65. Thereby, a plurality of contents CON
T launches an application service provider (ASP) 66 provided with these contents CONT.
Can be output to the terminals 50-1 and 50-2 through the gateway station 65.

Hereinafter, a terminal 50 (terminal 50-) serving as a data transmission device according to the present invention, which is applied to the visual communication service system 100 as described above.
The specific circuit configuration of the signal processing system of (1, 50-2) will be described step by step as first, second, and third embodiments with reference to the drawings.

First Embodiment FIG. 3 is a circuit diagram showing a first embodiment of a signal processing system of a terminal as a data transmission device according to the present invention.

As shown in FIG. 3, the signal processing system 500 includes a microphone 501, a camera circuit 502, a pre-processing circuit 503 as a first circuit, and a video compression encoder 5
04, audio compression encoder 505, multiplexer (MU)
X) 506, transmission circuit 507, reception circuit 508, demultiplexer (DeMUX) 509, video decoder 51
0, an audio decoder 511, an image synthesizing device 512, a display circuit 513, a speaker 514, a GUI interface (I
/ F) circuit 515 and CPU 516.
Note that the video compression encoder 504 and the audio compression encoder 5
05, multiplexer (MUX) 506, transmission circuit 50
7 constitutes a second circuit.

The microphone 501 converts a voice uttered by a user or the like into an electric signal and outputs the electric signal to the voice compression encoder 505.

The camera circuit 502 includes the camera module 5
For example, an image of the upper body part of the user, which is imaged through the lens 4 and converted from an optical signal to an electric signal, is subjected to predetermined processing, and is output to the preprocessing circuit 503 as a digital image signal S502.

Upon receiving the image signal S502, the preprocessing circuit 503 extracts an image to be transmitted (hereinafter, referred to as a target image) from the captured image of the upper body of the user,
An image signal adjusted so as to fill the image frame of the image to be transmitted is generated and output to the video compression encoder 504 and the image synthesizing device 512. The pre-processing circuit 503 performs matching using, for example, an image of a characteristic portion of the transmission image as a process of fully fitting the target image to the image frame, and focuses the image around the most-matched image on the basis of the best matching. The clipping enables tracking of the image. Here, “full image frame” means that the target image is filled to the extent that the background is not known. In particular, in an image transmission apparatus for communication, the target image is often a face image.

Hereinafter, the processing of the pre-processing circuit 503 will be described in more detail.

FIG. 4 is a circuit diagram showing a specific configuration example of a main part of the preprocessing circuit according to the first embodiment. Also,
FIG. 5 is a schematic diagram for explaining the tracking processing in the preprocessing circuit.

The pre-processing circuit 503, as shown in FIG.
Search range extraction unit 5031, binarization unit 5032, pattern matching unit 5033, reference memory unit 503
4 and a rectangular area extraction unit 5035.

The search range extracting unit 5031 stores the previous image P in the reference memory unit 5034 shown in FIG.
5 based on the coordinate information of the IM reference image RIM.
The search range portion SRG shown in (B) is extracted.

The binarizing section 5032 includes a search range extracting section 5
The luminance signal of the search range data extracted in 031 is binarized and supplied to the pattern matching unit 5033.

The pattern matching unit 5033 converts the binarized data of the luminance signal of the search range data by the binarizing unit 5032 and the reference image of the previous image PIM stored in the reference memory unit 5034 shown in FIG. A pattern matching process of the luminance information with the binarized data of the RIM is performed, and the result is referred to as a rectangular area extraction unit 50.
35. This pattern matching process is performed as shown in FIG.
(B) employs the image data MMT at the position with the highest matching degree in the search range portion SRG shown in FIG.
This is a tracking result in IM. In addition, the pattern matching unit 5033 stores the image data MMT at the position with the highest matching degree, which is the tracking result at this time, in the reference memory unit 5034 as a reference image in the next image.

The rectangular area extracting section 5035 cuts out and outputs a rectangular area including the periphery of the tracking result MMT obtained by the pattern matching section 5033.

The preprocessing circuit 503 having such a configuration
Then, tracking of an input image is performed in order to maintain a desired image state transmitted by the user (hereinafter, referred to as an initial reference image). The tracking is performed at a frame rate equal to or higher than the transmission frame rate, and at the same time, a reference image serving as a tracking reference is sequentially updated.

The tracking in the pre-processing circuit 503 is performed by the CPU 516 from the initial reference image information (FIG. 6).
(To be described later with reference to FIG. 7 and FIG. 7).

First, the input image data (current image) CIM newly input by the camera circuit 502 as shown in FIG. 5B is stored in the reference memory unit 5034 as the reference image RIM of the previous image PIM. The search range extracting unit 5031 extracts a search range portion SRG based on the coordinate information of. Next, the luminance signal of the extracted data is binarized by a binarization unit 5032, and then a reference memory unit 503 by a pattern matching unit 5033.
4, a pattern matching process of the luminance information with the binarized data of the reference image RIM stored in the reference image RIM is performed. The image data MMT at the position with the highest coincidence, which is the tracking result at this time, is stored and held in the reference memory unit 5034 as a reference image in the next image. Then, the rectangular region including the periphery of the tracking result MMT obtained by the pattern matching unit 5033 is cut out and output by the rectangular region extraction unit 5035.

Next, the first and second methods for determining the initial reference image will be described with reference to FIGS.

The first determination method is determined by monitoring a transmission image size frame. Specifically, as shown in FIG. 6A, the transmission image size frame FR is set in the initial image IIM, and the image in the frame FR is output as the monitor image MNTa. As shown in ()), the display is displayed on the display monitor 53 by the display circuit 513 in FIG. When the user sees the monitor output and adjusts to the desired image state, for example, the user presses the switch 56 to transmit an image lock instruction, thereby making it possible to perform the operation shown in FIG.
As shown in (C), an initial reference image IRI is determined.

On the other hand, the second determination method is determined by monitoring a reduced image. Specifically, FIG.
As shown in FIG. 7A, the initial image IIM is reduced, and FIG.
Output as a monitor image MNTb as shown in (B),
The user designates a desired reference center and a display area (image capture range) using a pointer POI or the like, thereby determining an initial reference image IRI. The movement instruction of the pointer POI on the display monitor 53 is performed by a switch 56, for example, a ball switch. That is, by turning the switch 56 in a desired direction, the pointer POI is moved up, down, left, right, or obliquely.

As described above, the initial reference image information (coordinate information, luminance information) determined by the first or second method
Are the GUI interface circuit 515 to the CPU 516
The reference memory unit 50 of the preprocessing circuit 503
34.

In FIG. 3, the video compression encoder 504 compresses and encodes the image data to be transmitted extracted by the pre-processing circuit 503 and outputs the image data to the multiplexer 506.

The audio compression encoder 505 compresses and encodes the audio data from the microphone 501 and outputs it to the multiplexer 506.

The multiplexer 506 multiplexes (multiplexes) the image data to be transmitted which has been compression-encoded by the video compression encoder 504 and the audio stream which has been compression-encoded by the audio compression encoder 505. Output to the transmission circuit 507.

The transmission circuit 507 includes a multiplexer 506
Performs conversion and modulation of a predetermined transmission protocol on image and audio data multiplexed by the radio base station (MB).
S) Transmit to 61-1 or 61-2.

The receiving circuit 508 receives a transmission signal transmitted from the communication partner and transmitted through the transmission path 60, demodulates the signal, decompresses the transmission protocol, and outputs the result to the demultiplexer 509.

The demultiplexer 509 is connected to the receiving circuit 50.
8, the image data and the audio data are separated, and the image data stream is supplied to the video decoder 510 and the audio data stream is supplied to the audio decoder 511.

The video decoder 510 decodes the image data stream separated by the demultiplexer 509 and outputs it to the image synthesizer 512.

The audio decoder 511 decodes the audio data stream separated by the demultiplexer 509, supplies it to the speaker 514, and reproduces it.

The image synthesizing device 512 includes a pre-processing circuit 503
Face image data of a user using a terminal according to the present invention and face image data of the other party by the video decoder 510 or an application service provider (A
SP) 66 and synthesizes a predetermined content image and supplies it to the display circuit 513.

The display circuit 513 receives the synthesized image from the image synthesizing device 512, and as shown in FIGS. 2 and 3,
On the display monitor 53 of the terminal 50-1 (or 50-2),
The face image of the user and the transmitted face image of the other party are displayed in parallel above the lens portion of the camera module 54, for example, and a predetermined content image is displayed below.

The GUI interface circuit 515 includes the switch 56, and the CPU 516 controls the entire terminal 50-1 (or 50-2) according to an instruction from the user, such as shifting to an image lock state and switching monitor display. I do.

Next, the operation of the terminal 50 in FIG. 3 will be described.
For example, the image of the upper body of the user is stored in the camera module 54.
An image is taken through a lens portion of the camera circuit 502, is converted from an optical signal into an electric signal in a camera circuit 502, and is input to a preprocessing circuit 503 as a digital image signal S502. The image data of the upper body of the user is supplied to the display circuit 513 through the image synthesizing device 512, and is displayed on the display monitor 5 of the terminal.
3 displays an initial image IIM which is an image of the upper body of the user. The user monitors, for example, the transmission image size frame, which is the first determination method, while watching the image of the upper body of the user, that is, the initial image IIM displayed on the display monitor 53, to thereby display the initial reference image I.
Determine RI. That is, the transmission image size frame FR is set in the initial image IIM, the image in the frame FR is output as the monitor image MNTa, and displayed on the display monitor 53 by the display circuit 513. Then, when the user views the monitor output and adjusts to the desired image state, the user presses the switch 56 to transmit an image lock instruction to the CPU 516, thereby determining the initial reference image IRI.

When the initial reference image information is transmitted from the CPU 516, the tracking in the preprocessing circuit 503 is started. In this state, the camera circuit 50
2. Input image data (current image) C newly input by
The IM is searched by the search range extraction unit 5031 based on the coordinate information of the reference image RIM of the previous image PIM stored in the reference memory unit 5034.
Is extracted. Next, the luminance signal of the extracted data is 2
After being binarized by the binarizing unit 5032, the pattern matching unit 5033 performs a pattern matching process of luminance information with binarized data of the reference image RIM stored in the reference memory unit 5034. The processing result is stored and held in the reference memory unit 5034 as a reference image in the next image.
Then, a rectangular area around the tracking result MMT obtained by the pattern matching section 5033 is cut out by the rectangular area extraction section 5035 and supplied to the video compression encoder 504 and the image synthesis apparatus 512.

In such a state, the voice fetched by the microphone 501 is converted into a digital signal and then compressed by the voice compression encoder 505. Then, as described above, the image data from the pre-processing circuit 503 is compression-encoded by the video compression encoder 504, multiplexed with the compressed audio stream by the multiplexer 506, and supplied to the transmission circuit 507. . And
The transmission circuit 507 receives the multiplexed data, converts and modulates the transmission protocol, transmits the multiplexed data to, for example, the wireless base station 61-1 and transmits the multiplexed data to the other party, for example, the terminal 50-2. Can be

On the other hand, the transmission signal arriving from the other party is
After being taken into the receiving circuit 508, demodulation and decompression of the transmission protocol are performed, the signal is separated by the demultiplexer 509 into audio and video streams. The separated audio stream is decoded by the audio decoder 511, sent to the speaker 514, and reproduced. After the image stream is decoded by the video decoder 510, the image
In step 2, the image is combined with the user's own image by the preprocessing circuit 503. The composite image data is supplied to the display circuit 513 and displayed on the display monitor 53. At this time, the face image of the user and the transmitted face image of the other party are displayed in parallel on the display monitor 53, for example, above the lens portion of the camera module 54, and the predetermined content image is displayed below. Is displayed.

As described above, according to the first embodiment, when communication or the like is performed between the plurality of terminals 50-1 and 50-2, the target image fills the image frame of the image to be transmitted. After the adjustment, the data is compressed and transmitted, so that it is possible to reduce the amount of information as much as possible while transmitting the expression of the partner most necessary for communication. In addition, since the face image is matched to the entire image frame, transmission of the background image with high resolution can be naturally deleted, and the amount of information after compression can be greatly reduced.
This makes it possible to transmit high-quality images. Furthermore, the frame correlation of an image can be greatly improved, and the amount of information can be significantly reduced by using a compression technique for performing inter-frame compression such as MPEG as an image compression method. This makes it possible to transmit high-quality images.

Further, the state of the image to be transmitted by the user (the state of the image lock) can be automatically kept,
Further, it assists in detecting a characteristic portion of an image, and enables tracking of a target image such as a human image. As a result, the correlation in the time axis direction of the image is increased, and the amount of information after compression can be reduced. As a result, a high-quality image can be transmitted.

Further, since the face image and the topic content are arranged vertically on the display monitor 53, and the lens portion of the camera module 54 is arranged near the middle thereof, it is possible to realize a state in which the eyes of each other are better. In addition, we can tell each other how we are talking while watching the video of the topic,
In addition, there is an advantage that virtual reality communication can be realized, that is, the eyes are aligned when the faces of each other are viewed on the monitor. Further, an application service provider 66 that enables content distribution is set up in the transmission path 60, and the content CONT prepared by the application service provider 66 is transferred to the gateway station 65.
, The same video can be output to the terminals 50-1 and 50-2.

Second Embodiment FIG. 8 is a circuit diagram showing a second embodiment of a signal processing system of a terminal as a data transmission device according to the present invention.

The difference between the signal processing system of the terminal according to the second embodiment and the signal processing system according to the first embodiment described above lies in the processing of the preprocessing circuit 503A and the CPU 516A. Specifically, when communicating, etc.,
A face organ is detected from the input image, the size of the face is calculated from this information, the image is scaled up and down so that the face image is fully displayed on the monitor, and the face image is cut out. In this case, the distance of the characteristic portion is calculated by using the image of the characteristic portion of the image to be transmitted, and thereby the enlargement, reduction, and tracking of the image are performed. In particular, in the case of a face image, for example, an eye portion is taken out of this characteristic portion, and the enlargement / reduction values of the image are determined from the distance between the two eyes so as to be the same distance as the reference image, and tracking is performed. . In addition, as a characteristic portion, at least a part of an organ portion of an eye, a nose, a mouth, an ear, and a face contour is used.
Perform reduction and tracking.

The other structure is the same as the structure shown in FIG. Therefore, in FIG. 8, the same components as those in FIG. 3 are denoted by the same reference numerals. That is, as shown in FIG. 8, the signal processing system 500A of the terminal includes a microphone 501, a camera circuit 502, a preprocessing circuit 503A, a video compression encoder 504, an audio compression encoder 505, a multiplexer (MUX) 506, a transmission A circuit 507, a receiving circuit 508, a demultiplexer (DeMUX) 509,
Video decoder 510, audio decoder 511, image synthesizer 512, display circuit 513, speaker 514, GUI
Interface (I / F) circuit 515 and CPU 5
16A.

Thus, preprocessing circuits 503A and CP
Since the function of U516A is different from the first embodiment,
Here, only the functions of the preprocessing circuit 503A and the CPU 516A will be described in detail, and description of the other components will be omitted.

FIG. 9 is a circuit diagram showing a specific configuration example of a main part of a preprocessing circuit according to the second embodiment. In addition,
In FIG. 9, for easy understanding, the CPU 516
A is also illustrated.

As shown in FIG. 9, the pre-processing circuit 503A includes a face organ position detecting section 5036 and an image scaling section 503.
7, an image cutout unit 5038, and a reference face image storage unit 5039.

The face organ position detecting section 5036
Within the face organ detection search range, which is information on the initial range of the face position according to 6A, the face organs, for example, both eyes, nose, mouth,
Using the brightness and color information, the position of eyebrows, ears, face outline, etc.
The position information S5036 of the detected face organ is detected by the CPU.
516A.

The image enlargement / reduction unit 5037 includes a CPU 516
Based on the enlargement / reduction ratio calculated in A, the input image is enlarged or reduced and supplied to the image cutout unit 5038.

The image cut-out unit 5038 is provided with a CPU 516
A face image of an area displayed in the monitor screen is cut out from the enlarged or reduced image based on the face position information (coordinates and the like) already sent from A.

The reference face image storage unit 5039 stores
The cut out face image is sent. At this time, a reference image is composed of this image and the enlargement / reduction ratio.

Next, the operation of the preprocessing circuit 503A, including the processing of the CPU 516A, will be described with reference to FIGS. FIG. 10 is a diagram for explaining an example of calculation of the face size, FIG. 11 is a diagram for explaining an example of a face organ detection search range, and FIGS. 14 and 15 are diagrams for explaining a process of enlarging a face image in a face organ detection search range and displaying the image on a display monitor.

First, as for the first (initial image) of the input image, the GUI interface circuit 51
5 (switch 56), the information of the initial range of the face position is supplied to the CPU 516A. CPU 516A
Then, the transmitted initial range is the face organ detection search range signal S
516a is supplied to the face organ position detection unit 5036.

The face organ position detecting unit 5036 detects the face organs, for example, both eyes, nose,
The positions of the mouth, eyebrows, ears, face outline, and the like are detected using luminance and color information. Then, the face organ position detecting unit 5036
Is supplied to the CPU 516A as a signal S5036.

The CPU 516A calculates the size of the face in the image based on the position information of the face organ. For example, as shown in FIG.
When the distance b between the eyes and the mouth in the vertical direction is set, the size of the face is 3a in the horizontal direction and 4b in the vertical direction. In this case, the outline of the face may be extracted by detecting the skin color of the face, and the size of the face may be calculated from the size.

Further, the CPU 516A calculates the face organ detection search range of the next image based on the position information of the face organ. For example, as shown in FIG. 11, a range of ± ax N in the horizontal direction and a range of + bx M and -bx L in the vertical direction around an ax b rectangle including both eyes and a mouth are included in the face image detection search of the next image. Range. Note that the search range does not become larger than the input image.

For the second and subsequent input images, the face organ detection search range obtained in the previous image is determined by the CPU 516.
A supplies it to the face organ position detection unit 5036. The face organ position detection unit 5036 detects the position of the face organ in the supplied search range in the same manner as described above. At this time, information on the face portion of the previous image stored in the reference face image storage unit 5039 is used as a reference image for detecting a face organ. The facial organ position information detected by the facial organ position detection unit 5036 is
516A is supplied as a signal S5036. And
As in the case of the initial image, the CPU 516A calculates the size of the face in the image and the search range for the face organ detection of the next image. FIGS. 12 and 14 show the search range of the face organ.

Next, the CPU 516A calculates an enlargement or reduction magnification from the calculated face size.
This magnification is such that the face is enlarged or reduced so that the face image is fully displayed in the monitor screen on which the user's face is displayed. For example, as shown in FIG. 13, the reduction rate d / 3a is calculated from the horizontal size 3a of the face and the horizontal size d of the monitor screen. FIG.
In the case of 5, the enlargement ratio d / 3a 'is calculated.

The magnification calculated by the CPU 516A is supplied to the image enlargement / reduction unit 5037 as a signal S516c. In the image scaling unit 5037, the signal S5
The input image is enlarged or reduced based on 16c. The enlarged or reduced image is output to the image cutout unit 50.
38. In the image clipping unit 5038, the CP
Based on the face position information (coordinates and the like) already supplied as the signal S516b from the U516A, the face image of the area displayed in the monitor screen is cut out from the enlarged or reduced image.

FIGS. 13 and 15 show the case where the size of the face in the input image is larger than the monitor screen and the case where the size of the face in the input image is smaller than the monitor screen, respectively. Although the size of the face of the input image is different, the size of the face image displayed on the monitor screen is the same in both cases.

The cut-out face image is supplied to the video compression encoder 504 and the image synthesizing unit 512 at the subsequent stage. The cut-out face image is sent to the reference face image storage unit 5039. At this time, a reference image is formed from this image and the scaling ratio.

According to the second embodiment, the above-described first embodiment
The same effect as that of the embodiment can be obtained.

Third Embodiment FIG. 16 is a circuit diagram showing a third embodiment of a signal processing system of a terminal as a data transmission device according to the present invention. FIG. 17 is a circuit diagram showing a specific configuration example of a main part of a preprocessing circuit according to the third embodiment.

The difference between the signal processing system of the terminal according to the third embodiment and the signal processing system according to the second embodiment described above lies in the processing of the preprocessing circuit 503B and the CPU 516B. Specifically, the face image is scaled up and down so as to always keep the size of the face image specified by the user, and the face image is cut out. For this purpose, the pre-processing circuit 503B is different from the circuit of FIG.
It has a circuit configuration in which 0 is added.

Other structures are the same as those shown in FIGS. 8 and 9. Therefore, in FIGS. 16 and 17, the same components as those in FIGS. 8 and 9 are denoted by the same reference numerals. That is, the signal processing system 500B of the terminal
As shown in FIG. 16, a microphone 501, a camera circuit 502, a pre-processing circuit 503B, a video compression encoder 504, an audio compression encoder 505, a multiplexer (MUX) 506, a transmission circuit 507, a reception circuit 508,
A demultiplexer (DeMUX) 509, a video decoder 510, an audio decoder 511, an image synthesizer 512,
The display includes a display circuit 513, a speaker 514, a GUI interface (I / F) circuit 515, and a CPU 516B.

As shown in FIG. 17, the pre-processing circuit 503B includes a face organ position detecting section 5036 and an image scaling section 50.
37, an image cutout unit 5038, a reference face image storage unit 5039, and a monitor-to-face image ratio storage unit 50
40.

As described above, the preprocessing circuits 503B and CP
Since some of the functions of the U516B are different from those of the second embodiment, the preprocessing circuit 503B and the CPU
Only the function B will be described in detail, and description of the other components will be omitted.

The monitor-to-face image ratio storage unit 5040 stores
Based on the facial organ position information extracted by the CPU 516B, the size of the face to be displayed on the monitor screen and the value for calculating the position of the face in the monitor are stored. The stored data is used as a reference value in the second and subsequent input images.

Next, the operation of the preprocessing circuit 503B, including the processing of the CPU 516B, will be described with reference to FIGS. 18 and 19 are diagrams for explaining a process of reducing a face image in a designated transmission image range and displaying the reduced image on a display monitor. FIG. Figure for explaining the ratio of
FIGS. 21 and 22 are diagrams for explaining a process of reducing the face image in the face organ detection search range and displaying the reduced face image on the display monitor.

First, in the first screen (in the initial screen), the user specifies a range of his / her face to be displayed (transmitted) on the monitor, for example, on a GUI of a display monitor, and the information of the range is displayed. , CPU 516B. If this range is similar to the monitor, it is easy for the user to specify the range.

In the CPU 516B, the supplied initial range is supplied to the face organ position detecting section 5036 as a face organ detection search range signal S516a. Facial organ position detector 50
At 36, the facial organs, for example, the eyes, nose, mouth, eyebrows, ears, and the position of the contour of the face are detected within the supplied search range. The face organ position information detected by the face organ position detection unit 5036 is supplied to the CPU 516B as a signal S5036.

The CPU 516B extracts a value for calculating the size of the face displayed on the monitor screen and the position of the face in the monitor based on the position information of the face organ. For example,
In the first (initial image) of the input image, as shown in FIG. 19, the horizontal size c of the range specified by the user, the distance a between the eyes of the face in the initial image, the display monitor 53
Can be used. Or Figure 2
As shown by 0, the skin color of the face may be detected, and the ratio of the number of pixels e, which is the skin color, to the number of pixels f in the monitor image may be calculated. These values are sent to the ratio storage unit 5040 and used as reference values in the second and subsequent input images. Note that this value may be stored in CPU 516B.

Further, the CPU 516B calculates the face organ detection search range of the next image based on the face organ position information. For example, as in the case of the second embodiment, as shown in FIG. 11, ± ax N in the horizontal direction around the rectangle of ax b including both eyes and the mouth, and + bx M,-in the vertical direction.
The range of bx L is defined as a face organ detection search range of the next image.

For the second and subsequent input images, the CPU 516 determines the face organ detection search range obtained in the previous image.
B supplies the face organ position detection unit 5036. In the face organ position detecting unit 5036, within the supplied search range,
Similarly to the above, the position of the facial organ is detected. At this time, the information of the face part of the previous image stored in the reference face image storage unit 5039 is used as a reference image for detecting a face organ. The facial organ position information detected by the facial organ position detecting unit 5036 is represented by a signal S50.
36 is supplied to the CPU 516B.

Then, the CPU 516B calculates an enlargement or reduction magnification for displaying on the display monitor 53. On the first screen (initial screen), for example,
The reduction ratio d / c is calculated from the horizontal length c of the range designated by the user and the horizontal length d of the monitor screen stored in the ratio storage unit 5040. For the second and subsequent screens, the ratio is calculated from the face position information sent from the face organ position detection unit 5036 and the reference value stored in the ratio storage unit 5040. For example, as shown in FIG. 22, the reduction ratio a ′ / a is calculated based on the distance a ′ between both eyes from the face organ position detecting unit 5036 and the distance a between eyes of the initial image stored as a reference value in the ratio storage unit 5040. Is calculated. Alternatively, the reduction ratio may be obtained from the number g of skin color pixels from the face organ position detection unit 5036 and the number e of skin color pixels of the initial image stored in the ratio storage unit 5040.

The magnification calculated by the CPU 516B is supplied to the image enlargement / reduction unit 5037 as a signal S516c. In the image scaling unit 5037, the signal S5
The input image is enlarged or reduced based on 16c. The enlarged or reduced image is output to the image cutout unit 50.
38. In the image clipping unit 5038, the CP
Based on the face position information (coordinates and the like) already supplied as the signal S516b from the U516B, the face image of the area displayed in the monitor screen is cut out from the enlarged or reduced image.

FIG. 22 shows that the size of the face in the input image larger than the monitor screen is reduced so as to be displayed on the initial screen in the same size as the face specified by the user.
The figure shows that the size of the face image displayed on the monitor screen is always the same. When the face in the input image is smaller than the initial image, enlargement processing is performed.

The extracted face image is supplied to the video compression encoder 504 and the image synthesizing device 512 at the subsequent stage. The cut-out face image is sent to the reference face image storage unit 5039. At this time, a reference image is formed from this image and the scaling ratio.

According to the third embodiment, the above-described first embodiment
The same effect as that of the embodiment can be obtained.

Fourth Embodiment Next, in order to improve the tracking accuracy of an image, an image is captured at a frame rate equal to or higher than that of a transmitted image, and a tracking operation is performed.
A specific example of updating a reference image serving as a reference for tracking at a rate equal to or higher than the transmission frame rate will be described as a fourth embodiment of the present invention.

FIG. 23 shows an example in which an image is captured at a frame rate equal to or higher than that of a transmitted image and tracking operation is performed in order to improve tracking accuracy according to the fourth embodiment. FIG. 9 is a conceptual diagram for explaining a process of updating a reference image serving as a reference for tracking at a rate equal to or higher than the transmission frame rate.

FIG. 23 shows a case where the frame rate of the transmitted image is 1/3 of the input frame rate. In practice, the frame rate ratio is controlled according to the bandwidth of the transmission path and the state of congestion, and is not determined uniformly.

In the example of FIG. 23, the face image of the previous image is updated for each input frame as a reference image, and matching is performed within the search range of the next image. For example, the face image detected at 100-1 in FIG. 23 is used as the reference image 100-8 of the next screen, and the face image is detected by matching 100-8 within the image of 100-2. Then, the face image detected here is used as the next reference image.

Further, the search range in the next input image is determined from the position of the face image and updated as in the second and third embodiments described above. The image to be transmitted is selected from the face images detected in the input frame according to the frame rate of the image to be transmitted. In FIG. 23, 100-
The face images cut out from the images 1, 100-4, and 100-7 are transmitted.

When the frame rate of the transmission image and the update of the reference image are the same, the temporal distance between the reference image and the image to be matched is long, so that, for example, when the face image gradually decreases, However, when the expression changes greatly, or when the camera is tilted obliquely, there is a possibility that the matching accuracy is deteriorated. For example, 100
When matching between −8 and 100−4 is performed, the accuracy of matching deteriorates due to the difference in size of the face images. Also, if the update of the search range is also the same as the frame rate of the transmission image, there may be cases where no face image is included in the search range, such as when the face is moving sideways,
The accuracy of the matching may be reduced. For example,
When the search range of 100-1 is applied to the matching of 100-4, all the face images are not included in the search range, and the matching accuracy is deteriorated.

As in the fourth embodiment, at the same or higher frame rate as the transmitted image,
Tracking accuracy can be improved by capturing an image, performing a tracking operation, and updating a reference image serving as a reference for tracking at a rate equal to or higher than the transmission frame rate.

[0130]

According to the present invention, it is possible to reduce the amount of information as much as possible while transmitting the expression of the partner most necessary for communication. Further, according to the present invention, since the face image is matched to the entire image frame, transmission of the background image with high resolution can be naturally deleted, and the amount of information after compression can be greatly reduced. This makes it possible to transmit high-quality images.

According to the present invention, the frame correlation of an image can be greatly improved.
By using a compression technique for performing inter-frame compression such as EG, it is possible to greatly reduce the amount of information. This makes it possible to transmit high-quality images.

Further, according to the present invention, it is possible to automatically keep the state of the image that the user wants to transmit (the state of the image lock), and to help detect the characteristic portion of the image, Tracking of the target image is possible. As a result, the correlation in the time axis direction of the image is increased, and the amount of information after compression can be reduced. This allows
High quality images can be transmitted.

Further, according to the present invention, it is possible to realize a state in which the eyes are more aligned.

Further, according to the present invention, it is possible to tell each other how to talk while watching the video of the topic,
In addition, there is an advantage that virtual reality communication can be realized, that is, the eyes are aligned when the faces of each other are viewed on the monitor.

[Brief description of the drawings]

FIG. 1 is a system configuration diagram showing an embodiment of a visual communication service system as a data transmission system according to the present invention.

FIG. 2 is an external view illustrating a configuration example of a mobile terminal according to the embodiment.

FIG. 3 is a circuit diagram showing a first embodiment of a signal processing system of a terminal as a data transmission device according to the present invention.

FIG. 4 is a circuit diagram showing a pre-processing circuit of FIG. 3 according to the first embodiment.

FIG. 5 is a diagram for explaining an outline of tracking in a preprocessing circuit.

FIG. 6 is a diagram illustrating a first method of determining an initial reference image.

FIG. 7 is a diagram illustrating a second method of determining an initial reference image.

FIG. 8 is a circuit diagram showing a second embodiment of a signal processing system of a terminal as a data transmission device according to the present invention.

FIG. 9 is a circuit diagram showing a preprocessing circuit of FIG. 8 according to a second embodiment.

FIG. 10 is a diagram illustrating an example of calculating the size of a face.

FIG. 11 is a diagram for describing an example of a face organ detection search range.

FIG. 12 is a diagram illustrating a process of reducing a face image in a face organ detection search range and displaying the reduced image on a display monitor.

FIG. 13 is a diagram illustrating a process of reducing a face image in a face organ detection search range and displaying the reduced face image on a display monitor.

FIG. 14 is a diagram for explaining a process of enlarging a face image in a face organ detection search range and displaying the enlarged face image on a display monitor.

FIG. 15 is a diagram for explaining a process of enlarging a face image in a face organ detection search range and displaying the enlarged face image on a display monitor.

FIG. 16 is a circuit diagram showing a third embodiment of a signal processing system of a terminal as a data transmission device according to the present invention.

FIG. 17 is a circuit diagram showing the preprocessing circuit of FIG. 16 according to the third embodiment.

FIG. 18 is a diagram illustrating a process of reducing a face image in a specified transmission image range and displaying the reduced face image on a display monitor in the third embodiment.

FIG. 19 is a diagram illustrating a process of reducing a face image in a designated transmission image range and displaying the reduced face image on a display monitor in the third embodiment.

FIG. 20 is a diagram for explaining a ratio between the number of pixels e, which is the skin color of the face, and the number of pixels f in the monitor image in the third embodiment.

FIG. 21 is a diagram illustrating a process of reducing a face image in a face organ detection search range and displaying the reduced face image on a display monitor in the third embodiment.

FIG. 22 is a diagram illustrating a process of reducing a face image in a face organ detection search range and displaying the reduced face image on a display monitor in the third embodiment.

FIG. 23 is a diagram showing an example in which an image is captured at a frame rate equal to or higher than that of a transmitted image in order to improve tracking accuracy according to the fourth embodiment;
FIG. 9 is a conceptual diagram for describing a process of performing a tracking operation and updating a reference image serving as a reference for tracking at a rate equal to or higher than a transmission frame rate.

FIG. 24 is an explanatory diagram of a conventional visual communication terminal and a transmission path.

[Explanation of symbols]

100: visual communication service system, 50-1, 50-2: mobile terminal, 60: transmission line,
61-1, 61-2 ... wireless base station (MSB), 62-
1, 62-2 ... mobile switching center (MSC), 63-1, 63
-2: Gateway Mobile Switching Center (GMSC), 64: Network, 6
5: Gateway station (GW), 66: Application service provider (ASP), 500, 500A, 500B ...
Terminal signal processing system, 501 ... microphone, 502 ...
Camera circuit, 503, 503A, 503B pre-processing circuit, 504 video compression encoder, 505 audio compression encoder, 506 multiplexer (MUX), 507
Transmission circuit, 508 reception circuit, 509 demultiplexer (DeMUX), 510 video decoder, 511
Audio decoder, 512: image synthesizing device, 513: display circuit, 514: speaker, 515: GUI interface (I / F) circuit, 516, 516A, 516B: CP
U, 5036: face organ position detection unit, 5037: image scaling unit, 5038: image cutout unit, 5039: reference face image storage unit, 5040: monitor and face image ratio storage unit.

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) H04N 7/24 H04N ^{7 /} 13Z 5L096 (72) Inventor Masatoshi Takashima 6-7 Kita Shinagawa, Shinagawa-ku, Tokyo No. 35 Inside Sony Corporation (72) Hideyuki Narita Inventor 6-7-35 Kita-Shinagawa, Shinagawa-ku, Tokyo Tokyo Inside (72) Inventor Daisuke Hiranaka 6-35, Kita-Shinagawa, Shinagawa-ku, Tokyo No. within Sony Corporation (72) Inventor Yoshihiro Ishida 6-35 Kita Shinagawa, Shinagawa-ku, Tokyo F-term within Sony Corporation (reference) 5B057 BA02 CA08 CA12 CA16 CB08 CB12 CB16 CC01 CD05 CH08 DA08 DB02 DB09 DC03 DC32 5C059 MA00 SS07 SS10 UA02 UA03 UA05 5C064 AA01 AB04 AC04 AC06 AC08 AC12 AC16 AD02 AD14 5C075 AB06 CA05 5C076 AA02 AA21 AA22 BA06 5L096 AA06 BA20 CA02 DA01 EA03 EA35 FA66 GA08 H A07 JA20

Claims (48)

[Claims] 1. A data transmission method for transmitting at least image data in an image frame of a preset size, wherein an image including a target image to be transmitted is captured, and the image is transmitted from the captured image. A data transmission method in which a target image is adjusted so as to substantially fill the image frame, and image data in the adjusted image frame is compressed and transmitted. 2. When adjusting a target image to fill an image frame, matching is performed using an image of a characteristic portion of an image to be transmitted. 2. The data transmission method according to claim 1, wherein the tracking of the image is performed by cutting out the image. 3. When adjusting the target image so as to fill the image frame, the distance of the characteristic portion is calculated by using the image of the characteristic portion of the transmission image, whereby the image is enlarged, reduced, and tracked. The data transmission method according to claim 1, wherein 4. When detecting a characteristic portion of an image, the state of the image is locked in a desired image state while monitoring the state in which the transmitted image is captured, and the vicinity of the center of the image at the time of locking is determined. 3. The data transmission method according to claim 2, wherein the data transmission method is a characteristic part. 5. When detecting a characteristic portion of an image, the state of the image is locked in a desired image state while monitoring the state in which the transmission image is captured, and the vicinity of the center of the image at the time of locking is determined. 4. The data transmission method according to claim 3, wherein the data transmission method is a characteristic part. 6. When a characteristic portion of an image is detected, a state in which a transmission image is captured is displayed, a predetermined portion is designated on a display screen, and a characteristic point of the image is set around the designated point. 2. The data transmission method according to 2. 7. When detecting a characteristic portion of an image, a state in which the transmitted image is captured is displayed, a predetermined portion is designated on the display screen, and the characteristic portion of the image is set around the designated point. 3. The data transmission method according to 3. 8. When a target image is locked by using a characteristic portion of an image, the image is captured at a frame rate equal to or higher than that of a transmitted image.
5. The data transmission method according to claim 4, wherein a tracking operation is performed and a reference image serving as a reference for tracking is updated at a rate equal to or higher than the transmission frame rate. 9. When locking a target image using a characteristic portion of an image, the image is captured at a frame rate equal to or higher than that of a transmitted image.
6. The data transmission method according to claim 5, wherein a tracking operation is performed, and a reference image serving as a reference for tracking is updated at a rate equal to or higher than the transmission frame rate. 10. The data transmission method according to claim 1, wherein a state in which the transmission image is captured is displayed, a range to be transmitted is specified on the display screen, and a range to transmit the image is determined. 11. A data transmission method for transmitting at least image data, wherein an image including a target image to be transmitted is imaged, and the target image to be transmitted has a predetermined size from the captured image. A data transmission method in which the image data in the image frame after adjustment is compressed and transmitted. 12. When adjusting a target image to have a predetermined size, matching is performed using an image of a characteristic portion of an image to be transmitted, and a portion around the best matching is centered. The data transmission method according to claim 11, wherein the tracking of the image is performed by cutting out the image. 13. When adjusting a target image to have a predetermined size, a distance of a characteristic portion is calculated by using an image of a characteristic portion of a transmission image, and thereby, enlargement, reduction, and tracking of the image are performed. 12. The data transmission method according to claim 11, wherein: 14. When detecting a characteristic portion of an image, the state of the image is locked in a desired image state while monitoring the state where the transmitted image is captured. The data transmission method according to claim 12, which is a characteristic portion. 15. When detecting a characteristic portion of an image, the state of the image is locked in a desired image state while monitoring the state in which the transmitted image is captured, and the vicinity of the center of the image at the time of locking is determined. 14. The data transmission method according to claim 13, which is a characteristic part. 16. When detecting a characteristic portion of an image, a state in which the transmitted image is captured is displayed, a predetermined portion is designated on the display screen, and the characteristic portion of the image is set around the designated point. 13. The data transmission method according to item 12. 17. When detecting a characteristic portion of an image, a state in which the transmission image is captured is displayed, a predetermined portion is designated on a display screen, and the characteristic portion of the image is set around the designated point. 14. The data transmission method according to item 13. 18. When a target image is locked using a characteristic portion of an image, the image is captured at a frame rate equal to or higher than that of a transmitted image, a tracking operation is performed, and a transmission frame is transmitted. 15. The data transmission method according to claim 14, wherein a reference image serving as a reference for tracking is updated at a rate equal to or higher than the rate. 19. When a target image is locked using a characteristic portion of an image, an image is captured at a frame rate equal to or higher than that of a transmitted image, a tracking operation is performed, and a transmission frame is transmitted. The data transmission method according to claim 15, wherein the reference image serving as a reference for tracking is updated at a rate equal to or higher than the rate. 20. The data transmission method according to claim 11, wherein a state in which the transmission image is captured is displayed, a range to be transmitted is specified on the display screen, and a range to transmit the image is determined. 21. A data transmission method for transmitting at least image data while displaying image data of at least a user on a display screen among a plurality of terminals and displaying received image data on the display screen. A data transmission method for capturing an image including a target image of a user to be transmitted from a substantially central portion of the data. 22. Display of at least user image data on a display screen among a plurality of terminals, display of received image data from the other party on the display screen, and display of a common topic on the display screen. A data transmission method for transmitting at least image data, wherein an image including a target image of a user to be transmitted is captured from a substantially central portion of the display screen, and the image is captured about a substantially central portion of the display screen as an imaging portion. A data transmission method in which the image data is displayed on one side of an upper or lower side of a display screen, and the content is displayed on the other side. 23. The data transmission method according to claim 22, wherein image data is displayed above the content on the display screen. 24. The data transmission method according to claim 22, wherein image data is displayed below the content on the display screen. 25. A data transmission device for transmitting at least image data in an image frame of a preset size, wherein said imaging device captures an image including a target image to be transmitted. Data having a first circuit for adjusting a target image to be transmitted from a captured image to substantially fill the image frame, and a second circuit for compressing and transmitting image data in the adjusted image frame Transmission equipment. 26. The first circuit performs matching using an image of a characteristic portion of an image to be transmitted when adjusting the target frame so as to fill the image frame. 26. Image tracking is performed by cutting out an image around the center of the image.
A data transmission device according to claim 1. 27. The first circuit calculates a distance of a characteristic portion using an image of a characteristic portion of a transmission image when adjusting the target image to fill the image frame.
26. The data transmission apparatus according to claim 25, wherein the enlargement, reduction, and tracking of the image are performed. 28. A display device comprising: display means for displaying image data; and lock means for locking an image displayed on the display means in a desired image state. 27. The data transmission apparatus according to claim 26, wherein when detecting a characteristic portion of the image, a vicinity of the center of the image when the image is locked by the lock unit is a characteristic portion of the image. 29. A display device comprising: display means for displaying image data; and lock means for locking an image displayed on the display means in a desired image state. 28. The data transmission apparatus according to claim 27, wherein when detecting a characteristic portion of the image, a vicinity of the center of the image when the image is locked by the lock unit is a characteristic portion of the image. 30. A display means for displaying image data and including a pointer capable of designating a predetermined location, and a designation means capable of designating a predetermined location of an image displayed on the display means with the pointer. 27. The data transmission apparatus according to claim 26, wherein the first circuit, when detecting a characteristic portion of the image, sets the characteristic portion of the image around a point designated by the designation means. 31. A display means for displaying image data and including a pointer capable of designating a predetermined location, and a designation means capable of designating a predetermined location of an image displayed on the display means with the pointer. 28. The data transmission apparatus according to claim 27, wherein the first circuit, when detecting a characteristic portion of the image, sets the characteristic portion of the image around a point designated by the designation means. 32. The first circuit captures an image at a frame rate equal to or higher than that of a transmitted image when locking a target image using a characteristic portion of the image, 29. The data transmission apparatus according to claim 28, wherein a tracking operation is performed to update a reference image serving as a reference for tracking at a rate equal to or higher than the transmission frame rate. 33. The first circuit captures an image at a frame rate equal to or higher than that of a transmitted image when locking a target image using a characteristic portion of the image, 30. The data transmission apparatus according to claim 29, wherein a tracking operation is performed to update a reference image serving as a reference for tracking at a rate equal to or higher than the transmission frame rate. 34. A display means for displaying image data and including a pointer capable of designating a predetermined location, and a designation means capable of designating a predetermined location of an image displayed on the display means with the pointer. 26. The data transmission device according to claim 25, wherein the first circuit determines a range designated by the designation means as a range for transmitting an image. 35. A data transmission device for transmitting at least image data, comprising: an image pickup means for picking up an image including a target image to be transmitted; A data transmission apparatus comprising: a first circuit that adjusts the image data so that the image data in the image frame after the adjustment is compressed; 36. The first circuit, when adjusting a target image to have a predetermined size, performs matching using an image of a characteristic portion of an image to be transmitted, and obtains a best matching portion. 36. The data transmission apparatus according to claim 35, wherein the tracking of the image is performed by cutting out an image around the image. 37. When adjusting the target image to have a predetermined size, the first circuit calculates a distance of the characteristic portion by using an image of the characteristic portion of the transmission image, and thereby, The data transmission device according to claim 35, wherein the data transmission device performs enlargement, reduction, and tracking of an image. 38. A display device comprising: display means for displaying image data; and lock means for locking an image displayed on the display means in a desired image state. 37. The data transmission apparatus according to claim 36, wherein when detecting a characteristic portion of the image, a vicinity of the center of the image when locked by the locking means is a characteristic portion of the image. 39. A display device, comprising: display means for displaying image data; and lock means capable of locking an image displayed on the display means in a desired image state. 38. The data transmission apparatus according to claim 37, wherein when detecting a characteristic portion of the image, a vicinity of the center of the image when the image is locked by the lock unit is a characteristic portion of the image. 40. A display means for displaying image data and including a pointer capable of designating a predetermined location, and a designation means capable of designating a predetermined location of an image displayed on the display means with the pointer. 37. The data transmission apparatus according to claim 36, wherein the first circuit, when detecting a characteristic portion of the image, sets the characteristic portion of the image around a point designated by the designation means. 41. A display means for displaying image data and including a pointer capable of designating a predetermined location, and a designation means capable of designating a predetermined location of an image displayed on the display means with the pointer. 38. The data transmission apparatus according to claim 37, wherein the first circuit, when detecting a characteristic part of the image, sets the characteristic part of the image around a point designated by the designation means. 42. The first circuit captures an image at a frame rate equal to or higher than that of a transmitted image when locking a target image using a characteristic portion of the image. 39. The data transmission apparatus according to claim 38, wherein a tracking operation is performed to update a reference image serving as a reference for tracking at a rate equal to or higher than the transmission frame rate. 43. The first circuit captures an image at a frame rate equal to or higher than a transmitted image when locking a target image using a characteristic portion of the image, The data transmission apparatus according to claim 39, wherein a tracking operation is performed to update a reference image serving as a reference for tracking at a rate equal to or higher than the transmission frame rate. 44. A display means for displaying image data and including a pointer capable of designating a predetermined location, and a designation means capable of designating a predetermined location of an image displayed on the display means with the pointer. 36. The data transmission device according to claim 35, wherein the first circuit determines a range designated by the designation means as a range for transmitting an image. 45. A data transmission system for transmitting at least image data while displaying at least image data of a user on a display screen among a plurality of terminals and displaying received image data on the display screen. Image capturing means for capturing an image including a target image to be transmitted; a first circuit for adjusting the target image to be transmitted from the image captured by the image capturing means so as to substantially fill the image frame; A second circuit for compressing and transmitting image data in the frame. 46. A data transmission system for transmitting at least image data while displaying at least image data of a user on a display screen among a plurality of terminals and displaying the received image data on the display screen. Image capturing means for capturing an image including a target image to be transmitted, a first circuit for adjusting the target image to be transmitted from the captured image to have a predetermined size, and an image after the adjustment. A second circuit for compressing and transmitting image data in the frame. 47. A data transmission system for transmitting at least image data while displaying at least image data of a user on a display screen among a plurality of terminals and displaying the received image data on the display screen. A data transmission system in which an image pickup means for picking up an image including a target image of a user to be transmitted is arranged at a substantially central portion of the data transmission system. 48. At least a user's image data is displayed on a display screen among a plurality of terminals, received image data from a partner is displayed on the display screen, and a common topic content is displayed on the display screen. A data transmission system for transmitting at least image data, wherein the imaging means is arranged at a substantially central portion of the display screen and captures an image including a target image of a user to be transmitted; and a display screen as an imaging part And a display means for displaying the image data on one side of the upper or lower side of the display screen, and displaying the content on the other side, about the center of the display screen.