JP2011129995A - Communication terminal device, image processing method and program for the same, and image communication system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a communication terminal device, along with an image processing method for the same, capable of maintaining the quality of a communication image. <P>SOLUTION: At the communication terminal device 10, a frame rate of image data output from a camera 110 is determined by a controller 140, based on the results of comparison between an image photographed by the camera 110 and an image photographed by the camera 110 just before the image. Utilizing the decision, the communication terminal device is equipped with a function for controlling and executing to delay the frame rate of the image photographed by a camera 110 when the image has few changing parts or to quicken the frame rate of the image when the image has much changing parts. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a communication terminal device, an image processing method and program for the communication terminal device, and an image communication system.

  In recent years, with the widespread use of portable or fixed communication devices having a camera function for taking images and the increase in the capacity and speed of communication networks, two-way image communication such as videophone and videoconferencing has been widely used. Yes. When performing such bidirectional image communication, the communication data transmitted over the communication network inevitably requires a larger amount of data than communication data by voice communication. In addition, the amount of communication data further increases in order to realize high-quality bidirectional image communication with a communication image such as a detailed screen and a smooth moving image.

  Therefore, in bidirectional image communication, the amount of communication data increases in proportion to the quality of the communication image. Therefore, in bidirectional image communication between mobile communication terminals such as mobile phones, communication is performed according to the amount of communication data. In many cases, a fee is charged, that is, the communication fee at the time of executing bidirectional image communication becomes higher than that at the time of executing interactive voice communication.

  To solve this problem, as a technique to reduce the amount of image data to be transmitted, it detects the presence or absence of image changes by comparing consecutive image frames of image data, and extracts only the changed region including this changed part. The difference between the pixel values of the technology (Patent Document 1) and the basic image block and the change image block is obtained, and a block having a difference larger than a predetermined threshold is extracted. An image transmission method (Patent Document 2) for transmitting image information is known.

JP 2001-320707 A JP-A-8-32974

However, the techniques disclosed in Patent Document 1 and Patent Document 2 both detect a change portion of an image and transmit image information corresponding to the change portion of the image. Therefore, when transmitting an image with a strong motion or an image with a sharp change in brightness, the detected image change portion increases, and the amount of image information for transmitting the image increases. The amount of communication data transmitted during execution also increases, resulting in a problem that the communication fee increases.
Furthermore, as the amount of change in the detected image increases, the amount of image information for transmitting the image increases, which makes it difficult to maintain the quality of the communication image as follows. . In other words, if there are many image changing parts, the amount of image information to be transmitted is limited depending on the hardware performance of the transmitting device, so that when receiving the amount of image information received by the receiving device, jerky due to dropped frames, etc. There is a risk of quality degradation of the communication image, such as feeling and block noise.

  Therefore, in order to solve the above-described problem, the present invention can suppress communication charges by reducing the amount of communication data and ensure the quality of communication images even when transmitting an image with many image change portions. An object of the present invention is to provide a communication terminal device and an image processing method for the communication terminal device.

  In order to achieve the above object, the present invention captures an image and outputs a captured image at a predetermined frame rate, and a camera output image that temporarily stores the image output by the camera unit. A storage unit, a communication interface unit that transmits and receives data, a first image that is an image output by the camera unit, and a camera output image storage unit that is output from the camera unit immediately before the first image. A camera image frame rate control unit that compares the stored second image and determines the frame rate based on a result of the comparison, and shows a change portion of the first image with respect to the second image And a transmission data generation unit that generates data including difference information and transmits the data to the communication interface unit.

According to the present invention, by determining the frame rate of the image data output from the camera unit based on the comparison result between the image captured by the camera unit and the image captured by the camera unit immediately before this image, The frame rate can be controlled according to the comparison result of the captured images. Further, based on the comparison result between the image photographed by the camera unit and the image photographed by the camera unit immediately before this image, difference information including the changed part of the image is extracted, and transmission data including this difference information is extracted. By generating, the amount of image data included in the transmission data can be reduced.
Therefore, the frame rate of the image to be compared is controlled in accordance with the changed portion of the image, and the transmission data including the difference information extracted from the changed portion of the image is generated. By reducing the amount of communication data, it is possible to suppress communication charges and ensure the quality of communication images.

It is a block diagram which shows the structure of the communication terminal device concerning the 1st Embodiment of this invention. It is a flowchart which shows operation | movement of the communication terminal device concerning 1st Embodiment. It is a block diagram which shows the structure of the communication terminal device concerning the 2nd Embodiment of this invention. It is a figure which illustrates notionally an example of the difference information based on the comparison result of a 1st image and a 2nd image. It is a figure which illustrates notionally the determination of the frame rate according to the pixel or pixel block change area | region based on difference information. It is a flowchart which shows operation | movement of the communication terminal device concerning 2nd Embodiment. It is a figure which illustrates notionally extraction of the image information (difference information) contained in the transmission data in the communication terminal device concerning 2nd Embodiment. It is a block diagram which shows the structure of the image communication system containing the communication terminal device concerning the 3rd Embodiment of this invention. It is a sequence diagram which shows the operation | movement sequence of the image communication system in 3rd Embodiment. It is a figure which illustrates notionally the image information transmitted / received by the image communication system in 3rd Embodiment.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
[First Embodiment]
The communication terminal apparatus according to the first embodiment of the present invention determines a frame rate for outputting an image captured by a camera based on a comparison result of consecutive image frames, and continuously based on the difference information. The difference information of the image frame to be extracted is extracted, and transmission data including the extracted difference information is generated.

As shown in FIG. 1, the communication terminal apparatus 10 according to the present embodiment includes a camera unit 110, a camera output image storage unit 120, a communication interface unit 130, a camera image frame rate control unit 140, and a transmission data generation unit 150. It is configured.
The camera unit 110 optically captures an image of a landscape or the like and outputs digital data of the captured image (hereinafter, “image digital data” is collectively referred to as “image data”) at a predetermined frame rate.
The camera output image storage unit 120 is a memory device that temporarily stores image data, and temporarily stores image data output from the camera unit 110.

  The communication interface unit 130 transmits data (transmission data) including image data based on an image photographed by the camera unit 110 to a communication partner, and data (reception data) including image data transmitted from the communication partner. Receive.

  The camera image frame rate control unit 140 stores the first image that is the image data output from the camera unit 110 and the image data that is output from the camera unit 110 immediately before the first image and stores the camera output image. The second image stored in the unit 121 is compared, and the frame rate of the image data output from the camera unit 110 is determined based on the comparison result.

The transmission data generation unit 150 generates transmission data including difference information indicating a changed portion of the first image with respect to the second image from the comparison result between the first image and the second image.
Each component of the communication terminal device 10 according to the present embodiment is realized by installing a computer program (software) in a computer including a CPU (Central Processing Unit), a memory, and an interface. Various functions of the apparatus 10 are realized by cooperation of various hardware resources of the computer and the computer program.

Next, the operation of the communication terminal apparatus according to this embodiment will be described with reference to FIG.
As shown in FIG. 2, the communication terminal device 10 according to the present embodiment temporarily stores image data of an image taken by the camera unit 110 in the camera output image storage unit 120 (S101).

When the image data output from the camera unit 110 is stored in the camera output image storage unit 120, the camera image frame rate control unit 140 stores image data (first image) of the next image output from the camera unit 110. And the image data (second image) stored in the camera output image storage unit 120 are compared (S102).
The camera image frame rate control unit 140 determines the frame rate of the image data output from the camera unit 110 based on the comparison result between the first image and the second image (S103).

  When the frame rate of the image data output from the camera unit 110 is determined by the camera image frame rate control unit 140, the transmission data generation unit 150 determines the first image derived by the camera image frame rate control unit 140 and the first image. From the comparison result with the second image, transmission data including difference information indicating a changed portion of the first image with respect to the second image is generated (S104).

As described above, according to the communication terminal apparatus according to the present embodiment, the image is output from the camera unit based on the comparison result between the image captured by the camera unit and the image captured by the camera unit immediately before this image. By determining the frame rate of the image data, it is possible to control such that the frame rate of the image is reduced when there are few changed portions, and the frame rate of the image is increased when there are many changed portions. Further, based on the comparison result between the image photographed by the camera unit and the image photographed by the camera unit immediately before this image, difference information including the changed portion of the image is extracted, and transmission data including this difference information is extracted. By generating, the amount of image data included in the transmission data can be reduced.
Therefore, the frame rate of the image to be compared is controlled in accordance with the changed portion of the image, and the transmission data including the difference information extracted from the changed portion of the image is generated. The communication charge can be suppressed by ensuring the quality of the communication image and reducing the amount of communication data.

[Second Embodiment]
FIG. 3 is a block diagram showing a configuration of the communication terminal apparatus 20 according to the second embodiment of the present invention.
The communication terminal device 20 according to the present embodiment determines the frame rate when outputting an image captured by the camera, the comparison result of the data amount in pixels or pixel blocks of successive image frames, and the luminance distribution of successive image frames. The data including the difference information between successive image frames is generated and transmitted.
In addition, about the structure and function of the communication terminal device 20 concerning this Embodiment, what has the same structure and function as the communication terminal device 10 demonstrated in 1st Embodiment attaches | subjects the same code | symbol, and these The detailed description of is omitted.

As shown in FIG. 3, the communication terminal device 20 according to the present embodiment captures an image and outputs image data of the captured image at a predetermined frame rate, and temporarily stores the image data. A camera output image storage unit 120 that performs communication, a communication interface unit 130 that transmits and receives data including image data, a camera image frame rate control unit 240 that determines a frame rate of image data output from the camera unit 110, and a communication interface. The transmission data generation unit 150 generates transmission data transmitted from the unit 130.
Here, the communication interface unit 130 may be a wired data communication communication interface such as a LAN (Local Area Network) or ISDN (Integrated Services Digital Network), such as a W-LAN (Wireless-Local Area Network), Bluetooth. (Registered trademark) or a communication interface for wireless data communication such as infrared communication may be used.

  Comparison between the first image, which is image data output from the camera unit 110, and the second image, which is image data output immediately before the first image and stored in the camera output image storage unit 120 The camera image frame rate control unit 240 that determines the frame rate of the image data output from the camera unit 110 based on the result further includes a data change portion amount comparison determination unit 241 and a luminance distribution comparison determination unit 242. Yes.

The data change partial amount comparison / determination unit 241 compares the first image and the second image, and outputs from the camera unit 110 according to the change amount of the pixel or pixel block of the first image with respect to the second image. The frame rate of the image data to be processed is determined.
The luminance distribution comparison / determination unit 242 compares the first image and the second image, and determines the image data output from the camera unit 110 according to the difference in luminance distribution of the first image with respect to the second image. Determine the frame rate.

  Each component of the communication terminal device 20 according to the present embodiment is realized by installing a computer program (software) in a computer including a CPU (Central Processing Unit), a memory, and an interface. Various functions of the apparatus 20 are realized by cooperation of various hardware resources of the computer and the computer program.

<Frame rate control>
Here, the function of determining the frame rate of the image data output from the camera unit 110 based on the comparison between the first image and the second image by the camera image frame rate control unit 240 will be described in detail.
FIG. 4 shows a comparison between image data a first output from the camera unit 110 and image data b output from the camera unit 110 immediately after the image data a out of continuous image data output from the camera unit 110. It is a figure which shows notionally the difference information c corresponding to the pixel or pixel block of the image data a and b derived | led-out based on this.

  The data change partial amount comparison / determination unit 241 of the camera image frame rate control unit 240 detects a pixel or a pixel block having a change between the image data a and the image data b, and calculates difference information c of the image data b with respect to the image data a. Extract. The difference information c shown in FIG. 4 indicates a pixel or pixel block that is newly displayed with the image data b as a pixel or pixel block indicated by black painting, and a pixel or pixel that is not displayed as the pixel or pixel block indicated by diagonal lines as the image data b. Indicates a block.

The data change partial amount comparison / determination unit 241 derives, from the extracted difference information c, the ratio of the change amount of the pixel or the pixel block of the image data b to the total number of pixels or the total pixel block amount of the image data a. The frame rate of the image data output from the camera unit 110 is determined according to the rate of change of the pixel b or pixel block.
Here, FIG. 5 shows an example of the relationship between the frame rate and the ratio of the change amount of the pixel or pixel block to the total number of pixels or the total pixel block amount of the image data to be compared.
For example, the data change portion amount comparison unit 241 stores in advance a condition indicating the relationship between the change rate of the pixel or pixel block and the frame rate as shown in FIG. The image data derived to the condition stored in advance by deriving the ratio of the change amount of the pixel or pixel block of the image data b to the total number of pixels or the total pixel block amount of the image data a from the extracted difference information c The frame rate of the image data output from the camera unit 110 can be determined by associating the rate of change of the pixel or pixel block of b.

The luminance distribution comparison / determination unit 242 of the camera image frame rate control unit 240 detects a change in the luminance distribution of the image data by comparing the image data a and the image data b, and responds to the change in the luminance distribution. The frame rate of the image data output from the camera unit 110 is determined.
Specifically, the luminance distribution comparison determination unit 242 stores in advance a condition indicating the relationship between the luminance distribution change amount between the two image data and the frame rate, and the luminance distribution comparison determination unit 242 detects the detected image. By making the luminance distribution change amount of the image data b with respect to the data a correspond to the condition stored in advance, the frame rate of the image data output from the camera unit 110 can be determined.
Further, the luminance distribution comparison / determination unit 242 may be based on the luminance distribution change amount of the signal component in the image data indicating the luminance distribution having the greatest influence (large change) when determining the frame rate. It may be based on the luminance distribution change amount of each signal component derived by weighting each signal component of data or the average of the luminance distribution change amounts of all signal components.

The camera image frame rate control unit 240 sets the frame rate of the image data output from the camera unit 110 to either the data change partial amount comparison determination unit 241 or the luminance distribution comparison determination unit 242 according to the characteristics of the image data. Control using the decision result.
For example, the determination result of the data change partial amount comparison / determination unit 241 may be used for an image with intense movement, and the determination result of the luminance distribution comparison / determination unit 242 may be used for an image with intense luminance change. Further, the camera image frame rate control unit 240 controls the frame rate of the image data output from the camera unit 110 using the determination results of both the data change partial amount comparison determination unit 241 and the luminance distribution comparison determination unit 242. Also good.

Next, the operation of the communication terminal apparatus 20 according to the present embodiment will be described with reference to the flowchart shown in FIG. 6 and the image data difference information shown in FIG.
As shown in FIG. 6, when image data is output by the camera unit 110 (S201), the camera output image storage unit 120 determines whether or not the previous image data output from the camera unit 110 is stored. (S202).

  When the image data output immediately before is not stored (“NO” in S202), the camera output image storage unit 120 temporarily stores the image data output from the camera unit 110 in the camera output image storage unit 121. In step S203, image data to be output next from the camera unit 110 is waited. Here, among the image data output from the camera unit 110 shown in FIG. 7, the image data A <b> 1 is stored in the camera output image storage unit 120.

  When the image data is stored in the camera output image storage unit 120 (“YES” in S202), that is, the image data output immediately before the image data output from the camera unit 110 is stored in the camera output image storage unit 120. When stored, the camera image frame rate control unit 240 outputs the image data (first image) output by the camera unit 110 and the image data (second image) stored in the camera output image storage unit 120. Are compared (S204). Here, among the image data output from the camera unit 110 shown in FIG. 7, the image data A2 corresponds to the first image, and the image data A1 corresponds to the second image.

  For comparison between the first image and the second image executed by the camera image frame rate control unit 240, a pixel or pixel block in the first image and the second image by the data change partial amount comparison determination unit 241 is used. There are comparison of a change area of a pixel or a pixel block for detecting a change portion, and comparison of luminance distributions in the first image and the second image by the luminance distribution comparison determination unit 242.

When each comparison operation of the first and second images described above is executed, the camera image frame rate control unit 240 extracts difference information of the first image with respect to the second image based on the comparison result. (S205).
Here, among the comparison operations performed by the camera image frame rate control unit 240, the comparison result of the pixel or pixel block change region by the data change partial amount comparison determination unit 241 will be described as an example.
The data change part amount comparison / determination unit 241 compares image data A2 and A1 in the image data shown in FIG. 7 to obtain image data B1 indicating a change part of a pixel or a pixel block of the image data A2 with respect to the image data A1. Extracted as difference information. The pixel or pixel block indicated by black in the image data B1 indicates a newly displayed pixel or pixel block, and the pixel or pixel block indicated by diagonal lines indicates that the pixel or pixel block is not displayed.

When the difference information of the first image with respect to the second image is extracted, the camera image frame rate control unit 240 determines the frame rate of the image data output from the camera unit 110 based on the difference information (S206). ).
Since the frame rate of the image data output from the camera unit 110 is determined based on the difference information between the two images, the frame rate is decreased when there are few changed portions between the two images, and there are many changed portions. In some cases, it is possible to control to increase the frame rate.

  When the frame rate is determined, the transmission data generation unit 150 generates transmission data including the extracted difference information (S207). Here, the transmission data including the image data B1 indicating the changed portion of the pixel or pixel block of the image data A2 and A1 shown in FIG. 7 is generated, and the transmission data generated from the communication interface unit 130 is transmitted to the communication partner. Sent.

When image communication is performed using the communication terminal device 20 according to the present embodiment, the frame rate of image data output from the camera unit 110 is controlled by repeating each of the above steps, and continuous. The transmission data including only the difference information of the processed image data is transmitted.
For example, when it is desired to transmit the image data A3 among the image data output from the camera unit 110 shown in FIG. 7, the transmission data including the image data B2 that is the difference information of the image data A3 with respect to the image data A2 is transmitted. When it is desired to transmit the image data A4, transmission data including image data B3 that is difference information of the image data A4 with respect to the image data A3 is transmitted.

  Thus, according to the communication terminal device 20 according to the present embodiment, of the comparison result between the image captured by the camera unit and the image captured by the camera unit immediately before this image, the pixel or pixel block By determining the frame rate of the image data output from the camera unit based on the amount of change or the amplitude distribution change amount in the frequency component of the image data, it is possible to control the frame rate in more detail, and the quality of the communication image Can keep. Further, based on the comparison result between the image photographed by the camera unit and the image photographed by the camera unit immediately before this image, difference information including the changed part of the image is extracted, and transmission data including this difference information is extracted. By generating, the amount of image data included in the transmission data can be reduced.

[Third Embodiment]
FIG. 8 is a block diagram showing a configuration of an image communication system including the communication terminal device 30 according to the third embodiment of the present invention.
The image communication system 300 including the communication terminal device 30 according to the present embodiment realizes image communication by the communication terminal devices 30 and 31 transmitting and receiving data including image data via the network X.
In addition, about the structure and function of the communication terminal device 30 concerning this Embodiment, the same code | symbol is attached | subjected to what has the same structure and function as the communication terminal device 20 demonstrated in 2nd Embodiment, and these. The detailed description of is omitted.

As shown in FIG. 8, the mobile terminal device 30 according to the present embodiment in the image communication system 300 captures an image and outputs image data of the captured image at a predetermined frame rate, and a camera unit. 110 determines the frame rate of image data output from the camera unit 110, the camera output image storage unit 120 that temporarily stores the image data output from the camera unit 110, the communication interface unit 130 that transmits and receives data including the image data. The camera image frame rate control unit 240, the transmission data generation unit 150 that generates transmission data transmitted from the communication interface unit 130, and the image data included in the reception data received by the communication interface unit 130 are temporarily stored. The received image data storage unit 320 to display the video A display unit 360, and a display image generating unit 370 Metropolitan for generating image data of an image to be displayed on the display unit 360.
In the present embodiment, the communication interface unit 130 of the communication terminal device 30 will be described as having a wireless communication interface function for performing data transmission / reception with the base station Y using a wireless band signal. Moreover, the communication terminal device 30 and the communication terminal device 31 shall have the same structure and function.

  The received image data storage unit 320 is a memory device that temporarily stores image data, and temporarily stores image data included in the received data received by the communication interface unit 130. Here, the received image data storage unit 320 may store image data of an image frame in a video displayed on the display unit 360 generated based on the difference information included in the received data.

The display video generation unit 370 generates video image data to be displayed on the display unit 360 from the image data included in the reception data received by the communication interface unit 130.
Specifically, the display video generation unit 370 extracts the difference information included in the reception data received by the communication interface unit 130, and receives the extracted difference information and the reception data including the difference information immediately before the reception data. Image data of a video to be displayed on the display unit 360 based on image data (third image) generated based on the received data and the third image stored in the received image data storage unit 322. A fourth image is generated.

Here, the image data generation function of the video displayed on the display unit 360 by the display video generation unit 370 will be described in detail.
Image data a shown in FIG. 4 is image data (third image) included in received data received first, and image data b is generated based on received data received next (fourth image). Image data a is stored in the received image data storage unit 320. Since the image data included in the received data to be received next is the difference information c between the image data b and the image data a, the display video generation unit 370 extracts the difference information c included in the received data.

  The difference information c includes a comparison result between the image data a stored in the received image data storage unit 320 and the image data b which is an image to be displayed on the display unit 360, that is, information indicating a changed portion of a pixel or a pixel block. Therefore, the display video generation unit 370 generates image data b from the extracted difference information c and the image data a stored in the received image data storage unit 320, and displays the image data b on the display unit 360. Display video based on.

  Each component of the communication terminal device 30 according to the present embodiment is realized by installing a computer program (software) in a computer including a CPU (Central Processing Unit), a memory, and an interface. Various functions of the device 30 are realized by cooperation of various hardware resources of the computer and the computer program.

Next, regarding the operation sequence of the image communication system 300 including the communication terminal device 30 according to the present embodiment, the sequence diagram shown in FIG. 9 and the image information transmitted and received in the image communication system 300 shown in FIG. 10 are conceptually described. This will be described with reference to the figure.
As illustrated in FIG. 9, when image communication is started between the communication terminal device 30 and the communication terminal device 31 (S301), the communication terminal device 30 transmits transmission data including image data captured by the camera unit 110. Generate (S302) and transmit the transmission data to the communication terminal device 31 (S303). Here, in the image data shown in FIG. 10, image data included in transmission data and image data captured and output by the camera unit 110 of the communication terminal device 30 is referred to as image data A1.

  When the communication terminal device 31 receives the transmission data from the communication terminal device 30 (S304), the communication terminal device 31 extracts the image data included in the received data, stores this image data in the storage unit, and stores it in the display unit. A video based on the extracted image data is displayed (S305). Here, in the image data shown in FIG. 10, the image data stored in the storage unit of the communication terminal device 31 is referred to as image data C1.

  The communication terminal device 30 generates transmission data including image data captured by the camera unit 110 to be transmitted next (S306), and transmits the generated transmission data to the communication terminal device 31 (S307). Here, in the image data shown in FIG. 10, the image data included in the transmission data generated by the communication terminal device 30 is the difference information B1 between the image data A2 captured by the camera unit 110 and the transmitted image data A1. .

When the communication terminal apparatus 31 receives transmission data from the communication terminal apparatus 30 (S308), the communication terminal apparatus 31 extracts difference information that is image data included in the reception data, and stores the image data stored in the storage unit. A display image to be displayed on the display unit is generated from the extracted difference information. The communication terminal apparatus 31 stores the image data of the generated display image in the storage unit and displays a video based on the generated display image on the display unit (S309).
Here, in the image data shown in FIG. 10, the image data of the display image stored in the storage unit of the communication terminal device 31 is an image generated from the extracted difference information B1 and the image data C1 stored in the storage unit. Data C2.

  The communication terminal device 30 generates transmission data including image data captured by the camera unit 110 to be transmitted next (S310), and transmits the generated transmission data to the communication terminal device 31 (S311). Here, in the image data shown in FIG. 10, the image data included in the transmission data generated by the communication terminal device 30 is the difference information B2 between the image data A3 captured by the camera unit 110 and the transmitted image data A2. .

When the communication terminal apparatus 31 receives transmission data from the communication terminal apparatus 30 (S312), the communication terminal apparatus 31 extracts difference information that is image data included in the reception data, and stores the image data stored in the storage unit. A display image to be displayed on the display unit is generated from the extracted difference information. The communication terminal device 31 stores the image data of the generated display image in the storage unit and displays a video based on the generated display image on the display unit (S313).
Here, in the image data shown in FIG. 10, the image data of the display image stored in the storage unit of the communication terminal device 31 is an image generated from the extracted difference information B2 and the image data C2 stored in the storage unit. Data C3.

By repeating the above steps, video communication is performed to transmit transmission data including image data of continuous images taken by a camera or the like, extract and display image data included in the received reception data, and a communication terminal It continues until the image communication between the apparatus 30 and the communication terminal apparatus 31 is complete | finished (S314).
Here, the communication terminal device 30 is described as a transmitting device that transmits image data, and the communication terminal device 31 is described as a receiving device that receives image data. However, in order to realize bidirectional video communication, the communication terminal device 30 This is performed by simultaneously transmitting and receiving image data with the communication terminal device 31.

  As described above, according to the video communication system including the communication terminal device according to the present embodiment, of the comparison result between the image captured by the camera unit and the image captured by the camera unit immediately before this image, the pixel Alternatively, by determining the frame rate of the image data output from the camera unit based on the change amount of the pixel block or the amplitude distribution change amount in the frequency component of the image data, more detailed control of the frame rate can be performed.

  Further, based on the comparison result between the image photographed by the camera unit and the image photographed by the camera unit immediately before this image, difference information including the changed part of the image is extracted, and transmission data including this difference information is extracted. Can reduce the data amount of the image data included in the transmission data, and generate a video to be displayed from the difference information of the image data included in the reception data, thereby reducing the processing load in the image display. It can be reduced.

  Therefore, it is possible to realize the image communication that secures the quality of the communication image by controlling the frame rate according to the photographed video and reducing the processing load in the image display, and reduces the communication data amount, thereby suppressing the communication fee. It becomes possible.

  The present invention can be used for a device that performs image processing, such as a mobile phone with a built-in camera or a mobile communication terminal device, or a digital video camera or recording storage device. Furthermore, the present invention can also be used for an image communication system such as a security camera system or a remote operation system for continuously photographing a landscape in a remote place.

  DESCRIPTION OF SYMBOLS 10,20,30,31 ... Communication terminal device, 110 ... Camera part, 120 ... Camera output image storage part, 320 ... Received image data storage part, 130 ... Communication interface part, 140, 240 ... Camera image frame rate control part, 241 ... Data change partial amount comparison determination unit, 242 ... Frequency distribution comparison determination unit, 150 ... Transmission data generation unit, 360 ... Display unit, 370 ... Display video generation unit, X ... Network, Y ... Base station.

Claims (7)

A camera unit that captures an image and outputs the captured image at a predetermined frame rate;
A camera output image storage unit for temporarily storing an image output by the camera unit;
A communication interface for transmitting and receiving data;
A first image that is an image output by the camera unit is compared with a second image that is output from the camera unit and stored in the camera output image storage unit immediately before the first image, A camera image frame rate control unit for determining the frame rate based on a result of the comparison;
A communication terminal apparatus comprising: a transmission data generation unit that generates data including difference information indicating a changed portion of the first image with respect to the second image and transmits the data to the communication interface unit. In the communication terminal device according to claim 1,
The communication terminal apparatus, wherein the camera image frame rate control unit determines the frame rate according to a change amount of a pixel block of the first image with respect to the second image. In the communication terminal device according to claim 1 or 2,
The communication terminal apparatus, wherein the camera image frame rate control unit determines the frame rate according to a difference in luminance distribution of the first image with respect to the second image. In the communication terminal device according to any one of claims 1 to 3,
A display unit for displaying video;
A received image data storage unit that temporarily stores image data included in the received data received by the communication interface unit;
The difference information included in the received data and the image data generated based on the received data received immediately before the received data including the difference information are stored in the received image data storage unit. A communication terminal device further comprising: a display video generation unit that generates fourth image data which is image data of a video to be displayed on the display unit based on image data. A camera unit that outputs captured images at a predetermined frame rate, a camera output image storage unit that stores images output by the camera unit, a communication interface unit that transmits and receives data, and a camera image that determines the frame rate In an image processing method for a communication terminal device, comprising: a frame rate control unit; and a transmission data generation unit that generates data transmitted from the communication interface unit.
A camera output image storage step for temporarily storing the image output from the camera unit in the camera output image storage unit;
A first image that is an image output from the camera unit, and a second image that is output from the camera unit and stored in the camera output image storage unit immediately before the first image are stored in the camera image. An image information comparison step to be compared by the frame rate controller;
A frame rate control step in which the camera image frame rate control unit determines the frame rate based on a comparison result of the image information comparison step;
An image processing method for a communication terminal device, comprising: a transmission data generation step in which the transmission data generation unit generates data including difference information indicating a changed portion of the first image with respect to the second image. .   An image processing program for causing a computer to execute the image processing method for a communication terminal device according to claim 5. A plurality of communication terminal devices each including at least a camera unit that captures an image and a display unit that displays the image transmit data including an image captured by the camera unit via a network and are included in the received data In an image communication system for realizing image communication by displaying an image to be displayed by the display unit,
The said communication terminal device is comprised by the communication terminal device of Claim 4, The image communication system characterized by the above-mentioned.

Images