JP2010157778A - Data transmission terminal device, data receiving terminal device, and moving image transmission system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a data transmission terminal device, a data receiving terminal device, and a moving image transmission system which can reduce an amount of data related to transmission without interrupting display with respect to a significant display area and can continuously perform switching without making a user feel a sense of discomfort at the time of the switching of display images by the cancellation of data amount reduction even if a communication state is deteriorated in the transmission of image data. <P>SOLUTION: The data transmission terminal device is provided with: a communication instrument 1012 for receiving a determination signal showing the necessity/non-necessity of data reduction determined on the basis of the communication state of moving image data which are transmitted and received to and from a receiving terminal device; an image data reduction device 1011 for generating reduced moving image data by executing the processing of the data reduction of the moving image data according to the determination signal; and the communication instrument 1012 for transmitting moving image data output by the image data reduction device 1011 to the receiving terminal device. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a data transmission terminal device, a data reception terminal device, and a moving image transmission system that transmit moving image data of a digital signal.

  Conventionally, various systems for photographing a predetermined object and generating and displaying moving image data in real time have been proposed. According to such a system, the user can perform an operation of applying some kind of treatment to the predetermined object while referring to the moving image data in real time.

  By the way, in the system as described above, when an image processing circuit that generates moving image data has a processing delay time corresponding to, for example, several frames, the situation is shown in the moving image related to the display. There will be a gap with the actual situation. In such a case, the operation of applying a treatment to the predetermined object as described above involves a great risk.

  Furthermore, when transmitting image signals via wireless communication or the like, since retransmission corresponding to transmission errors is required, it takes time to transmit and the situation that display of moving image data is delayed or cannot be transmitted completely. There also occurs a situation in which the display of moving image data becomes impossible.

  In view of these circumstances, a technique for reducing the amount of data during transmission to shorten the delay time and a technique for reducing the amount of data even if the amount of data is reduced have been proposed. ing. As a technique related to such a technique, for example, Patent Document 1 discloses the following technique.

  That is, in Patent Document 1, the user's voice data from the microphone of one terminal and the moving image data captured by the camera are compressed by the compression device, multiplexed by the multiplexing device, and stored in the data buffer. After transmission from the first communication device to the second communication device of the opposite terminal via the communication line, the received voice data and moving image data are stored in the data buffer of the opposite terminal, and then the voice data and moving image data are separated by the separation device. When the decompression device detects the occurrence of an error in the moving image data, the decompression device displays the data of the block at the same position in the frame immediately before the error block as to the image buffer. A video conference image data control system is disclosed. According to the TV conference image data control system disclosed in Patent Document 1, even if an error occurs in moving image data, the conference is continued without showing the user the disturbance of the moving image displayed on the terminal display. It becomes possible to do.

In Patent Document 2, the variable length decoding unit decodes the frame coding mode, the error detection unit detects an error in the image data during decoding by the variable length decoding unit, and the reception interval measurement unit detects the frame coding mode. The key frame reception interval (random access interval) is measured based on the switch, and the switching control unit controls the connection / disconnection of the switch based on the presence / absence of an error, the frame encoding mode, and the random access interval. An image receiving apparatus that switches displayed images is disclosed. According to the image receiving device disclosed in Patent Document 2, when an error is detected in the image data, an easy-to-view image can be displayed by autonomous control on the receiving side.
Japanese Patent Laid-Open No. 11-331795 JP 2003-284064 A

  Incidentally, the techniques disclosed in Patent Document 1 and Patent Document 2 are techniques for switching the image data to be displayed to alternative image data when transmission of the image data is delayed. Such display switching is, of course, discontinuous switching, and is unsightly for the user. Furthermore, since the image data that should be originally displayed is switched to other image data, the user cannot confirm the image data that should be originally confirmed.

  The present invention has been made in view of the above circumstances, and even when the communication state deteriorates at the time of transmission of image data, the important display area can be transmitted without interruption. Data transmission terminal device, data reception terminal device, and moving image transmission system capable of performing continuous switching to reduce the amount of data and to make the user feel uncomfortable when switching the display image when canceling the reduction of the data amount The purpose is to provide.

In order to achieve the above object, a data transmission terminal device according to the first aspect of the present invention provides:
A data transmission terminal device for transmitting moving image data,
A communication state detection unit for detecting a communication state of the moving image data transmitted / received to / from a receiving terminal device;
A data amount adjusting unit that executes data reduction processing of the moving image data according to the communication state detected by the communication state detecting unit to generate reduced moving image data;
A transmission unit that transmits image data output from the data amount adjustment unit to the reception terminal device;
It is characterized by providing.

In order to achieve the above object, a data receiving terminal device according to the second aspect of the present invention provides:
A receiving terminal device for receiving moving image data,
A communication state detection unit for detecting a communication state of the moving image data;
A determination unit that generates a determination signal instructing a data amount reduction of data related to the communication based on a detection result by the communication state detection unit;
A communication unit that transmits the determination signal generated by the determination unit to a transmission terminal device of the moving image data;
It is characterized by providing.

In order to achieve the above object, a moving image transmission system according to the first aspect of the present invention comprises:
A moving image transmission system comprising: a transmitting terminal device; and a receiving terminal device that receives moving image data transmitted from the transmitting terminal device,
The receiving terminal device
A communication state detection unit for detecting a communication state of the moving image data;
A determination unit that generates a determination signal instructing a data amount reduction of data related to the communication based on a detection result by the communication state detection unit;
A communication unit that transmits the determination signal generated by the determination unit to the transmission terminal device;
With
The transmitting terminal device
A determination signal receiving unit that receives the determination signal transmitted from the receiving terminal device;
Based on the determination signal, a data amount adjusting unit that performs data reduction processing of the moving image data to generate reduced moving image data;
A transmission unit that transmits image data output from the data amount adjustment unit to the reception terminal device;
It is characterized by providing.

  According to the present invention, even when the communication state deteriorates during transmission of image data, the amount of data related to the transmission is reduced without interrupting the display of important display areas, and the amount of data is reduced. It is possible to provide a data transmission terminal device, a data reception terminal device, and a moving image transmission system that can perform continuous switching without causing the user to feel uncomfortable at the time of switching the display image associated with the release of.

  Hereinafter, a data transmission terminal device, a data reception terminal device, and a moving image transmission system according to embodiments of the present invention will be described with reference to the drawings.

[First Embodiment]
FIG. 1A is a diagram illustrating a configuration example of a data transmission device in a moving image transmission system according to the first embodiment of the present invention. FIG. 1B is a diagram illustrating a configuration example of a data reception device in the moving image transmission system according to the first embodiment of the present invention.

  As shown in FIGS. 1A and 1B, the moving image transmission system according to the first embodiment includes a data transmission device 1001 and a data reception device 1002.

  The data transmission device 1001 includes an image data reduction device 1011 and a transmitter 1012. The image data reduction device 1011 includes an image data separation device 1021, a selection device 1022, and an image sensor (not shown). The selection device 1022 includes a relaxation device 1031 and a selector 1034. The relaxation device 1031 includes a counter 1032 and an arithmetic device 1033.

  The image data reduction device 1011 is a device that reduces the amount of image data in, for example, RGB format or YCrCb format, and performs the following processing in detail.

  The image data separation device 1021 separates the image data into luminance data and color data and outputs them to the selection device 1022. That is, when image data in the YCrCb format is input, the image data separation device 1021 separates the image data into luminance data Y and color data Cr and Cb. When image data in RGB format is input, the image data separation device 1021 first converts the image data into YCrCb format and then separates it into luminance data Y and color data Cr and Cb.

  The selection device 1022 performs the following operation according to the input determination signal.

(I) When “determination signal for instructing reduction of data amount” is input The selector 1034 selects and outputs only luminance data among the data output from the image data separation device 1021.

(Ii) When a “determination signal indicating that data amount reduction is unnecessary” is input The selector 1034 outputs both luminance data and color data among the data output from the image data separation device 1021. . That is, in this case, the amount of image data is not reduced.

  As described above, the selection device 1022 transmits color information composed of luminance data Y and color data Cr and Cb in a normal state. On the other hand, when it is desired to reduce the amount of data because the communication state deteriorates, the selection device 1022 transmits only luminance data Y (monochrome information). Therefore, when the data amount reduction processing is performed by the selection device 1022 because the communication state deteriorates, display switching between a color image and a monochrome image is performed.

  The mitigation device 1031 is a device for making display switching between a color image and a monochrome image visually more natural, and includes a counter 1032 and an arithmetic unit 1033.

  The counter 1032 starts counting up when a “determination signal indicating that data amount reduction is unnecessary” is input. Note that the counter 1032 resets the count value when a determination signal for instructing reduction of the data amount is input.

  The arithmetic unit 1033 multiplies the input color data value by a predetermined coefficient according to the count value of the counter 1032 and outputs the result. This predetermined coefficient is a coefficient having a characteristic that becomes larger as the count value of the counter 1032 increases in the range of 0 to 1.

  By the operation of the mitigation device 1031 as described above, when the display is switched from the monochrome image to the color image, the value of the color data gradually returns to a predetermined value over a certain period of time. As a result, the display switching from the black and white image to the color image becomes a visually more natural display switching, and the user is not unsightly.

  In the first embodiment, the example in which the mitigation device 1031 is provided in the data transmission device 1001 is shown. However, the image data restoration device 1014 in the data reception device 1002 may of course be provided with the mitigation device 1031.

  The transmitter 1012 is a transmitter that transmits the data output from the selection device 1022 to the data reception device 1002. The transmitter 1012 also performs a reception process of the determination signal transmitted from the data receiving device 1002. The determination signal transmitted from the data receiving device 1002 is a determination signal indicating whether or not the data amount reduction in the image data reduction device 1011 is necessary / unnecessary.

  As described above, the data transmission device 1001 is the image data obtained by digitizing the electrical signal generated by receiving the light from the image sensor (not shown), or the image data according to the determination signal for the image data. The reduced image data subjected to the process of reducing the data amount by the reduction device 1011 is transmitted to the data reception device 1002 by the transmitter 1012.

  The data reception device 1002 includes a receiver 1013, an image data restoration device 1014, a determination device 1015, and an image processing unit (not shown).

  The receiver 1013 is a receiver that receives the image data / reduced image data transmitted by the data transmitting apparatus 1001.

  The image data restoration device 1014 holds the image data / reduced image data received by the receiver 1013 and uses it when switching the display. The image data / reduced image data is displayed by display means (not shown) after being subjected to predetermined image processing by an image processing unit (not shown).

  The determination device 1015 determines a communication state from the deterioration state of the reduced image data, generates a determination signal based on the determination result, and outputs the determination signal to the receiver 1013. The determination signal is transmitted to the data transmission device 1001 by the receiver 1013. Specifically, the determination device 1015 operates as follows.

(When it is determined that the communication status is good)
The determination device 1015 generates a “determination signal indicating that data amount reduction is unnecessary”.

(If it is determined that the communication status is not good)
The determination device 1015 generates a “determination signal for instructing reduction of the data amount”.

  As a method for determining the communication state, specifically, for example, a determination method using error detection using an error correction code, or detection of information such as the fact that image data could not be transmitted for a frame due to repeated retransmissions. The determination method by can be mentioned.

  As described above, according to the first embodiment, even when the communication state deteriorates during transmission of image data, data related to the transmission is displayed without interrupting display for important display areas. Provided are a data transmission terminal device, a data reception terminal device, and a moving image transmission system that can reduce the amount and can perform continuous switching without causing the user to feel uncomfortable at the time of switching the display image due to the cancellation of the data amount reduction. be able to.

  In the first embodiment, the determination signal is generated by the determination device 1015. However, the determination signal must not be generated based on the data transmitted to the data reception device 1002. Accordingly, it is needless to say that the data transmission apparatus 1001 may be separately provided with a device for generating a determination signal.

[Second Embodiment]
FIG. 2A is a diagram illustrating a configuration example of a data transmission device in the moving image transmission system according to the second embodiment of the present invention. FIG. 2B is a diagram illustrating a configuration example of the data receiving device in the moving image transmission system according to the second embodiment of the present invention.

  As illustrated in FIGS. 2A and 2B, the moving image transmission system according to the second embodiment includes a data transmission device 2001 and a data reception device 2002.

  The data transmission device 2001 includes an image data reduction device 2011 and a transmitter 2012. The image data reduction device 2011 includes an image data separation device 2021, a selection device 2022, and an image sensor (not shown). The selection device 2022 includes a relaxation device 2031, a selector 2034, and an anomaly selection device 2035. The relaxation device 2031 includes a counter 2032 and an arithmetic device 2033.

  In detail, the image data reduction device 2011 performs the following processing.

  The image data separation device 2021 separates input image data according to a display area, and outputs image data generated by the separation processing. Specifically, the image data separation device 2021 separates the image data into, for example, a central area and a peripheral area in the display area.

  In the image data separation process, in addition to the separation process for the central area and the peripheral area as described above, the separation process may be performed for the right area and the left area. Of course, the upper region and the lower region may be separated. Further, the format of the image data is not particularly specified.

  Although details will be described later, the image data separated by the image data separation device 2021 is determined as important region image data or non-important region image data by the determination device 2105 of the data reception device 2002. Then, the determination device 2105 generates and outputs a second determination signal indicating the determination result. The second determination signal is transmitted to the data transmission device 2001 by the receiver 2013, received by the transmitter 2012, and input to the image data reduction device 2021.

  Further, the determination device 2015 determines the communication state from the degradation state of the image data, and generates and receives a determination signal indicating the necessity / unnecessity of the data amount reduction in the image data reduction device 2011 based on the determination result. To the machine 2013. The determination signal is transmitted to the data transmission device 2001 by the receiver 2013, received by the transmitter 2012, and input to the image data reduction device 2021. Specifically, the determination device 2015 operates as follows.

(When it is determined that the communication status is good)
The determination device 2015 generates a determination signal indicating that it is not necessary to reduce the amount of data.

(If it is determined that the communication status is not good)
The determination device 2015 generates a determination signal for instructing the reduction of the data amount.

  As a method for determining the communication state, specifically, for example, a determination method using error detection using an error correction code, or detection of information such as the fact that image data could not be transmitted for a frame due to repeated retransmissions. The determination method by can be mentioned.

  Hereinafter, the operation of the selection device 2022 will be described in detail. The selection device 2022 performs the following operation according to the input determination signal.

(I) When “determination signal for instructing reduction of data amount” is input The selector 2034 is based on a second determination signal among the image data separated by the image data separation device 2021. Then, only the data determined as the important area image data by the determination device 2015 is selected and output as reduced image data.

(Ii) When a “determination signal indicating that data amount reduction is unnecessary” is input The selector 2034 outputs the input image data without reducing it. That is, the selection device 2022 outputs both important area image data and non-important area image data.

  By the above operation, all image information is normally transmitted (transmitted without reducing image data), but when it is necessary to reduce the amount of data related to transmission due to deterioration of communication status, etc. Only the important display area of the image data is transmitted and displayed in real time.

  By the way, when the data amount reduction processing is performed by the selection device 2022 due to deterioration of the communication state and the communication state returns to a good state after that, only the important region image is displayed. It is necessary to switch display from an image to a normal display image. In the second embodiment, the following display switching is performed so that the user is not unsightly when switching the display.

  The relaxation device 2031 is a device for making the above-described display switching visually more natural display switching, and includes a counter 2032 and an arithmetic unit 2033.

  The counter 2032 starts counting up when a “determination signal indicating that data amount reduction is unnecessary” is input. Note that the counter 2032 resets the count value when the “determination signal for instructing reduction of the data amount” is input.

  The arithmetic unit 2033 multiplies the value of the input non-important region image data by a predetermined coefficient according to the count value of the counter 2032 and outputs the result. The predetermined coefficient is a coefficient having a characteristic that becomes larger as the count value of the counter 2032 increases in the range of 0 to 1.

  By the above-described operation by the relaxation device 2031, when the display is switched to restore the non-important region image, the display image is such that the image of the non-important region that has been gradually (visually naturally) reduced emerges.

  In the second embodiment, an example in which the mitigation device 2031 is provided in the data transmission device 2001 is shown. However, the image data restoration device 2014 in the data reception device 1002 may of course be provided with the mitigation device 2031.

  The transmitter 2012 is a transmitter that transmits data output from the selection device 2022 to the data reception device 2002. The transmitter 2012 also performs reception processing of the determination signal transmitted from the data reception device 2002. Note that the determination signal transmitted from the data receiving device 2002 is a determination signal indicating whether or not the data amount reduction in the image data reduction device 2011 is necessary / unnecessary.

  As described above, the data transmission device 2001 has the image data obtained by digitizing the electrical signal generated by receiving the light from the image sensor (not shown), or the determination signal and the second determination for the image data. The reduced image data subjected to the process of reducing the data amount by the image data reduction device 2011 according to the signal is transmitted to the data reception device 1002 by the transmitter 2012.

  The data reception device 2002 includes a receiver 2013, an image data restoration device 2014, a determination device 2015, and an image processing unit (not shown).

  The receiver 2013 is a receiver that receives the reduced image data transmitted by the data transmission device 2001. The receiver 2013 transmits the determination signal and the second determination signal output from the determination device 2015 to the data transmission device 2001.

  The image data restoration device 2014 holds the image data / reduced image data received by the receiver 2013 and uses it when switching the display. The image data / reduced image data is displayed by display means (not shown) after being subjected to predetermined image processing by an image processing unit (not shown).

  The determination device 2015 generates and outputs a determination signal and a second determination signal. The determination signal generation process is as described above. Hereinafter, the generation process of the second determination signal will be described in detail.

  The determination device 2015 determines each image data separated by the image data separation device 2021 as important region image data or non-important region image data based on, for example, a luminance level, a change amount, and the like. A second determination signal indicating the result is generated.

  More specifically, for example, when the luminance data is uniformly distributed in the vicinity of the minimum value or maximum value that can be taken, there is a possibility that blackout or whiteout has occurred in the area, and the area Can be determined to be a non-important region.

  In addition, a storage unit that stores an important image pattern in advance may be provided, and the important area may be determined by performing matching with the important image pattern. Furthermore, noise detection may be performed and it may be determined that a region including a lot of noise is not an important region.

  Further, it is not always necessary to determine the important region based on the image information. For example, in a case where a tubular portion such as an esophagus is photographed like an endoscope, the tubular portion is photographed in the direction of the central axis of the tube. In such a case, the peripheral portion of the display image can be determined as a non-important region. In this way, a method of determining an important area according to the photographing position can be mentioned.

  In the second embodiment, the second determination signal is generated by the determination device 2015. However, the second determination signal must be generated based on the data transmitted to the data reception device 2002. is not. Therefore, it is needless to say that the data transmission apparatus 2001 may be separately provided with a device for generating the second determination signal.

  As described above, according to the second embodiment, even when the communication state deteriorates during transmission of image data, data related to the transmission is displayed without interrupting display for important display areas. It is possible to provide a moving image transmission system that can reduce the amount and can perform continuous switching without causing the user to feel uncomfortable at the time of switching the display image due to the cancellation of the data amount reduction.

  The data reduction method described above is a method for temporarily and completely erasing a part of image data (non-important area data) related to transmission when the communication state is poor. However, part of the image data (non-important area data) can be kept displayed at a reduced frame rate without being completely lost.

(Modification)
In this modification, when the “determination signal for instructing reduction of the data amount” is input, the irregular selection device 2035 outputs not only important area image data but also non-important area image data as reduced image data. Output every predetermined cycle.

  By this operation, the important area image data is transmitted from the data transmission apparatus 2001 to the data reception apparatus 2002 and displayed in real time for the important area image data and at a predetermined cycle.

  The image data restoration device 2014 stores, in the memory 2036, non-important area image data received at predetermined intervals when a “determination signal for instructing reduction of the data amount” is input. When the non-important area image data is not included in the reduced image data, the latest non-important area image data stored in the memory 2036 is used.

  As described above, in the moving image transmission system according to the present modification, when the communication state is poor, the important area is transmitted / displayed in real time, and the non-important area is dropped / thinned and transmitted / displayed. .

[Third Embodiment]
FIG. 3A is a diagram illustrating a configuration example of a data transmission terminal device in a moving image transmission system according to a third embodiment of the present invention. FIG. 3B is a diagram illustrating a configuration example of a data reception terminal device in the moving image transmission system according to the third embodiment of the present invention.

  As shown in FIGS. 3A and 3B, the moving image transmission system according to the third embodiment includes a data transmission device 3001 and a data reception device 3002.

  The data transmission device 3001 includes an image data reduction device 3011 and a transmitter 3012. The image data reduction device 3011 includes an LPF (low pass filter) 3021, an image compression device 3022, a relaxation device 3031, and an image sensor (not shown). The relaxation device 3031 includes a counter 3032 and an arithmetic device 3033.

  The image data reduction device 3011 is a device that reduces the amount of image data in RGB format or YCrCb format, for example, and generates and outputs reduced image data by the processing by the LPF 3021 and the processing by the image compression device 3022.

  The LPF 3021 multiplies the input image data by an LPF coefficient input from a relaxation device 3031 described later and outputs the result.

  The image compression apparatus 3022 compresses image data by an image compression method such as JPEG. Note that image data from which high-frequency components have been cut by the LPF 3021 can be compressed at a high compression rate.

  The relaxation device 3031 is a device for making display switching, which will be described later, visually more natural, and includes a counter 3032 and an arithmetic device 3033.

  The counter 3032 starts counting up when a “determination signal indicating that data amount reduction is unnecessary” is input. Note that the counter 3032 resets the count value when a “determination signal for instructing reduction of the data amount” is input.

  The arithmetic unit 3033 sets the LPF coefficient according to the input determination signal and the count value of the counter 3032 and outputs the LPF coefficient to the LPF 3021.

(I) When “determination signal for instructing reduction of data amount” is input The arithmetic unit 3033 sets a coefficient of a value that cuts a high-frequency component of image data and outputs the coefficient to the LPF 3021.

(Ii) When “determination signal indicating that data amount reduction is not required” is input The arithmetic unit 3033 sets a coefficient for outputting the input image data as it is and outputs the coefficient to the LPF 3021.

  That is, the LPF 3021 outputs the input image data as it is in the normal state to the image compression device 3022, and when it is necessary to reduce the amount of data due to deterioration of the communication state or the like, the reduced image data in which the high frequency component is cut off. Is generated and output to the image compression apparatus 3022.

  By the way, when the data amount reduction processing is performed by the image data reduction device 3011 due to deterioration of the communication state and the communication state returns to a good state after that, the compression rate is high. Display switching from reduced image data that has been subjected to compression processing to normal image data that is not subjected to compression processing must be performed. In the third embodiment, the following display switching is performed so that the user is not unsightly when switching the display.

  That is, the arithmetic unit 3033 sets the LPF coefficient so that the high frequency component of the image data is gradually changed from the characteristic of cutting the image data to the characteristic of not cutting as the count value of the counter 3032 increases. With this operation, display switching from image data that has been compressed at a high compression rate due to deterioration of the communication state or the like to normal image data that is not subjected to compression processing is gradually (visually naturally) performed.

  In the third embodiment, an example in which the mitigation device 3031 is provided in the data transmission device 3001 has been described. However, the mitigation device 3031 may be provided in the image data restoration device 3014 in the data reception device 3002.

  The transmitter 3012 is a transmitter that transmits the data output from the image compression device 3022 to the data receiving device 3002. The transmitter 3012 also performs reception processing of the determination signal transmitted from the data reception device 3002. Note that the determination signal transmitted from the data receiving device 3002 is a determination signal indicating whether or not the data amount reduction in the image data reduction device 3011 is necessary.

  As described above, the data transmission device 3001 is the image data obtained by digitizing the electrical signal generated by receiving the light from the image sensor (not shown), or the image data according to the determination signal for the image data. The reduced image data subjected to the process of reducing the data amount by the reduction device 3011 is transmitted to the data reception device 3002 by the transmitter 3012.

  The data reception device 3002 includes a receiver 3013, an image data restoration device 3014, a determination device 3015, and an image processing unit (not shown).

  The receiver 3013 is a receiver that receives the reduced image data transmitted by the data transmission device 3001. The receiver 3013 transmits the determination signal output from the determination device 3015 to the data transmission device 3001. Note that the image data / reduced image data received by the receiver 3013 is subjected to predetermined image processing by an image processing unit (not shown) and then displayed by display means (not shown).

  The image data restoration device 3014 performs image expansion processing corresponding to the image compression method on the image data compressed by the image compression device 3022. The image data restoration device 3014 holds the image data / reduced image data received by the receiver 3013, and uses it when switching the display.

  The determination device 3015 determines the communication state from the deterioration state of the reduced image data, generates a determination signal based on the determination result, and outputs the determination signal to the receiver 3013. The determination signal is transmitted to the data transmission device 3001 by the receiver 3013. Specifically, the determination device 3015 operates as follows.

(When it is determined that the communication status is good)
The determination device 3015 generates a “determination signal indicating that data amount reduction is unnecessary”.

(If it is determined that the communication status is not good)
The determination device 3015 generates a “determination signal for instructing reduction of the data amount”.

  As a method for determining the communication state, specifically, for example, a determination method using error detection using an error correction code, or detection of information such as the fact that image data could not be transmitted for a frame due to repeated retransmissions. The determination method by can be mentioned.

  As described above, according to the third embodiment, even when the communication state deteriorates during transmission of image data, data related to the transmission is displayed without interrupting display for important display areas. It is possible to provide a moving image transmission system that can reduce the amount and can perform continuous switching without causing the user to feel uncomfortable at the time of switching the display image due to the cancellation of the data amount reduction.

(Appendix)
The invention having the following configuration can be extracted from the specific embodiment.

(1) An image data reduction device that generates and outputs reduced image data by reducing the amount of image data, and transmission of the reduced image data and reception of a determination signal that indicates whether or not reduction of the data amount is necessary A data transmission device comprising:
A data receiving device comprising: a receiver that receives the reduced image data and transmits the determination signal; and a determination device that generates and outputs the determination signal according to a communication state;
Comprising
The image data reduction device is an image data separation device that separates image data into luminance data and color data, and selects at least one of luminance data and color data based on the determination signal as reduced image data. A moving image transmission system comprising: a selection device that outputs the moving image;

(Corresponding embodiment)
The embodiment relating to the moving image transmission system described in (1) corresponds to the first embodiment.

(Function and effect)
When the communication state deteriorates, the amount of data related to transmission can be reduced by transmitting only luminance data, and the chance of data retransmission can be increased.

(2) An image data reduction device that generates and outputs reduced image data by reducing the data amount of the image data, and transmission of the reduced image data and reception of a determination signal that indicates whether or not reduction of the data amount is necessary A data transmission device comprising:
A receiver that receives the reduced image data and transmits the determination signal; generates the determination signal according to a communication state; and indicates a reduction part in the field area according to an imaging target of the image data A data receiving device comprising: a determination device that generates a second determination signal;
Comprising
An image data reduction device includes: an image data separation device that separates image data into central data indicating a central portion of the object scene and peripheral data indicating a peripheral portion of the object field; and the determination signal and the second determination signal And a selection device that generates and outputs the reduced image data based on the video image transmission system.

(Corresponding embodiment)
The embodiment relating to the moving image transmission system described in (2) corresponds to the second embodiment.

(Function and effect)
When the communication state deteriorates, the amount of data related to transmission can be reduced by transmitting only the central part of the scene of the image data or only the peripheral part of the scene, and the chance of retransmission can be increased.

  (3) The image data reduction device gradually executes a restoration process from reduced image data to image data when the determination signal generated by the determination device is an instruction determination signal not to reduce image data. The moving image transmission system according to (1) or (2), further comprising a display switching mitigation device.

(Corresponding embodiment)
The embodiment relating to the moving image transmission system described in (3) corresponds to the first embodiment and the second embodiment.

(Function and effect)
When the bad communication state becomes good and a part of the reduced image data starts to be transmitted again, the display switching mitigation device gradually reflects the data related to the transmission on the display. In other words, when the reduction of the image data is canceled, the display image is prevented from changing suddenly, so that a display that is not unsightly for the user is realized.

(4) a counter that starts counting from when the instruction content of the determination signal changes;
An arithmetic device for performing an operation of gradually changing the value of the image data to a predetermined value in accordance with the count value of the counter;
The moving image transmission system according to (3), further comprising:

(Corresponding embodiment)
The embodiment related to the moving image transmission system described in (4) corresponds to the first embodiment and the second embodiment.

(Function and effect)
When the determination signal changes, the counter starts counting, and the calculation device performs a predetermined calculation on the image data according to the count value of the counter. That is, it is possible to measure the elapsed time after the change of the determination signal from the count value of the counter and gradually change the value of the image data to a predetermined value according to the elapsed time.

(5) The image data reduction device may store both the central data and the peripheral data at a predetermined frame interval in a situation where only the central data or the peripheral part is selected as reduced image data. Equipped with an anomaly selection device to output,
The moving image transmission system according to (2), wherein the data receiving device includes a memory that stores data that is not selected as reduced image data among central data and peripheral data.

(Corresponding embodiment)
The embodiment relating to the moving image transmission system described in (2) corresponds to the second embodiment.

(Function and effect)
In the situation where the central data or the peripheral data is selected as the reduced image data, the data of both the central data and the peripheral data is provided at a predetermined frame interval by the anomaly selecting device provided in the image data reducing device. Is output from the image data reduction device. The memory provided in the data receiving device stores the data not selected as the reduced image data among the central data and the peripheral data. That is, all image data is transmitted at a rate of once every several frames even in a situation where the communication state is deteriorated. For data that is not transmitted every frame, the data at the time of transmission continues to be displayed.

(6) An image data reduction device that outputs the reduced image data by reducing the data amount of the image data, and a transmission that transmits the reduced image data and a determination signal that indicates whether or not the reduction of the data amount is necessary. A data transmission device comprising:
A data receiving device comprising: a receiver that receives the reduced image data and transmits the determination signal; and a determination device that generates the determination signal based on a communication state;
Comprising
The image data reduction device includes a low-pass filter that removes high-frequency components of image data based on the determination signal, and an image compression device that performs image compression processing according to the spatial frequency of the image data,
The data reception apparatus includes an image expansion device corresponding to the image compression device.

(Corresponding embodiment)
The embodiment relating to the moving image transmission system described in (6) corresponds to the third embodiment.

(Function and effect)
A determination device provided in the data receiving device determines a communication state based on the received data, generates a determination signal, and transmits the determination signal to the data transmitting device. In the data transmitting apparatus, the image data is reduced by the low-pass filter based on the determination signal, and the reduced image data in which the image compression according to the spatial frequency is further performed by the image compressing apparatus is transmitted to the data receiving apparatus by the transmitter. Is transmitted to. In the data receiving device, the reduced image data is decompressed and restored in the image data restoring device. In other words, when the communication state deteriorates, image data that can be subjected to high compression processing by functioning a low-pass filter can reduce the amount of data related to transmission and increase the chance of retransmission.

  (7) The image data reduction device gradually executes restoration processing from reduced image data to image data when the determination signal generated by the determination device is an instruction determination signal that does not reduce image data. The moving image transmission system according to (6), further comprising a display switching mitigation device.

(Corresponding embodiment)
The embodiment relating to the moving image transmission system described in (7) corresponds to the third embodiment.

(Function and effect)
When the bad communication state becomes good, the coefficient of the low-pass filter gradually changes by the display switching mitigation device. In other words, when the reduction of the image data is canceled, the display image is prevented from changing suddenly, so that a display that is not unsightly for the user is realized.

(8) The display switching mitigation device includes:
A counter that starts counting from when the instruction content of the determination signal changes;
An arithmetic unit that determines a coefficient of a low-pass filter in accordance with a count value of the counter;
The moving image transmission system according to (7), further comprising:

(Corresponding embodiment)
The embodiment relating to the moving image transmission system described in (8) corresponds to the third embodiment.

(Function and effect)
When the bad communication state becomes good, the coefficient of the low-pass filter gradually changes by the display switching mitigation device. In other words, when the reduction of the image data is canceled, the display image is prevented from changing suddenly, so that a display that is not unsightly for the user is realized.

  The present invention has been described based on the first to third embodiments. However, the present invention is not limited to the above-described embodiments, and various modifications and applications can be made within the scope of the gist of the present invention. Of course, it is possible.

  Further, the above-described embodiments include inventions at various stages, and various inventions can be extracted by appropriately combining a plurality of disclosed constituent elements. For example, even if some constituent requirements are deleted from all the constituent requirements shown in the embodiment, the problem described in the column of the problem to be solved by the invention can be solved, and the effect described in the column of the effect of the invention Can be extracted as an invention.

It is a figure which shows the example of 1 structure of the data transmitter in the moving image transmission system which concerns on 1st Embodiment of this invention. It is a figure which shows the example of 1 structure of the data receiver in the moving image transmission system which concerns on 1st Embodiment of this invention. It is a figure which shows one structural example of the data transmission device in the moving image transmission system which concerns on 2nd Embodiment of this invention. It is a figure which shows one structural example of the data receiver in the moving image transmission system which concerns on 2nd Embodiment of this invention. It is a figure which shows the example of 1 structure of the data transmitter in the moving image transmission system which concerns on 3rd Embodiment of this invention. It is a figure which shows the example of 1 structure of the data receiver in the moving image transmission system which concerns on 3rd Embodiment of this invention.

Explanation of symbols

    DESCRIPTION OF SYMBOLS 1001 ... Data transmission apparatus, 1002 ... Data reception apparatus, 1011 ... Image data reduction apparatus, 1012 ... Transmitter, 1013 ... Receiver, 1014 ... Image data restoration apparatus, 1015 ... Determination apparatus, 1021 ... Image data separation apparatus, 1022 ... Selection device, 1031 ... Mitigation device, 1032 ... Counter, 1033 ... Arithmetic device, 1034 ... Selector, 2001 ... Data transmission device, 2002 ... Data reception device, 2011 ... Image data reduction device, 2012 ... Transmitter, 2013 ... Receiver, 2014 ... Image data restoration device, 2015 ... Determination device, 2021 ... Image data separation device, 2022 ... Selection device, 2031 ... Mitigation device, 2032 ... Counter, 2033 ... Arithmetic device, 2034 ... Selector, 2035 ... Anomaly selection device, 2036 ... Mori, 2105 ... determination device, 3001 ... data transmission device, 3002 ... data reception device, 3011 ... image data reduction device, 3012 ... transmitter, 3013 ... receiver, 3014 ... image data restoration device, 3015 ... determination device, 3021 ... LPF, 3022 ... image compression device, 3031 ... relaxation device, 3032 ... counter, 3033 ... arithmetic device.

Claims (10)

A data transmission terminal device for transmitting moving image data,
A communication state detection unit for detecting a communication state of the moving image data transmitted / received to / from a receiving terminal device;
A data amount adjusting unit that executes data reduction processing of the moving image data according to the communication state detected by the communication state detecting unit to generate reduced moving image data;
A transmission unit that transmits image data output from the data amount adjustment unit to the reception terminal device;
A data transmission terminal device comprising:   The data amount adjustment unit includes a data extraction unit that extracts predetermined data from the moving image data as a main data unit, and a communication state detected by the communication state detection unit is worse than a predetermined communication state The data transmission terminal device according to claim 1, wherein thinning processing is performed on data that has not been extracted by the extraction unit in the moving image data.   The data extraction unit extracts data obtained by removing color difference data from the moving image data as the main data unit, or data corresponding to a predetermined area in a display image of the moving image data. The data transmission terminal device according to claim 2, wherein the data transmission terminal device is extracted as   When the communication state detected by the communication state detection unit transitions from a communication state worse than the predetermined communication state to a state better than the predetermined communication state, the data amount adjustment unit 3. The data transmission terminal device according to claim 2, wherein a reduction amount in the data reduction processing of the image data is gradually reduced. A correction unit that corrects data that has not been extracted as the main data unit by the extraction unit from the moving image data,
5. The data amount adjustment unit according to claim 4, wherein the data amount adjustment unit outputs the main data unit and the corrected data when the reduction amount in the data reduction processing of the moving image data is gradually reduced. Data transmission terminal device. The data amount adjustment unit includes:
A low-pass filter for removing a predetermined high-frequency component in the image data;
A compression processing unit that compresses image data from which a predetermined high-frequency component has been removed by the low-pass filter;
The data transmission terminal device according to claim 1, comprising: The data amount adjustment unit includes a filter setting unit that sets a frequency component to be removed by the low-pass filter,
The filter setting unit is configured to remove the frequency when the communication state detected by the communication state detection unit transitions from a communication state worse than the predetermined communication state to a better state than the predetermined communication state. The data transmission terminal device according to claim 6, wherein the value of the component is gradually decreased.   The data amount adjustment unit outputs only the main data part extracted by the data extraction unit when the communication state detected by the communication state detection unit is worse than the predetermined communication state. The data transmission terminal device according to claim 1. A receiving terminal device for receiving moving image data,
A communication state detection unit for detecting a communication state of the moving image data;
A determination unit that generates a determination signal instructing a data amount reduction of data related to the communication based on a detection result by the communication state detection unit;
A communication unit that transmits the determination signal generated by the determination unit to a transmission terminal device of the moving image data;
A data receiving terminal device comprising: A moving image transmission system comprising: a transmitting terminal device; and a receiving terminal device that receives moving image data transmitted from the transmitting terminal device,
The receiving terminal device
A communication state detection unit for detecting a communication state of the moving image data;
A determination unit that generates a determination signal instructing a data amount reduction of data related to the communication based on a detection result by the communication state detection unit;
A communication unit that transmits the determination signal generated by the determination unit to the transmission terminal device;
With
The transmitting terminal device
A determination signal receiving unit that receives the determination signal transmitted from the receiving terminal device;
Based on the determination signal, a data amount adjusting unit that performs data reduction processing of the moving image data to generate reduced moving image data;
A transmission unit that transmits the image data output from the data amount adjustment unit to the reception terminal device;
A moving image transmission system comprising:

Images