JP2010011287A - Image transmission method and terminal device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image transmission method allowing a user of one-side terminal device to appropriately and rapidly confirm that an image transmitted from the one-side terminal device is surely received by the other-side terminal device. <P>SOLUTION: In this image transmission method, a camera 100 transmits image data and a server 300 receives main image data thereof (S108); and the server 300 generates proxy image data corresponding to the received image data (S110), detects a delay time of data transmission/reception (S106), adjusts the data volume thereof so that the longer the detected delay time is, the more the data volume of the generated proxy image data is reduced (S110), and transmits the proxy image data to the camera 100; and the camera 100 receives the proxy image data from the server 300 (S112). <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a video transmission method in which a terminal device that records video such as a video camera transmits the recorded video to another terminal device at a remote location via a network.

  In recent years, wireless network technologies such as mobile phones and WiMAX (Worldwide Interoperability for Microwave Access) are rapidly spreading. Using such a wireless network, digital data such as video shot by a digital camera is transmitted and received. In such a data transmission technique using a wireless network, it is common to perform transmission / reception by packetizing data so that an IP (Internet Protocol) technique can be cited as a representative example. Although packetization is an effective means of efficiently sharing the network bandwidth among multiple users, the transmission rate varies depending on the sharing situation at that time, resulting in packet arrival delay and packet loss. there is a possibility. As a result, the transmitted video is delayed or disturbed.

  Therefore, a mechanism has been proposed in which a communication abnormality on the network is detected, and the state of the transmission video transmitted from one terminal device and actually reproduced on the other terminal device is reproduced and displayed on the one terminal device. (For example, refer to Patent Document 1).

  Specifically, in the video transmission method disclosed in Patent Document 1, two terminal devices capable of capturing, playing back, and transmitting and receiving video are used. Each of these terminal devices transmits a video obtained by photographing to the partner terminal device, and receives and reproduces the video transmitted from the partner terminal device. Further, each terminal device reproduces a video obtained by photographing with its own terminal device as well as a video transmitted from the partner terminal device for confirmation. At that time, each terminal device sends information indicating disturbance on the received video, which is caused by the deterioration of the network state, to the partner terminal device. When each terminal device receives the information from the partner terminal device, each terminal device reflects the disturbance or the like indicated by the received information in the reproduction of the confirmation video obtained by shooting with its own terminal device. .

By doing so, each terminal device can reproduce and display the video transmitted from its own terminal device and reproduced by the other terminal device in real time on its own terminal device. Furthermore, the user of each terminal device can confirm each other's network state in real time.
JP 2007-150916 A

  However, in the video transmission method disclosed in Patent Document 1, the user of one terminal device cannot appropriately confirm whether the video transmitted from the one terminal device is reliably received by the other terminal device. There is a problem.

  That is, in the video transmission method of Patent Document 1, one terminal device reproduces a video obtained by shooting for confirmation, but the video is the video itself obtained by the one terminal device. Thus, the video is not actually received by the other terminal device. In addition, the confirmation video reflects disturbance based on the above information, but even if the disturbance is reflected, the confirmation video is not actually received by the other terminal device. It is a simulated video of the video received by the other terminal device.

  Thus, for example, a video transmission method is assumed in which a video transmitted from one terminal device and received by the other terminal device is sent back from the other terminal device to the one terminal device. In this video transmission method, one terminal device receives the video received and sent back by the other terminal device, and reproduces it as a confirmation video. Therefore, the user of one terminal device can confirm that the video obtained by photographing with the one terminal device is surely received by the other terminal device.

  However, even with such a video transmission method, there is a problem in that there is a delay in confirming whether or not the other terminal device is receiving.

  Such a problem will be described below on the assumption that one terminal device described above is a video camera and the other terminal device is a server.

  In general, when a video camera records video, video data representing the video is converted into a file and recorded in a random accessible manner such as a DVD-R (Digital Versatile Disk Recordable) or SD (Secure Digital) (registered trademark) card. Record to media. Such a video camera, when transmitting the video, reads a video file, which is filed video data, from the recording medium and transmits it.

  However, in general, when a server uses a video file, if all the data originally stored in the video file is not received, the video file is received only halfway or the video file is damaged. The video file cannot be used. In particular, the file format includes MXF (Material eXchange Format) conforming to SMPTE-377M, and the file format is stored in the header or footer at the beginning or end of the file. It contains information such as the type of data, frame frequency, and data size. Therefore, if transmission of a video file is interrupted due to packet loss or transmission interruption, the server cannot obtain the information contained in the header or footer and cannot determine the data type of the video file. obtain.

  On the other hand, as described above, in the wireless network, the transmission rate fluctuates, so that packet arrival delay and packet loss inevitably occur.

  Therefore, in a video transmission method in which video captured by the video camera is transmitted to the server, and the server receives the video and sends it back to the video camera, the transmission rate decreases due to the large amount of data in the video file to be transmitted and received. This can easily cause packet delay and loss. Furthermore, as described above, if a packet loss occurs, the video file cannot be used, so that the lost packet needs to be retransmitted. As a result, the packet delay, that is, the confirmation video reproduced by the video camera is further delayed. In other words, in the video camera, the confirmation moving image sent back from the server is reproduced with a large delay with respect to the captured moving image.

  As a result, the video camera user waits until the reproduction of the confirmation video ends, although the video camera has completed the shooting and transmission of the video.

  Furthermore, in the television broadcasting and video production industries, high-rate video data with a bit rate exceeding 50 Mbps, such as MPEG-2 (ISO / IEC 13818) and DVCPRO-HD (SMPTE-370M), is handled. It takes a very long time to send and check video data.

  Such a problem is, for example, that a video camera user wants to send a news video shot by the video camera to a broadcasting station as soon as possible using a wireless network, and start the next shooting. If you get serious.

  Therefore, the present invention has been made in view of such a problem, and the user of one terminal device appropriately determines whether the video transmitted from one terminal device is reliably received by the other terminal device. It is another object of the present invention to provide a video transmission method that can be confirmed quickly.

  In order to achieve the above object, a video transmission method according to the present invention is a video transmission method for transmitting and receiving data relating to a video between a first and a second terminal device, wherein the first terminal device is the first terminal device. A main video transmission step of transmitting main video data indicating the main video to the second terminal device; a main video reception step of receiving the main video data from the first terminal device by the second terminal device; A sub-picture generating step of generating sub-picture data indicating a sub-picture corresponding to the main video indicated by the received main video data; and the second terminal apparatus is configured to The amount of sub-picture data generated in the sub-picture generation step as the delay time detected by the second terminal device increases, Less An adjustment step of adjusting the data amount so that the second terminal device transmits the sub-video data to the first terminal device, and the first terminal device includes the first terminal device. And a sub-picture receiving step of receiving the sub-picture data from the two terminal devices. Here, in the adjustment step, the second terminal device decreases the sub video data amount as the detected delay time is longer, and transmits the sub video data transmitted in the sub video transmission step. Reduce the bit rate.

  Accordingly, since the main video data transmitted from the first terminal device is sent back as sub-video data from the second terminal device, the user of the first terminal device can send the transmitted main video data to the second terminal. It is possible to properly confirm that the device is surely receiving it. Furthermore, when the data transmission / reception delay time, that is, when the first and second terminal devices are transmitting / receiving data via the network, the network delay time is detected. Since the amount of sub-video data transmitted from the terminal device to the first terminal device is reduced, it is possible to prevent the reception of the sub-video data by the first terminal device from being delayed. Therefore, the user of the first terminal device can appropriately and quickly confirm whether the video data transmitted from the first terminal device is reliably received by the second terminal device.

  Further, in the sub video generation step, the second terminal device may generate the sub video data by reducing the resolution of the main video indicated by the main video data.

  As a result, the resolution of the main video indicated by the main video data is reduced, so that sub-video data indicating a so-called proxy video corresponding to the main video is generated. As a result, since the sub-video data has a smaller data amount than the main video data, it is possible to suppress traffic congestion due to transmission / reception of the main video data and sub-video data, and to further suppress the transmission / reception delay.

  The video transmission method further includes a delay determination step in which the second terminal apparatus determines whether or not the detected delay time is equal to or greater than a predetermined threshold value. When the second terminal apparatus determines that the delay time is equal to or greater than the threshold value in the delay determination step, the second terminal apparatus determines the data amount of the sub-picture data to be less than the threshold value. The amount of sub-picture data may be smaller than the amount of sub-picture data at that time.

  As a result, the second terminal device only needs to reduce the data amount of the sub-picture data only when the delay time is equal to or greater than the threshold value. Can be adjusted easily.

  In the delay time detecting step, the second terminal device transmits first data to the first terminal device, and notifies the first terminal device that the data has been received. Second data (for example, ACK packet) is received from the first terminal device, and the time from when the first data is transmitted to when the second data is received is measured. The delay time may be detected. For example, in the delay time detecting step, the second terminal device transmits the sub-video data generated in the sub-video generating step as the first data to the first terminal device. That is, the second terminal device transmits the sub-video data to the first terminal device in this delay time detection step to detect the delay time, and then receives the main video data again when the main video data is received again. The amount of sub-picture data to be generated corresponding to the data is adjusted according to the previously detected delay time.

  Thereby, since data is actually transmitted / received and delay time is measured, the 2nd terminal device can detect the delay time correctly.

  In the adjustment step, the second terminal apparatus reduces the data amount of the sub-picture data by thinning out any picture from a plurality of pictures constituting the received main picture data.

  As a result, since the picture is thinned out from the main video data that is the source of the sub-video data before the sub-video data is generated, the amount of data of the sub-video data is reduced after the sub-video data is generated. The processing load for generating sub-picture data can be reduced.

  The video transmission method may further include a section on the time axis of the sub-video data in which the first terminal device has an image quality of the sub-video indicated by the received sub-video data that is less than a threshold value. May be included as a low image quality section, and a recording step in which the first terminal device records section information indicating the specified low image quality section on a recording medium. For example, in the section specifying step, the first terminal device detects a bit rate of the sub video data while receiving the sub video data, and the bit rate becomes less than a rate threshold value. Is specified as the low image quality interval.

  Thereby, since the section information is recorded on the recording medium, the user of the first terminal device can confirm the low image quality section indicated by the section information. For example, even when there is a low-quality section when the first terminal device reproduces sub-picture data and displays a sub-picture, the user may not be able to fully check the low-quality section. In such a case, the user can confirm the low image quality section using the recorded section information.

  The video transmission method may further include a section reading step in which the first terminal device reads the section information from the recording medium, and a low image quality indicated by the section information read out by the first terminal device. And a section display step for displaying a main video of the section of the main video data corresponding to the section.

  Thereby, the user of the first terminal device can confirm the main video in the section of the main video data corresponding to the low image quality section. Therefore, if the main video data is recorded in the first terminal device, the user determines that the main video data should be saved because the main video of the section is important based on the confirmation result. It is possible to determine that the main video data may be deleted because the main video of the section is not important. Thus, user convenience can be improved.

  The video transmission method may further include a main video generation step of generating the main video data by photographing with the first terminal device.

  Thereby, the user of the first terminal device uses the first terminal device as a video camera, and transmits the main video data obtained by shooting with the video camera to a server or the like as the second terminal device. can do.

  The video transmission method further includes a video display step in which the first terminal device displays a main video of the main video data generated by shooting and a sub-video of the received sub-video data. This may be a feature.

  As a result, the user of the first terminal device can view the main video and the sub-video in comparison, and simultaneously confirms the video being shot and the video received by the second terminal device. be able to. Furthermore, the user can easily grasp how much the sub-video is delayed from the main video. That is, the user can easily grasp the progress status of the main video data received by the second terminal device by viewing the sub-video displayed on the first terminal device.

  The present invention can be realized not only as such a video transmission method but also as a terminal device used in the method, an integrated circuit, a method of transmitting / receiving data by the terminal device, and transmitting / receiving data by the method. The present invention can also be realized as a program for storing the program and a storage medium for storing the program.

  In the video transmission method according to the present invention, the user of one terminal device can appropriately and quickly confirm whether the video transmitted from one terminal device is reliably received by the other terminal device. There is an operational effect.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a diagram showing a configuration of a video transmission system according to an embodiment of the present invention.

  The video transmission system 10 includes a camera 100 and a server 300 that communicate with each other via a wireless network 400.

  The camera 100 generates video data indicating a moving image by shooting, and records the video data on a recording medium. Here, the video data is composed of a plurality of encoded pictures. That is, every time the above-described picture constituting the video data is generated, the camera 100 sequentially records the picture on the recording medium.

  Hereinafter, the moving image indicated by the video data is referred to as a captured video. In the present embodiment, camera 100 corresponds to a first terminal device, and video data and captured video correspond to main video data and main video, respectively.

  Furthermore, the camera 100 has a wireless communication function, and transmits the generated video data to the server 300 via the wireless network 400. Specifically, every time the data unit forming the video data is generated, the camera 100 packetizes the data unit and transmits the packet to the server 300. That is, the video data is divided into a plurality of packets and transmitted to the server 300 as a stream.

  In addition, the camera 100 displays the captured video by reproducing the generated video data in real time while generating the video data. Specifically, the camera 100 sequentially reproduces the already generated pictures every time a picture constituting the video data is generated.

  Furthermore, when the camera 100 receives proxy video data (to be described later) from the server 300 via the wireless network 400, the camera 100 reproduces the proxy video data, thereby moving a moving image (hereinafter referred to as proxy video) indicated by the proxy video data. Are displayed together with the above-described captured video. Here, the proxy video data is packetized for each data unit constituting the proxy video data and transmitted to the camera 100 as a stream. That is, the camera 100 sequentially receives packets of proxy video data, and each time a picture is formed from these packets, the picture is displayed in a size smaller than the picture of the shot video.

  In the present embodiment, proxy video data and proxy video correspond to sub-video data and sub-video, respectively.

  When the server 300 receives the video data transmitted from the camera 100 via the wireless network 400, the server 300 converts the video data into proxy video data. The proxy video data is composed of a plurality of encoded pictures, and the number of pixels of the picture is smaller than the number of pixels of the picture of the video data. That is, the server 300 converts the video data into proxy video data by decoding a picture of the video data to reduce the resolution and further encoding the picture.

  Then, the server 300 packetizes the proxy video data for each predetermined data unit, and sequentially transmits each packet to the camera 100.

  In the present embodiment, server 300 corresponds to a second terminal device.

  In such a video transmission system 10, the camera 100 generates video data and transmits it to the server 300. The server 300 converts the video data into proxy video data and sends it back to the camera 100, and displays the proxy video on the camera 100. I am letting. Accordingly, the user of the camera 100 can confirm that the video data transmitted from the camera 100 is received by the server 300 by viewing the proxy video displayed on the camera 100.

  Further, the present embodiment is characterized in that the server 300 adjusts the data amount of the proxy video data and controls the bit rate, thereby preventing the proxy video display from being delayed from the captured video display. be able to.

  Hereinafter, configurations and operations of the camera 100 and the server 300 in the present embodiment will be described in detail.

  FIG. 2 is a block diagram showing a configuration of the camera 100 in the present embodiment.

  The camera 100 includes a system control unit 110, an I / O bus 120, a recording medium 131, a recording medium I / F unit 130, an input unit 140, a display unit 141, a video input unit 143, and a video encoder. 142, a video decoder 144, and a transmission / reception unit 150.

  The input unit 140 receives an instruction to start or end shooting from the user.

  The I / O bus 120 enables data input and output between the above-described components included in the camera 100.

  The transmission / reception unit 150 is connected to the wireless network 400 by WiMAX or the like, packetizes the above-described video data stored in the recording medium 131 and transmits the packetized data to the server 300, or receives packetized proxy video data from the server 300. To do. Further, the transmission / reception unit 150 outputs the proxy video data received from the server 300 to the system control unit 110.

  Further, the transmission / reception unit 150 uses, for example, TCP (Transmission Control Protocol) as a protocol for communication with the server 300, and when the proxy video data packet is correctly received from the server 300, the transmission / reception unit 150 notifies that the packet has been correctly received. ACK (ACKnowledgement) packet is transmitted to server 300.

  The recording medium 131 has a shape determined by, for example, PCMCIA (Personal Computer Memory Card International Association), and is configured to be detachable from the camera 100. That is, the recording medium 131 is connected to the recording medium I / F unit 130 by being inserted into a card slot as a recording medium mounting unit provided in the camera 100. The recording medium 131 is formatted by a file system such as FAT (File Allocation Tables) 16.

  The recording medium I / F unit 130 conforms to the card bus standard and is connected to the recording medium 131. The recording media I / F unit 130 serves as an interface between the system control unit 110 and the recording medium 131.

  The video input unit 143 includes, for example, a CCD (Charge Coupled Devices) and the like, converts an optical signal into a video signal that is an electrical signal, and outputs the video signal to the video encoder 142.

  The video encoder 142 acquires the video signal output from the video input unit 143 and compresses and encodes the video signal. As a result, the video encoder 142 outputs, for example, compressed data in the DVCPRO-HD (SMPTE-370M) format as the above-described video data.

  The video decoder 144 decodes the video data stored in the recording medium 131 and outputs the decoded video data to the display unit 141. Further, when the video decoder 144 obtains the proxy video data from the transmission / reception unit 150 via the system control unit 110, the proxy video data is also encoded. Therefore, the video decoder 144 decodes the proxy video data, and the decoded proxy Video data is output to the display unit 141.

  The display unit 141 has, for example, a liquid crystal screen. When the video data decoded from the video decoder 144 is acquired, the display unit 141 displays a captured video indicated by the video data. Further, when the display unit 141 acquires the proxy video data decoded from the video decoder 144, the display unit 141 displays the proxy video indicated by the proxy video data. Here, the display unit 141 displays the proxy video in the corner of the captured video, for example, when displaying the proxy video while displaying the captured video.

  The system control unit 110 is realized by a computer including, for example, a CPU (Central Processing Unit) and a memory, and controls each component included in the camera 100. The system control unit 110 includes a video file processing unit 111 and a file system management unit 112 as functional components.

  These functional components are realized by the CPU executing various programs. In addition, the implementation form of each memory included in the system control unit 110 is not limited to one implementation form. For example, the system control unit 110 may include a memory for each use, or may include only a single memory and use the memory area separately for each use.

  When video data is written as a file on the recording medium 131, the file system management unit 112 updates FAT management information for managing the file.

  When the input unit 140 receives an instruction to start shooting, the video file processing unit 111 causes the video input unit 143 and the video encoder 142 to execute the above-described processing, and outputs video data (DVCPRO-HD compressed data). . When the video file processing unit 111 receives an instruction to end shooting, the video input processing unit 143 and the video encoder 142 terminate the above-described processing and stop outputting video data. Here, the video file processing unit 111 writes the video data as a file to the recording medium 131 while updating the FAT management information in the file system management unit 112 from when the video data is output until it is stopped. Go.

  That is, when there is an instruction to start shooting, the video file processing unit 111 opens a file via the file system management unit 112 and records video data in the file. Then, when there is an instruction to end shooting, the video file processing unit 111 closes the file. By executing such a series of recording operations, video data is newly created on the recording medium 131 as a video file.

  In parallel with the writing of the video data to the recording medium 131, the video file processing unit 111 reads the video data from the recording medium 131 via the file system management unit 112 and transmits the video data to the transmission / reception unit. 150 transmits the data to the server 300 and causes the video decoder 144 to decode the video data. Furthermore, when the proxy video data is received by the transmission / reception unit 150, the video file processing unit 111 also causes the video decoder 144 to execute decoding of the proxy video data. As a result, the captured image and the proxy image are displayed on the display unit 141.

  FIG. 3 is a diagram illustrating an example of a captured video and a proxy video displayed on the display unit 141.

  For example, as shown in FIG. 3, the display unit 141 displays a captured video V1 indicated by the video data on the entire screen, and displays the proxy video V2 indicated by the proxy video data on the right corner of the screen (right corner as viewed in FIG. 3). And superimposed on a part of the photographed video V1.

  The user of the camera 100 views the captured video V1 and the proxy video V2 displayed on the display unit 141 as described above, and the video captured and recorded by the camera 100 and the video received by the server 300 Can be confirmed. Further, when the captured video V1 and the proxy video V2 displayed at the present time are different, the user delays the video data received by the server 300 from the video data recorded by the camera 100. I can grasp that. Further, the user confirms how much the reception of the video data is delayed from the time difference between the proxy video V2 currently displayed and the captured video V1 displayed at the past time corresponding to the proxy video V2. can do.

  FIG. 4 is a block diagram showing a configuration of server 300 in the present embodiment.

  The server 300 includes a system control unit 310, an I / O bus 320, a recording medium 330, a video encoder 341, a video decoder 340, and a transmission / reception unit 350.

  The recording medium 330 is a large-capacity, randomly accessible recording medium such as an HDD (Hard Disk Drive), and is formatted by a file system such as NTFS (New Technology File System).

  The I / O bus 320 enables data input and output between the above-described components included in the server 300.

  The video decoder 340 decodes the video data stored in the recording medium 330 and outputs the decoded video data to the video encoder 341.

  When the video encoder 341 obtains the decoded video data from the video decoder 340, the video encoder 341 reduces the resolution of the decoded video data and uses a compression format (for example, DVCPRO-HD (SMPTE) used in the camera 100. Encoding is performed in a compression format with a higher compression ratio than -370M)). For example, the video encoder 341 reduces the image size of the decoded video data to a QVGA (Quarter VGA) size, and then converts the reduced video data in the MPEG-4 (ISO / IEC 14496) format. By encoding, compressed data having an image size of QVGA is generated and output as the proxy video data.

  The transmission / reception unit 350 receives video data transmitted from the camera 100 via the wireless network 400 and outputs the video data to the system control unit 310.

  Further, the transmission / reception unit 350 packetizes the proxy video data stored in the recording medium 330 and sequentially transmits the proxy video data packets to the camera 100 via the wireless network 400. In addition, when an ACK packet is transmitted from the camera 100 to the transmitted proxy video data packet, the transmission / reception unit 350 receives the ACK packet. Then, the transmission / reception unit 350 notifies the system control unit 310 of the transmission timing of the proxy video data packet and the reception timing of the ACK packet corresponding to the packet.

  The system control unit 310 is realized by a computer including a CPU and a memory, for example, and controls each component included in the server 300. The system control unit 310 includes a video file processing unit 311, a file system management unit 312, a transmission / reception status confirmation unit 313, and a frame thinning unit 314 as functional components.

  Note that the implementation method of these functional components and the implementation form of each memory included in the system control unit 310 are the same as the implementation method or implementation form of the system control unit 110 of the camera 100.

  When video data or proxy video data is written as a file on the recording medium 330, the file system management unit 312 updates management information of the file system for managing those files.

  The video file processing unit 311 writes the proxy video data output from the video encoder 341 to the recording medium 330 as a file while causing the file system management unit 312 to update the management information of the file system. The video file processing unit 311 reads proxy video data from the recording medium 330 via the file system management unit 312 in parallel with the writing of the proxy video data described above, and transmits and receives the proxy video data. To the unit 350.

  Further, when the video file processing unit 311 acquires the video data from the transmission / reception unit 350, the video file processing unit 311 writes the video data as a file to the recording medium 330 while the file system management unit 312 updates the management information of the file system. The video file processing unit 311 reads the video data from the recording medium 330 via the file system management unit 312 in parallel with the video data writing described above, and sends the video data to the video decoder 340. Output.

  Based on the timing notified from the transmission / reception unit 350, the transmission / reception status confirmation unit 313 measures the time from when the proxy video data packet is transmitted until the ACK packet corresponding to the packet is received, A delay time of data transmission / reception via the wireless network 400 is detected.

  The frame decimation unit 314 determines whether or not the delay time detected as described above is equal to or greater than a threshold value. If it is determined that the delay time is equal to or greater than the threshold value, proxy video data generated by the video encoder 341 is determined. Thin out the picture (or frame). Specifically, the frame thinning unit 314 thins out the pictures included in the decoded video data acquired by the video encoder 341, and causes the video encoder 341 to reduce the resolution and encode the remaining pictures. . For example, the frame decimation unit 314 decimates every other picture included in the video data, leaves only a picture that is used for double-speed playback, and performs video encoder 341 to reduce the resolution of the remaining picture. To run. Hereinafter, the process of thinning pictures in this way is referred to as frame thinning.

  Accordingly, each picture included in the proxy video data is sequentially transmitted from the transmission / reception unit 350 to the camera 100. When the delay time is equal to or greater than a threshold value, for example, one picture is thinned out at a rate of two and transmitted. Is done. That is, the bit rate of proxy video data is reduced to ½.

  Further, the frame thinning unit 314 ends the above-described frame thinning when the delay time that is equal to or greater than the threshold value becomes less than the threshold value.

  Note that the frame thinning unit 314 may hold a predetermined threshold value as described above, or may adaptively change the threshold value according to the band of the wireless network 400 to be used.

  Here, the detection of the delay time by the transmission / reception status confirmation unit 313 of the server 300 will be described in detail.

  FIG. 5 is a sequence diagram showing packet transmission / reception between the camera 100 and the server 300.

  The camera 100 sequentially transmits video data packets a1, a2,..., An to the server 300.

  Next, when receiving the packets a1, a2,..., An, the server 300 generates proxy video data packets b1, b2,..., Bm corresponding to the packets a1, a2,. It transmits to the camera 100 sequentially.

  Here, when the server 300 sequentially transmits packets of proxy video data, the server 300 transmits the packet, receives the ACK packet transmitted from the camera 100 for the packet, and then transmits the next packet of proxy video data. Send.

  Specifically, the server 300 first transmits a packet b 1 of proxy video data to the camera 100. When the camera 100 correctly receives the packet b1, the camera 100 transmits an ACK packet B1 corresponding to the packet b1 to the server 300. Next, when the server 300 receives the ACK packet B1, the server 300 transmits the next packet b2 of the proxy video data to the camera 100.

  As described above, the server 300 confirms that the transmitted proxy video data packet is correctly received by the camera 100 by receiving the ACK packet, and transmits the proxy video data packets b1, b2,. Are sent sequentially.

  Here, when the proxy video data packet b1 is transmitted, the transmission / reception status confirmation unit 313 of the server 300 is the time from when the packet b1 is transmitted until the ACK packet B1 corresponding to the packet b1 is received. Is measured as a delay measurement time td1. Similarly, when the proxy video data packet b2 is transmitted, the transmission / reception status confirmation unit 313 delays the time from when the packet b2 is transmitted until the ACK packet B2 corresponding to the packet b2 is received. Measurement is performed as a measurement time td2.

  In this way, the transmission / reception status confirmation unit 313 measures the delay measurement times td1, tb2,..., Tdn each time a proxy video data packet is transmitted, and calculates the average value thereof, for example, thereby calculating the above-described values. Detect the delay time.

  After that, the camera 100 sequentially transmits video data packets a11, a12,..., A1n to the server 300, and the server 300 sequentially receives the transmitted packets a11, a12,.

  As described above, between the camera 100 and the server 300, transmission / reception of each packet of video data and proxy video data and an ACK packet is repeatedly performed until transmission of video data from the camera 100 is completed. The transmission / reception status confirmation unit 313 detects the above-described delay time while such transmission / reception is performed.

  Further, the video encoder 341 and the frame decimation unit 314 of the server 300 change the processing for the video data according to the delay time detected by the transmission / reception status confirmation unit 313 in this way, and generate proxy video data. .

  FIG. 6 is a diagram illustrating pictures included in video data and proxy video data.

  The video encoder 341 acquires pictures p1, p2,... Included in the decoded video data. The video encoder 341 generates proxy video data including the pictures pa1, pa2,... By encoding the pictures p1, p2,. The display times of the proxy video data pictures pa1, pa2,... Coincide with the display times of the video data pictures p1, p2,.

  Here, if the frame thinning unit 314 determines that the delay time detected as described above is equal to or greater than the threshold value, the frame thinning unit 314 performs frame thinning on the video data acquired by the video encoder 341. For example, the frame thinning unit 314 thins out the pictures p2, p4, p6,... At a rate of one out of two pictures p1, p2, p3, p4, p5, p6. Leave p3, p5 ....

  As a result, when the delay time is equal to or greater than the threshold value, the video encoder 341 performs frame thinning by reducing only the pictures p1, p3, p5. Proxy video data, that is, proxy video data including pictures pa1, pa3, pa5.

  Next, the delay of proxy video data received by the camera 100 and the reduction of the delay will be described with reference to FIGS.

  FIG. 7 is a diagram illustrating an example of video data and proxy video data transmitted and received by the camera 100 and the server 300 at a certain point in time.

  The camera 100 has already transmitted the section from the head position of the video data to the position 500 as shown in FIG.

  At this time, the server 300 receives a section from the head position of the video data to the position 501 as shown in FIG. That is, the section (delayed portion 510) from the position 501 of the video data to the position 500 described above is not received by the server 300 and is delayed.

  Further, the server 300 uses a part of the received video data, generates a part of the proxy video data corresponding to the part, and, as shown in FIG. To the position 502 has already been transmitted.

  At this time, the camera 100 receives a section from the leading position of the proxy video data to the position 503 as shown in FIG. That is, the section (delayed portion 511) from the position 503 of the proxy video data to the position 502 described above is not received by the camera 100 and is delayed. Further, as shown in FIG. 7B, the server 300 receives video data from the head position to the position 501, but as shown in FIG. Video data is received only from the head position to position 503. Therefore, the user of the camera 100 has not confirmed the reception of the video data even though the positions 503 to 501 of the video data are received by the server 300.

  Therefore, the transmission / reception status confirmation unit 313 of the server 300 detects the time until the camera 100 receives all the delay portions 511 of the proxy video data as the above-described delay time.

  The transmission / reception status confirmation unit 313 measures the time from when the proxy video data packet is transmitted until the ACK packet corresponding to the packet is received, and detects the delay time based on the measurement result. The delay time may be detected based on the position 503 and the position 502 of the proxy video data as described above.

  That is, when the proxy video data packet is transmitted from the server 300, the transmission / reception status confirmation unit 313 sets the proxy video data transmission position (position 502 in FIG. 7) in the server 300 to the end of the data corresponding to the packet. Move to. Thereafter, when receiving the ACK packet corresponding to the packet, the transmission / reception status confirmation unit 313 obtains the reception position (position 503 in FIG. 7) of the proxy video data in the camera 100. Then, the transmission / reception status confirmation unit 313 calculates a section from the proxy video data reception position (position 503 in FIG. 7) to the transmission position (position 502 in FIG. 7) as a delay portion 511, and the delay portion 511 is calculated as the camera 100. Is detected as the above-mentioned delay time.

  As a result of detecting the delay time as described above, if the frame thinning unit 314 determines that the delay time is equal to or greater than the threshold value, the frame thinning unit 314 shortens the delay time by frame thinning. In other words, the frame thinning unit 314 tries to shorten the section of video data (the section from the position 503 to the position 501) for which reception by the server 300 is not confirmed on the camera 100 side.

  FIG. 8 is a diagram illustrating an example of video data and proxy video data transmitted and received by the camera 100 and the server 300 at other points in time. Note that the state of the video data and proxy video data shown in FIG. 8 is such that frame thinning is started by the frame thinning unit 314 from the time when the video data and proxy video data are transmitted and received as shown in FIG. This is a state after a predetermined time has elapsed.

  Specifically, at this time, the camera 100 has already transmitted a section from the head position of the video data to the position 600 behind the above-described position 500 as shown in FIG. .

  At this time, as shown in FIG. 8B, the server 300 has received a section from the head position of the video data to a position 601 behind the position 501 described above. That is, the section (delayed portion 610) from the position 601 of the video data to the position 600 described above is not received by the server 300 and is delayed. Note that the transmission rate of the wireless network 400 at this time is restored as compared with the state shown in FIG. 7, and the delay portion 610 is shorter than the delay portion 510 shown in FIG.

  Further, the server 300 uses a part of the received video data, generates a part of the proxy video data corresponding to the part, and, as shown in FIG. The section from the position to the position 602 has already been transmitted.

  Here, in the section from the beginning position of proxy video data to position 502, server 300 converts all the pictures included in the section of video data corresponding to the section into pictures of proxy video data and sequentially transmits them. ing. Then, in the section from the position 502 to the position 602 of the proxy video data, the server 300 thins out some pictures from all the pictures included in the section of the video data corresponding to the section and proxy the remaining pictures. The video data is converted into a picture and sequentially transmitted.

  That is, the frame thinning unit 314 determines that the delay time is equal to or greater than the threshold when proxy video data is transmitted from the head position to the position 502, and starts frame thinning. As a result, when transmitting / receiving the proxy video data in the section from the position 502 to the position 602, the transmission / reception unit 350 of the server 300 transmits the proxy video data at a lower bit rate than the previous proxy video data. .

  Further, as described above, in the section from the position 502 to the position 602 of the proxy video data, some of the pictures included in the video data are thinned out, and the resolution and coding of the remaining pictures are reduced. Has been done. Therefore, in the video encoder 341 of the server 300, the processing load related to generation of proxy video data after the position 502 is reduced. As a result, in the video data received by the server 300 shown in FIG. 8B, a section where the corresponding proxy video data is not transmitted (section from the position 602 to the position 601) is shown in FIG. It is shorter than the section shown (section from position 502 to position 501).

  At this time, the camera 100 receives a section from the leading position of the proxy video data to the position 603 as shown in FIG. Here, the camera 100 receives a section from the leading position of the proxy video data to the position 502 at a high bit rate, and receives a section from the position 502 to the position 603 at a low bit rate. That is, the number of pictures of proxy video data received per unit time is smaller in the section from the position 502 to the position 603 of the proxy video data than in the previous section.

  As a result, the section (delay portion 611) from the position 603 to the position 602 of the proxy video data is shorter than the delay portion 511 shown in FIG. 7 due to the decrease in the bit rate. Note that the transmission rate of the wireless network 400 at this time is recovered as compared with the state shown in FIG. 7 as described above. Therefore, the delay portion 611 is shortened by being influenced by the recovery of the transmission rate.

  As described above, in the present embodiment, among the video data received by the server 300, a section where the corresponding proxy video data is not transmitted (section from the position 602 to the position 601) can be shortened. The delay part 611 of the proxy video data can be shortened. As a result, it is possible to shorten the section of video data (the section from the position 603 to the position 601) for which reception by the server 300 is not confirmed on the camera 100 side.

  Accordingly, the camera 100 can receive proxy video data quickly and can quickly reproduce a section in which the picture of the proxy video data is thinned out. It becomes possible to approach.

  FIG. 9 is a diagram for explaining an overall video transmission method according to the present embodiment.

  The camera 100 sequentially transmits video data packets to the server 300, and the server 300 sequentially receives the transmitted packets (step S100).

  The server 300 decodes the video data of the part included in the received packets, and lowers the resolution and encodes it, thereby generating a part of the proxy video data corresponding to the part (step S102).

  Then, the server 300 packetizes a part of the proxy video data and sequentially transmits the proxy video data packet to the camera 100. Each time the camera 100 correctly receives the packet, the server 300 transmits an ACK corresponding to the packet. The packet is transmitted to the server 300 (step S104).

  Here, the server 300 detects the delay time based on the transmission timing of the proxy video data packet and the reception timing of the ACK packet (step S106). If the server 300 determines that the detected delay time is equal to or greater than the threshold value, then the server 300 performs frame thinning when generating proxy video data.

  Next, when the camera 100 sequentially transmits subsequent packets of video data to the server 300 as in step S100, the server 300 sequentially receives the transmitted packets (step S108).

  As in step S102, the server 300 decodes the video data of the portion included in the received packet, generates a part of the proxy video data corresponding to the portion by reducing and encoding the video data. (Step S110). At this time, since the delay time detected in step S106 is equal to or greater than the threshold value as described above, the server 300 performs frame thinning when generating a part of the proxy video data.

  Then, the server 300 packetizes a part of the proxy video data and sequentially transmits the proxy video data packet to the camera 100. Each time the camera 100 correctly receives the packet, the server 300 transmits an ACK corresponding to the packet. The packet is transmitted to the server 300 (step S112). In this step S112, since frame thinning is performed in the previous step S110, the server 300 packetizes proxy video data for a number of pictures smaller than the number of pictures of the proxy video data transmitted in step S104. And send it. That is, the server 300 transmits a part of the proxy video data to the camera 100 at a low bit rate.

  Similarly to step S106, the server 300 detects the delay time based on the transmission timing of the proxy video data packet and the reception timing of the ACK packet (step S114). Then, similarly to the previous time, when the server 300 determines that the detected delay time is equal to or greater than the threshold value, the camera 100 and the server 300 repeatedly execute the processes from step S108 to S114.

  Thereafter, the server 300 detects the delay time (step S116), and if it is determined that the delay time is less than the threshold value, thereafter, when generating proxy video data, frame decimation is not performed.

  Therefore, when a continuous packet of video data is transmitted from the camera 100 (step S118), the camera 100 and the server 300 repeatedly execute the same processing as steps S102 to S106 (steps S120 to S124).

  FIG. 10 is a flowchart showing the operation of the server 300 in the present embodiment.

  First, the server 300 initializes the delay time to 0 (step S200). Then, the server 300 determines whether a video data packet has been received from the camera 100 (step S202). If the server 300 determines that it has not been received (N in step S202), it ends the process, and if it is determined that it has been received (Y in step S202), whether the delay time is equal to or greater than a threshold value. It is determined whether or not (step S206).

  If the server 300 determines that the delay time is less than the threshold (N in step S206), the server 300 generates a part of the proxy video data corresponding to the received video data packet without thinning out the frames ( Step S208). On the other hand, if the server 300 determines that the delay time is equal to or greater than the threshold (Y in step S206), the server 300 generates part of the proxy video data corresponding to the received video data packet while performing frame thinning. (Step S210).

  Then, the server 300 transmits the proxy video data packet generated in step S208 or step S210 to the camera 100 (step S212), and receives an ACK packet corresponding to the packet from the camera 100 (step S214). Thereafter, the server 300 determines whether or not the last packet of the partial proxy video data generated in step S208 or step S210 has been transmitted (step S216).

  Here, when determining that the last packet has not been transmitted (N in Step S216), the server 300 repeatedly executes the processes from Step S212. On the other hand, when determining that the last packet has been transmitted (Y in step S216), the server 300 detects a delay time based on the timings in steps S212 and S214 (step S218). When the delay time is detected in step S218, the server 300 repeatedly executes the processing from step S202 again.

  As described above, in this embodiment, since the video data transmitted by the camera 100 is sent back from the server 300 as proxy video data, the user of the camera 100 is surely received by the server 300 of the transmitted video data. Can be confirmed appropriately. That is, the user can appropriately check the progress of the video data received by the server 300 by looking at the proxy video displayed on the camera 100.

  Furthermore, since the delay time of the wireless network 400 is detected and the amount of proxy video data transmitted from the server 300 to the camera 100 is reduced as the delay time increases, the camera 100 can be used even if the delay time increases. It is possible to prevent the reception of the proxy video data from being delayed. In other words, the delay can be recovered by reducing the image quality of the proxy video.

  Therefore, the user of the camera 100 can appropriately and quickly confirm whether the video data (captured video) transmitted from the camera 100 is reliably received by the server 300.

  In the present embodiment, when the frame thinning unit 314 performs frame thinning, the picture is thinned out from the video data before the resolution reduction and encoding are performed. Compared with the case of thinning out, it is possible to reduce the processing load for resolution reduction and encoding. As a result, the server 300 can quickly transmit the proxy video data to the camera 100.

(Modification)
In the above embodiment, when the delay time becomes long, the proxy video data is generated and transmitted while the picture is thinned out. As a result, in the section where the picture is thinned out, the image quality of the proxy video indicated by the proxy video data is degraded. Therefore, there is a possibility that the user cannot sufficiently confirm that the video data has been received by the server 300. Therefore, in the present modification, a section in which proxy video is displayed with low image quality (hereinafter referred to as a low image quality section) is specified in proxy video data, and information indicating the section is recorded as a list file (section information). Thus, the user can determine whether the video data (video file) should be saved or deleted using the low image quality interval indicated by the list file, and the convenience of the user can be determined. Can be improved.

  Such a modification will be described in detail below.

  FIG. 11 is a block diagram illustrating a configuration of a camera according to this modification.

  A camera 700 according to this modification includes a system control unit 710 having a function different from that of the system control unit 110 instead of the system control unit 110 of the camera 100 of the above embodiment, and excludes the system control unit 110 of the camera 100. It has the same components as the other components. Of the constituent elements included in the camera 700, the same constituent elements as those included in the camera 100 will not be described in detail.

  The system control unit 710 includes a video file processing unit 111 and a file system management unit 112 included in the system control unit 110 of the above embodiment, and also includes a list file processing unit 713.

  The list file processing unit 713 monitors the bit rate of proxy video data received by the transmission / reception unit 150. Then, the list file processing unit 713 specifies a section where the bit rate is lower than a predetermined rate threshold in the proxy video data as the above-described low image quality section, and generates a list file indicating the low image quality section To the recording medium 131.

  FIG. 12 is a diagram illustrating an example of the bit rate of proxy video data and a list file.

  As shown in FIG. 12A, the list file processing unit 713 monitors the bit rate of the proxy video data while the proxy video data is received by the transmission / reception unit 150. The list file processing unit 713 then includes a decrease time when the bit rate changes from the rate threshold value to less than the rate threshold value, and a rise time when the bit rate changes from less than the rate threshold value to the rate threshold value. Is identified. Specifically, the list file processing unit 713 specifies times t1, t3, and t5 as the fall times, and specifies times t2, t4, and t6 as the rise times.

  Note that the rate threshold is, for example, a bit that is ½ of the standard bit rate when the bit rate when the proxy video data generated without frame decimation is received is the standard bit rate. Rate.

  Further, as shown in FIG. 12B, the list file processing unit 713 generates a list file Lf that indicates a section from the fall time specified as described above to the rise time as a low image quality section. For example, the list file processing unit 713 treats the specified decrease time and the immediately subsequent increase time as an IN point and an OUT point, respectively, and sequentially records a set of the IN point and the OUT point in the list file Lf. Go.

  Specifically, the list file processing unit 713 executes the first group (IN point t1, OUT point t2), the second group (IN point t3, OUT point t4), and the third group (IN point t5, OUT point). The points t6) are recorded in the list file Lf in order.

  In the list file Lf generated in this way, the section from the IN point to the OUT point indicated by each group displays the section of the proxy video data generated by frame thinning, that is, the proxy video is displayed with low image quality. Indicates a low-quality section. Furthermore, the IN point and the OUT point correspond to the pictures at the start point and the end point of the low-quality section of the proxy video or the section of the captured video (video data) corresponding to the low-quality section.

  Further, when the list file processing unit 713 assigns a file name to the generated list file Lf, the part other than the extension included in the file name is made the same as the part other than the extension included in the file name of the video file. Keep it. The video file is a video file corresponding to the proxy video data used to generate the list file Lf. For example, the file name of the transmitted video file is 001a. If it is mxf, the list file processing unit 713 changes the file name of the list file Lf to 001a. It is set as txt. By doing so, the list file Lf can be associated with the video file, and the list file processing unit 713 can grasp which video file indicates the set of the IN point and OUT point of the list file Lf.

  Further, when the reconfirmation instruction from the user is received in the input unit 140, the list file processing unit 713 sequentially reads out the set of IN point and OUT point from the list file Lf recorded on the recording medium 131, and sets each set. Is displayed on the display unit 141.

  FIG. 13 is a diagram illustrating an example of icons displayed on the display unit 141.

  When the list file processing unit 713 reads out, for example, the list file Lf shown in FIG. 12B from the recording medium 131, the icon I1, which indicates the first, second, and third pairs arranged in the list file Lf, respectively. I2 and I3 are displayed on the display unit 141.

  Here, when the user selects one of the icons by operating the input unit 140, the list file processing unit 713 reads the section of the video data specified by the set indicated by the selected icon from the recording medium 131, The video data for the section is decoded by the video decoder 144 and the captured video is displayed on the display unit 141.

  As described above, in this modification, the user can check the low-quality section in which the video data has been received by the server 300 using the captured video of the recorded video data. Therefore, the user can easily decide whether to delete the file, such as leaving a video file in which the server 300 is surely received or uneasy, or deleting the video file because there is no important scene although it is unknown whether it was received reliably. Can be done.

  As described above, the video transmission method according to the present invention has been described using the above-described embodiment and its modifications, but the present invention is not limited to these.

  For example, in the present embodiment and its modifications, data is transmitted using a wireless network. However, even in a wired network such as ADSL (Asymmetric Digital Subscriber Line) or FTTx (Fiber To The x) It may be a combination of a network and a wired network.

  In the present embodiment and its modification, the video data packet and the ACK packet are transmitted on the same wireless network, but the ACK packet may be transmitted on another network having a small network delay. For example, an ACK packet may be transmitted using a CDMA (Code Division Multiple Access) wireless network employed in mobile phones.

  Further, in the present embodiment and its modification, it has been described that the reception position of data is grasped by using an ACK packet. However, the present invention is not limited to the ACK packet, and for example, the reception position information of the data itself may be transmitted. Good.

  In the frame thinning of the proxy video data of the confirmation video, it has been described that the bit rate is halved. However, the bit rate may be changed stepwise depending on the value of the network delay (delay time).

  Also, in frame thinning, pictures are not thinned out every other picture (frame), but in a format that performs interframe compression such as MPEG, P pictures and B pictures may be thinned out leaving only I pictures. Good. Further, the frame thinning processing unit may be a 1 GOP (Group Of Pictures) unit or a plurality of GOP units.

  Further, in this embodiment and its modification, the compression format of the video data recorded by the camera is the DVCPRO-HD format and the compression format of the proxy video data is the MPEG-4 format. However, the compression format is limited to the compression format. It is not a thing.

  Further, in the present embodiment and its modification, the proxy video data is transmitted to the camera that transmits the original video data, but may be a device other than the camera or a plurality of devices. For example, proxy video data may be transmitted to a mobile editing machine having a wireless network function at the same shooting site, and the transmission progress of the video data may be confirmed by the mobile editing machine.

  In this embodiment and its modification, the camera that recorded the video data transmitted the video. However, the recording medium on which the video data was recorded was transferred to another device, and the video data was transmitted from the device. Then, proxy video may be received.

  Further, in the present embodiment and its modification, the proxy video that is the confirmation video is displayed in the lower right small area as shown in FIG. 3, but the area is moved by the user's instruction or the proxy video is displayed. Display and non-display may be switched, and display in the small area may be switched to display on a large screen for photographing.

  In the present embodiment and the modification, the video data is recorded as a file on a removable recording medium. However, the recording medium may be an HD or a flash memory built in the camera, and an interface such as a USB (Universal Serial Bus). Or an HD connected to the camera.

  In the above modification, the IN point and the OUT point are set according to whether or not the bit rate is less than ½ of the standard bit rate. May be set freely.

  In the above modification, the list file is described as being recorded on the recording medium. However, the list file may be recorded in a nonvolatile memory such as a flash memory built in the camera.

  In the above modification, the list file is associated with the video file by making the part other than the file name extension the same, but the list file name is arbitrary and the video file is included in the list file. The file name may be recorded for association. In that case, there is no need to create a list file for each video file, only one list file is created, and for each video file, the file name of the video file and the IN point corresponding to the video file are created. And a set of OUT points may be recorded.

  In this modification, the set of IN points and OUT points on the list file is displayed as icons, but the contents of the list file may be simply displayed. Further, a thumbnail of a picture (frame) of video data corresponding to the IN point may be displayed as an icon, and further, a thumbnail of a picture (frame) of video data reproduced for a few seconds from the IN point may be displayed. Good.

  In this modification, the camera 700 reads and reproduces the video data section corresponding to the low image quality section from the recording medium 131, but the video data section may be retransmitted to the server 300. Thereby, the user of the camera 700 can reliably transmit the video data to the server 300.

  The video transmission method according to the present invention is useful for the purpose of reliably transmitting video data recorded as a file over a network. For example, the video transmission method is applied to a system having a camera having a communication function and a server communicating with the camera. In particular, it is suitable for the case where video data shot to a broadcasting station is reliably transmitted and the next shooting is started immediately, such as shooting performed with a video camera for broadcasting or business use.

It is a figure which shows the structure of the video transmission system in embodiment of this invention. It is a block diagram which shows the structure of the camera in embodiment of this invention. It is a figure which shows an example of the picked-up image and proxy image | video displayed on the display part in embodiment of this invention. It is a block diagram which shows the structure of the server in embodiment of this invention. It is a sequence diagram which shows transmission / reception of the packet between the camera of embodiment of this invention, and a server. It is a figure which shows the picture contained in the video data and proxy video data of embodiment of this invention. It is a figure which shows an example of the video data and proxy video data which were transmitted / received by the camera and the server at a certain time of embodiment of this invention. It is a figure which shows an example of the video data and proxy video data which were transmitted / received by the camera and server at the other time of embodiment of this invention. It is a figure for demonstrating the whole video transmission method in embodiment of this invention. It is a flowchart which shows operation | movement of the server in embodiment of this invention. It is a block diagram which shows the structure of the camera which concerns on the modification of embodiment of this invention. It is a figure which shows an example of the bit rate and list file of proxy video data which concern on the modification of embodiment of this invention. It is a figure which shows an example of the icon displayed on the display part which concerns on the modification of embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Camera 110 System control part 111 Video file processing part 112 File system management part 120 I / O bus 130 Recording media I / F part 131 Recording medium 140 Input part 141 Display part 142 Video encoder 143 Video input part 144 Video decoder 150 Transmission / reception part 300 server 310 system control unit 311 video file processing unit 312 file system management unit 313 transmission / reception status confirmation unit 314 frame thinning unit 320 I / O bus 330 recording medium 340 video decoder 341 video encoder 350 transmission / reception unit 700 camera 710 system control unit 713 List file processing section

Claims (16)

A video transmission method for transmitting and receiving data related to a video between first and second terminal devices,
A main video transmission step in which the first terminal device transmits main video data indicating a main video to the second terminal device;
A main video receiving step in which the second terminal device receives the main video data from the first terminal device;
A sub-picture generating step in which the second terminal device generates sub-picture data indicating a sub-picture corresponding to the main picture indicated by the received main video data;
A delay time detecting step in which the second terminal device detects a delay time of data transmission / reception with the first terminal device;
An adjustment step of adjusting the data amount so that the amount of sub-picture data generated in the sub-picture generation step decreases as the detected delay time increases.
A sub-video transmission step in which the second terminal device transmits the sub-video data to the first terminal device;
A sub-picture receiving step in which the first terminal apparatus receives the sub-picture data from the second terminal apparatus. In the sub-picture generation step,
The video transmission method according to claim 1, wherein the second terminal device generates the sub-video data by reducing a resolution of a main video indicated by the main video data. In the adjustment step,
The second terminal apparatus decreases a bit rate of the sub-picture data transmitted in the sub-picture transmission step by reducing a data amount of the sub-picture data as the detected delay time is longer. The video transmission method according to 1. The video transmission method further includes:
A delay determination step of determining whether or not the detected delay time is equal to or greater than a predetermined threshold value by the second terminal device;
In the adjustment step,
The second terminal device is:
When it is determined in the delay determining step that the delay time is equal to or greater than the threshold value, the data amount of the sub-video data is determined based on the amount of sub-video data when the delay time is less than the threshold value. The video transmission method according to claim 1, wherein the video transmission method is less than the data amount. In the delay time detecting step,
The second terminal device is:
Transmitting first data to the first terminal device;
Receiving from the first terminal device second data for notifying that the data has been received by the first terminal device;
The video transmission method according to claim 1, wherein the delay time is detected by measuring a time from when the first data is transmitted to when the second data is received. In the delay time detecting step,
The second terminal device is:
The video transmission method according to claim 5, wherein the sub-video data generated in the sub-video generation step is transmitted to the first terminal device as the first data. In the adjustment step,
The second terminal device is:
The video transmission method according to claim 1, wherein a data amount of the sub-video data is reduced by thinning out any picture from a plurality of pictures constituting the received main video data. The video transmission method further includes:
A section specifying step in which the first terminal device specifies a section on the time axis of the sub-video data as a low-quality section in which the image quality of the sub-picture indicated by the received sub-picture data is less than a threshold value. When,
The video transmission method according to claim 1, further comprising: a recording step in which the first terminal device records section information indicating the identified low image quality section on a recording medium. In the section specifying step,
The first terminal device is:
9. The video according to claim 8, wherein a bit rate of the sub video data is detected while the sub video data is received, and a section in which the bit rate is less than a rate threshold is specified as the low image quality section. Transmission method. The video transmission method further includes:
A section reading step in which the first terminal device reads the section information from the recording medium;
The video transmission method according to claim 8, further comprising: a section display step in which the first terminal device displays a main video of the section of the main video data corresponding to the low image quality section indicated by the read section information. The video transmission method further includes:
The video transmission method according to claim 1, further comprising a main video generation step of generating the main video data by photographing the first terminal device. The video transmission method further includes:
The video transmission method according to claim 11, further comprising: a video display step in which the first terminal device displays a main video of the main video data generated by shooting and a sub-video of the received sub-video data. A video transmission method in which a terminal device transmits / receives data related to video to / from another terminal device,
A main video receiving step of receiving main video data indicating the main video from the other terminal device;
A sub video generation step of generating sub video data indicating a sub video corresponding to the main video indicated by the received main video data;
A delay time detecting step of detecting a delay time of data transmission / reception with the other terminal device;
An adjustment step of adjusting the data amount so that the longer the detected delay time, the smaller the data amount of the sub-video data generated in the sub-video generation step;
And a sub-video transmission step of transmitting the sub-video data to the other terminal device. A terminal device that transmits and receives video-related data to and from other terminal devices,
A main video receiver that receives main video data indicating the main video from the other terminal device;
A sub-video generating unit that generates sub-video data indicating a sub-video corresponding to the main video indicated by the received main video data;
A delay time detecting unit for detecting a delay time of data transmission / reception with the other terminal device;
An adjustment unit that adjusts the amount of data so that the amount of sub-picture data generated by the sub-picture generation unit decreases as the detected delay time increases;
A terminal device comprising: a sub-video transmission unit that transmits the sub-video data to the other terminal device. A program for a terminal device to transmit and receive data related to video with other terminal devices,
A main video receiving step of receiving main video data indicating the main video from the other terminal device;
A sub video generation step of generating sub video data indicating a sub video corresponding to the main video indicated by the received main video data;
A delay time detecting step of detecting a delay time of data transmission / reception with the other terminal device;
An adjustment step of adjusting the data amount so that the longer the detected delay time, the smaller the data amount of the sub-video data generated in the sub-video generation step;
A program for causing a computer of the terminal device to execute a sub-video transmission step of transmitting the sub-video data to the other terminal device. An integrated circuit that transmits and receives video data to and from a terminal device,
A main video receiving unit that receives main video data indicating the main video from the terminal device;
A sub-video generating unit that generates sub-video data indicating a sub-video corresponding to the main video indicated by the received main video data;
A delay time detection unit for detecting a delay time of data transmission / reception with the terminal device;
An adjustment unit that adjusts the amount of data so that the amount of sub-picture data generated by the sub-picture generation unit decreases as the detected delay time increases;
An integrated circuit comprising: a sub-video transmission unit that transmits the sub-video data to the terminal device.

Images