WO2019207695A1 - Transfer device, transfer system, transfer method, and program - Google Patents

Abstract

According to the present invention, when data received from a transmission device is to be transferred to a reception device, if the received data does not meet the requirements of the reception device, data meeting these requirements is instead provided to the reception device. A reception unit (304) of a transfer device (130) receives time-series data from a transmission device. A determination unit (303) determines data transmission conditions by exchanging information with a reception device. A processing unit (306) processes the data in accordance with the determined transmission conditions. A transmission unit (307) transmits the processed data to the reception device if real-time data transfer is performed. If real-time data transfer is not performed, then a storage unit stores the data as time-series data.

Description

Transfer device, transfer system, transfer method, and program

The present invention relates to a technique for transferring data.

A technology for transferring data is known. For example, Patent Document 1 describes a technique for transmitting and recording a stream received from a network to an external device in real time.

JP 2006-246245 A

When transferring data received from a transmitting apparatus to the receiving apparatus, if the received data is transmitted to the receiving apparatus as it is, the data may not be processed when the receiving apparatus does not support the data format. In addition, when the amount of data received from the transmission device is larger than the amount of data desired by the reception device, if the received data is transmitted as it is to the reception device, the amount of communication with the reception device increases unnecessarily, and the data Will be delayed.
An object of the present invention is to provide data to a receiving apparatus in response to the request even when the data received from the transmitting apparatus is transferred to the receiving apparatus, even if the received data does not meet the request of the receiving apparatus. .

The present invention includes a receiving unit that receives data in time series from a transmitting device, a determining unit that determines a transmission condition of the data by exchanging information with the receiving device, and the transmission determined by the determining unit. A processing unit that processes the data according to conditions, a transmission unit that transmits the data processed by the processing unit to the receiving device when the data is transferred in real time, and the data in the real time In the case where the data is not transferred, a transfer device including a storage unit that stores the data along the time series is provided.

In the case where the data is transferred in real time, the storage unit may store a copy of the data after the data is transmitted to the receiving device.

In the transfer device, real-time information indicating whether or not to transfer the data in real time is added to the head of the data, and when the head of the data is received by the receiving unit, based on the real-time information A determination unit that determines whether to transfer the data in real time, and the processing unit receives the data when the determination unit determines to transfer the data in real time. Before completion, start processing the received data, sequentially process the received data, and when the transmission unit determines to transfer the data in real time by the determination unit, Before the reception of the data is completed, transmission of the processed data may be started and the processed data may be transmitted sequentially.

The storage unit stores the real-time information and the identifier of the data in association with each other, the identifier is added to the head of the data instead of the real-time information, and the determination unit Whether or not to transfer the data in real time may be determined based on the real-time information stored in the storage unit in association with the identifier added to the head.

The data is moving image data indicating a moving image, and the receiving device includes a display device that displays the moving image indicated by the moving image data, and the information includes a resolution or a frame rate corresponding to the display device. The processing unit may change the resolution or frame rate of the moving image data according to the resolution or the frame rate.

The receiving device includes a display device that displays the data on a designated time scale, the information includes the time scale, and the processing unit may thin the data according to the time scale. Good.

The information may include a status of communication with the receiving device, and the processing unit may thin out the data according to the status of the communication.

Last information indicating the end is added to the end of the data transmitted from the transmission device, and when the last information is received, the transfer device determines that reception of the data is completed. The transmission unit may further transmit the data stored in the storage unit to a predetermined processing unit when the determination unit determines that the reception of the data is completed.

The data transmitted by the transmission device includes a plurality of data blocks each including at least one of a plurality of detection amounts detected by a sensor, and each of the plurality of data blocks includes a data block. Number information indicating the number of the at least one detection amount included is added, and the transfer device uses the number information of the at least one detection amount included in the data block and the number information added to the data block. A determination unit that determines whether or not the data block is missing according to a difference from the number shown; and when the determination unit determines that the data block is missing, the transmission A retransmission request unit requesting retransmission of the lost data block to the apparatus; and retransmitting from the transmission apparatus in response to a request from the retransmission request unit. Using said data blocks may further include a completion unit which complements the missing data blocks.

In addition, the present invention includes a transfer device, a transmission device, and a reception device, and the transfer device exchanges information with the reception device and a first reception unit that receives data in time series from the transmission device. And a processing unit that processes the data according to the transmission condition determined by the determination unit, and the processing unit when transferring the data in real time. A first transmission unit that transmits the data processed by the unit to the reception device, and a storage unit that stores the data along the time series when the data is not transferred in real time, The transmission device includes an acquisition unit that acquires the data, and a second transmission unit that transmits the data acquired by the acquisition unit to the transfer device, and the reception device receives the information A third transmission unit that transmits to the transfer device, a second reception unit that receives the data transmitted from the transfer device, and a processing unit that performs processing using the data received by the second reception unit; A transfer system is provided.

The data includes moving image data indicating a moving image taken by the imaging device and measurement data output from the measuring device, and the processing unit sets the moving image data and the measured data along the same time axis. May be output.

Furthermore, the present invention includes a step of receiving data in time series from a transmission device, a step of determining a transmission condition of the data by exchanging information with the reception device, and a response to the determined transmission condition. A step of processing the data; a step of transmitting the data in real time; a step of transmitting the processed data to the receiving device; and a case of not transferring the data in real time. And a step of storing along the time series.

Furthermore, the present invention includes a step of receiving data in time series from a transmission device to a computer, a step of determining a transmission condition of the data by exchanging information with the reception device, and the determined transmission condition And processing the data, and transferring the data in real time, transmitting the processed data to the receiving device, and not transferring the data in real time, A program for executing the step of storing the data along the time series is provided.

According to the present invention, when data received from a transmission device is transferred to the reception device, even if the received data does not meet the request of the reception device, the data corresponding to the request can be provided to the reception device. .

1 is a diagram illustrating an example of a transfer system 100 according to an embodiment. It is a figure which shows an example of the hardware constitutions of the terminal device. It is a figure which shows an example of a function structure of the terminal device. 6 is a diagram illustrating an example of a format of a creation request message 220. FIG. 6 is a diagram illustrating an example of a format of a creation request message 230. FIG. It is a figure which shows an example of the format of a section. 3 is a diagram illustrating an example of a hardware configuration of a transfer device 130. FIG. 3 is a diagram illustrating an example of a functional configuration of a transfer device 130. FIG. It is a figure which shows an example of the format of the response message. 6 is a diagram illustrating an example of a stream table 320. FIG. It is a figure which shows an example of the database. It is a sequence chart which shows an example of a transmission process. It is a sequence chart which shows an example of a transfer process. It is a sequence chart which shows an example of a resending process. It is a figure which shows the example of an output of moving image data and measurement data.

1. Configuration FIG. 1 is a diagram illustrating an example of a transfer system 100 according to an embodiment. The transfer system 100 is a system that transfers data received from the terminal device 120 to another terminal device 120. The transfer system 100 includes an imaging device 110, a plurality of terminal devices 120, and a transfer device 130. In the example illustrated in FIG. 1, the plurality of terminal devices 120 include a terminal device 120A and a terminal device 120B. Note that the numbers of the imaging device 110, the terminal device 120, and the transfer device 130 illustrated in FIG. 1 are examples, and are not limited thereto. The imaging device 110 is connected to the terminal device 120A via the communication line 150. The terminal devices 120A and 120B are connected to the transfer device 130 via the communication line 160. The communication line 160 may include a wireless network such as a mobile line and the Internet, for example. For example, 3G, LTE (Long Term Evolution), Wi-Fi (registered trademark), WiMAX (registered trademark), or Bluetooth (registered trademark) may be used as a communication standard of the wireless network.

The imaging device 110 is a device that captures an image. The imaging device 110 is, for example, a digital video camera, and captures a moving image by converting an image onto an imaging element using an optical system, and converts the moving image data to output. This moving image data is composed of a plurality of frames arranged in time series. In addition, the imaging device 110 may include a microphone. In this case, the moving image data may include audio data indicating audio.

FIG. 2 is a diagram illustrating an example of a hardware configuration of the terminal device 120. The terminal device 120 is also called an edge, and is a device that transmits and receives data. The terminal device 120 is a computer that includes a processor 121, a memory 122, a storage 123, a communication device 124, and a display device 125. These devices are connected via a bus 126.

The processor 121 executes various processes by reading the program into the memory 122 and executing it. For example, the processor 121 may be configured by a CPU (Central Processing Unit). The memory 122 stores a program executed by the processor 121. For example, the memory 122 may be configured by a ROM (Read Only Memory) or a RAM (Random Access Memory). The storage 123 stores various data and programs. For example, the storage 123 may be configured by a hard disk drive or a flash memory. The communication device 124 is a communication interface connected to the communication lines 150 and 160. The communication device 124 transmits and receives data via the communication line 150 or 160. The display device 125 displays various information. For example, the display device 125 may be configured by a liquid crystal display.

FIG. 3 is a diagram illustrating an example of a functional configuration of the terminal device 120. In this example, the terminal device 120 has both a function as a transmission device that transmits data to the transfer device 130 and a function as a reception device that receives data from the transfer device 130. The terminal device 120 includes a connection control unit 201, a message transmission unit 202 (an example of a third transmission unit), a message reception unit 203, an acquisition unit 204, a generation unit 205, an addition unit 206, and a data transmission unit 207. (An example of a second transmission unit), a data retransmission unit 208, a data reception unit 209 (an example of a second reception unit), and a processing unit 210. These functions are realized when the processor 121 performs an operation or controls communication by the communication device 124 in cooperation with the program stored in the memory 122 and the processor 121 that executes the program.

The connection control unit 201 controls the connection between the terminal device 120 and the transfer device 130. This connection refers to a virtual dedicated communication path established in connection-type communication. For example, when establishing a connection, a message requesting establishment of a connection is transmitted from the connection control unit 201 to the transfer apparatus 130. When a response message is transmitted from the transfer device 130 to the terminal device 120 in response to this message, a connection is established between the terminal device 120 and the transfer device 130. When disconnecting the connection, a message requesting disconnection of the connection is transmitted from the connection control unit 201 to the transfer apparatus 130. When a response message is transmitted from the transfer device 130 to the terminal device 120 in response to this message, the connection is disconnected between the terminal device 120 and the transfer device 130.

The message transmission unit 202 transmits various messages used for negotiation performed between the terminal device 120 and the transfer device 130. For example, various messages include an information transmission message and a creation request message 220 or 230. The information transmission message includes information for determining data transmission conditions. This information may include performance information indicating the resource performance of the terminal device 120, for example. This resource may be hardware or software. For example, when data is displayed by a browser, the software may include a browser. Further, this information may be the transmission condition itself. The creation request message 220 or 230 includes information for determining data processing conditions.

FIG. 4 is a diagram illustrating an example of a format of the creation request message 220 transmitted from the terminal device 120 on the transmission side. In this example, the creation request message 220 includes a request ID, the number of streams, a stream ID, a flag, a measurement ID, a transmission source ID, and a transfer destination ID.

The request ID is information that uniquely identifies the creation request message 220. The number of streams indicates the number of stream attribute information included in the creation request message 220. This attribute information is composed of a set of a stream ID, a flag, a measurement ID, a transmission source ID, and a transfer destination ID. This attribute information generally has a larger data amount than the stream ID. The creation request message 220 includes stream attribute information as many as indicated by the number of streams. That is, the creation request message 220 may include attribute information of a plurality of streams.

Stream ID is information that uniquely identifies a stream. The stream ID is associated with the connection being connected. That is, the stream ID may be used as an identifier for uniquely identifying the stream in the connection being connected. The stream ID may be determined by the terminal device 120 or the transfer device 130.

The flag includes a retransmission flag and a storage flag. The retransmission flag is information indicating whether data has been retransmitted. When the retransmission flag is “0”, it indicates that the data is not retransmitted, that is, transmitted first. If the data is not retransmitted, the data is transferred in real time. On the other hand, when the retransmission flag is “1”, this indicates that the data has been retransmitted. If the data is retransmitted, it is not transferred in real time. As described above, the retransmission flag may be used as real-time information indicating whether to transmit data in real time. The save flag is information indicating whether to save data. When the save flag is “0”, it indicates that data is not saved. On the other hand, when the save flag is “1”, it indicates that the data is to be saved.

Measurement ID is information that uniquely identifies the measurement that generated the data. For example, when the data is moving image data output from the imaging device 110, one unit of measurement is from when the imaging device 110 starts shooting a moving image to when it ends. Therefore, the same measurement ID is assigned to the moving image data generated during this time. The transmission source ID is information that uniquely identifies the terminal device 120 that is the transmission source of the stream. The transfer destination ID is information that uniquely identifies the terminal device 120 that is a data transfer destination. When the transfer destination ID is “−”, it indicates that data is not transferred from the transfer device 130.

FIG. 5 is a diagram showing an example of the format of the creation request message 230 transmitted from the terminal device 120 on the receiving side. In this example, the creation request message 230 includes a request ID, the number of streams, a stream ID, and a transmission source ID, similar to the creation request message 220 described above. The creation request message 230 includes a transfer source ID. The transfer source ID is information for identifying the terminal device 120 that is the transfer source of the stream. The creation request message 230 may not include the flag, measurement ID, and transfer destination ID included in the creation request message 220.

The acquisition unit 204 acquires data from an external device. This data may be moving image data output from the imaging device 110, for example. The generation unit 205 generates a stream based on the data acquired by the acquisition unit 204. This stream refers to a data string handled by streaming. Streaming refers to a system in which output is started before input of the entire data is completed, and the input data is sequentially output while the data is being input. For example, a stream is composed of a plurality of sections. This section may be used as a data block constituting data.

FIG. 6 is a diagram showing an example of a section format. In this example, the section includes SOS (Start of Section), a plurality of units, and EOS (End of Section). The SOS is included at the beginning of the section. EOS is included at the end of the section. SOS is information indicating the start of a section. Each of the plurality of units includes one frame included in the moving image data. A sensor ID and a time stamp are added to each frame. This sensor ID is information for uniquely identifying the sensor from which data is acquired. The time stamp is information indicating the time when the data is acquired. This time may be, for example, an elapsed time from the time when the measurement is started to the time when the data is acquired, that is, a relative time based on the time when the measurement is started. EOS is information indicating the end of a section. The EOS includes a unit count and a serial number. The unit count is number information indicating the number of units included in the section. The serial number is information for uniquely identifying sections related to the same measurement unit. In the following description, SOS, unit, and EOS are also collectively referred to as “element”.

The adding unit 206 adds a stream ID for uniquely identifying the stream generated by the generating unit 205 to the SOS, the plurality of units, and the EOS included in each section of the stream.

The data transmission unit 207 transmits the stream to which the stream ID is added to the transfer device 130. This stream is transmitted by streaming. Transmission of this stream may be performed in section units or in element units.

If the section transmitted to the transfer device 130 is missing, the data retransmission unit 208 retransmits the section to the transfer device 130 in response to a request from the transfer device 130.

The data receiving unit 209 receives a stream from the transfer device 130. The processing unit 210 performs processing using data included in the stream received by the data receiving unit 209.

FIG. 7 is a diagram illustrating an example of a hardware configuration of the transfer device 130. The transfer device 130 is also called a hub, and is a device that relays data exchange between the plurality of terminal devices 120. The transfer device 130 performs non-real time processing and real time processing. The non-real time processing is processing for accumulating data received from the terminal device 120, for example. Thereby, the data received from the terminal device 120 are managed collectively. This data is used, for example, for various types of analysis on the status of the sensing target. Therefore, data accuracy is required in non-real time processing. On the other hand, the real-time processing is processing for transmitting data received from the terminal device 120 to another terminal device 120 in real time, for example. Note that this real time does not have to be completely simultaneous, and there may be a slight delay. This data is used, for example, for the user to grasp the current status of the sensing target. Therefore, immediacy is required in real-time processing.

The transfer device 130 is a computer that includes a processor 131, a memory 132, a storage 133 (an example of a storage unit), and a communication device 134. These devices are connected via a bus 135. Since the processor 131, the memory 132, the storage 133, the communication device 134, and the bus 135 are the same as the processor 121, the memory 122, the storage 123, the communication device 124, and the bus 126 described above, description thereof is omitted.

FIG. 8 is a diagram illustrating an example of a functional configuration of the transfer device 130. The transfer device 130 includes a message reception unit 301, a message transmission unit 302, a determination unit 303, a data reception unit 304 (an example of a reception unit or a first reception unit), a determination unit 305, a processing unit 306, data A transmission unit 307 (an example of a transmission unit or a first transmission unit), a retransmission request unit 308, and a complement unit 309 are provided. These functions are realized when the processor 131 performs an operation or controls communication by the communication device 134 in cooperation with the program stored in the memory 132 and the processor 131 that executes the program.

The message receiving unit 301 receives various messages from the terminal device 120. For example, the various messages include the information transmission message and the creation request message 220 described above. The message transmission unit 302 transmits various messages to the terminal device 120. For example, various messages include a response message 310 to the creation request message 220 received from the terminal device 120.

FIG. 9 is a diagram illustrating an example of the format of the response message 310. In this example, the response message 310 includes a request ID included in the creation request message 220 or 230 and result information. The result information is information indicating whether or not the creation of the stream attribute information is successful. For example, when the result information is “0”, it indicates that the creation of the attribute information of the stream is successful. On the other hand, when the result information is “1”, it indicates that the creation of the stream attribute information has failed. For example, when a necessary item is not included in the creation request message 220 or 230, or when an item included in the creation request message 220 or 230 does not have a valid value, creation of stream attribute information is performed. May fail. When a plurality of stream IDs are included in the creation request message 220 or 230, result information may be included for each stream ID.

The determination unit 303 determines the data transmission condition by exchanging information with the terminal device 120. For example, the transmission condition includes a condition regarding the data format. For example, when the data is moving image data output from the imaging device 110, the data format may include at least one of the compression method, resolution, and frame rate of the moving image data. The attribute information included in the creation request message 220 and the transmission conditions determined by the determination unit 303 are managed by the stream table 320, for example.

FIG. 10 is a diagram illustrating an example of the stream table 320. The stream table 320 is stored in the storage 133, for example. In this example, the stream table 320 is determined by the determination unit 303 with the stream ID, retransmission flag, storage flag, measurement ID, transmission source ID, transfer destination ID, or transfer source ID included in the creation request message 220 or 230. The transmission conditions are stored in association with each other.

The data receiving unit 304 receives a stream from the terminal device 120 on the transmission side. The determination unit 305 determines whether or not to transfer the stream received by the data reception unit 304 in real time. The processing unit 306 processes the data included in the stream according to the transmission conditions stored in the stream table 320 when the determination unit 305 determines to transfer in real time. This processing means, for example, changing the data format, although the data contents are not changed. For example, when the data format is included in the transmission condition, the data format may be changed according to the format included in the transmission condition. The data transmission unit 207 transmits a stream including the processed data to the terminal device 120 on the reception side. This stream is transmitted by streaming.

If the determination unit 305 determines not to transfer in real time, the data included in the stream is stored in the storage 133 in time series. If the determination unit 305 determines to transfer in real time, the data transmission unit 207 transmits a stream, and then a copy of the data included in the stream is stored in the storage 133 in time series. This data is managed by the database 330, for example.

FIG. 11 is a diagram illustrating an example of the database 330. The database 330 is stored in the storage 133, for example. For example, when the data is moving image data output from the imaging device 110, the database 330 includes information indicating the time when each frame included in the moving image data is acquired, a frame included in the moving image data, and a completion flag. Stored in association. The time is obtained based on, for example, a time stamp added to the frame. The completion flag is information indicating whether or not reception of the stream is completed. For example, when the completion flag is “0”, it indicates that the reception of the stream has not been completed. When the completion flag is “1”, it indicates that the reception of the stream has been completed.

When it is determined that the section received by the data receiving unit 304 is missing, the retransmission request unit 308 requests the terminal device 120 on the transmission side to retransmit the missing section. The complementing unit 309 supplements the missing section using the section retransmitted from the terminal device 120 on the transmission side in response to the request from the retransmission requesting unit 308.

2. Operation 2-1. Transmission Process First, a process in which the terminal apparatus 120 transmits moving image data to the transfer apparatus 130 will be described. Here, it is assumed that moving image data indicating a moving image shot by the imaging device 110 is transmitted from the terminal device 120 </ b> A to the transfer device 130. In the following description, the configuration of the terminal device 120A will be described with “A” added to the end of the reference numeral.

FIG. 12 is a sequence chart showing an example of transmission processing. The terminal device 120A negotiates with the transfer device 130 before transmitting the moving image data.

In step 401, in the terminal device 120A, the connection control unit 201A establishes a connection with the transfer device 130. In step 402, negotiation is performed between the terminal device 120A and the transfer device 130. Specifically, the message transmission unit 202A transmits a creation request message 220 as illustrated in FIG. 4 to the transfer apparatus 130 via the connection being connected.

For example, the creation request message 220 includes a request ID “1” and the number of streams “2”. This indicates that the creation request message 220 having the request ID “1” includes attribute information of two streams. The creation request message 220 includes a stream ID “1”, a retransmission flag “0”, a storage flag “1”, a measurement ID “M001”, and a transmission source ID “EAAA” that identifies the terminal device 120A. , A pair with a transfer destination ID “EBBB” for identifying the terminal device 120B is included. This is because the stream with the stream ID “1” includes the moving image data generated by the measurement with the measurement ID “M001”, and this moving image data is not retransmitted and needs to be stored in the transfer device 130. Furthermore, it indicates that it is necessary to transfer from the transfer device 130 to the terminal device 120B. Further, the creation request message 220 includes a stream ID “2”, a retransmission flag “1”, a storage flag “1”, a measurement ID “M001”, a transmission source ID “EAAA”, and a transfer destination ID “−”. Is included. This is because the stream with the stream ID “2” includes the moving image data generated by the measurement with the measurement ID “M001”, and this moving image data has been retransmitted and needs to be stored in the transfer device 130. Furthermore, it indicates that there is no need to transfer from the transfer device 130.

After the creation request message 220 is transmitted, the message transmission unit 202A transmits an information transmission message including the transmission condition of the stream with the stream ID “1” to the transfer apparatus 130 via the connection being connected. . For example, the transmission condition includes the resolution of the imaging device 110.

In step 403, when the message receiving unit 301 receives the creation request message 220 and the information transmission message in the transfer device 130, the attribute information included in the creation request message 220 and the transmission conditions included in the information transmission message are streamed. Stored in table 320. For example, in the first record of the stream table 320, as shown in FIG. 10, the stream ID “1”, the retransmission flag “0”, the storage flag “1”, the measurement ID “M001”, and the transmission source ID “ “EAAA”, the transfer destination ID “EBBB”, and the transmission condition included in the information transmission message are stored. The second record of the stream table 320 includes a stream ID “2”, a retransmission flag “1”, a storage flag “1”, a measurement ID “M001”, a transmission source ID “EAAA”, and a transfer. The destination ID “−” and the transmission condition included in the information transmission message are stored.

In step 404, when the storage of the attribute information and the transmission conditions is completed, the message transmission unit 302 transmits a response message as shown in FIG. 9 to the terminal device 120A via the connection being connected. For example, the response message 310 includes a request ID “1” and result information “0”. When the terminal device 120A receives this response message, the negotiation between the terminal device 120A and the transfer device 130 ends. When the negotiation is completed, the process of the terminal device 120A proceeds to step 405.

In step 405, when the moving image data output from the imaging device 110 is received by the acquisition unit 204A in the terminal device 120A, the generation unit 205 generates a stream using the moving image data. The generation of the stream is started before the reception of the entire moving image data is completed, and is sequentially performed while the moving image data is being received. The stream is composed of a plurality of sections shown in FIG. A stream ID “1” corresponding to this stream is added to the SOS, the plurality of units, and the EOS included in each section of the stream by the adding unit 109A. Furthermore, an end flag indicating the end of the stream is added to the last section constituting the stream by the adding unit 109A. This end flag may be used as last information indicating the end of data.

In step 406, the data transmission unit 207A sequentially transmits the stream to which the streaming ID is added to the transfer apparatus 130 via the connected connection. Transmission of this stream is started before the reception of the entire moving image data is completed, and is sequentially performed while the moving image data is being received. In the transfer device 130, the data reception unit 304 sequentially receives the streams transmitted from the terminal device 120A. When the transmission of this stream ends, the terminal device 120A proceeds to step 407.

In step 407, the connection control unit 201A disconnects the connection with the transfer apparatus 130. In step 408, the transfer device 130 deletes all the attribute information and transmission conditions stored in association with the stream IDs “1” and “2” from the stream table 320 in response to the disconnection of the connection. As a result, the association between the stream IDs “1” and “2” stored in the stream table 320 and the connection is released, and the stream IDs “1” and “2” stored in the stream table 320 are released. In this case, when other connections are established, these stream IDs can be used as identifiers of other streams. Note that the same processing as in step 408 may be performed even when the connection is disconnected without depending on the terminal device 120.

Note that the attribute information stored in the stream table 320 may be changed during connection. In this case, in the terminal device 120A, the message transmission unit 202A transmits a change request message including the change target stream ID and the changed attribute information to the transfer device 130. The stream table 320 is updated according to the change request message. For example, when a change request message including a combination of the stream ID “1” and the changed attribute information is transmitted, the attribute information associated with the stream ID “1” in the stream table 320 includes the attribute included in the change request. Rewritten with information.

Further, the stream ID stored in the stream table 320 may be released during connection. In this case, in the terminal device 120A, the message transmitting unit 202A transmits this stream ID release request message to the transfer device 130. The release request message may use the same format as the creation request message 220 shown in FIG. For example, when requesting the release of the stream ID “1”, a release request message including the stream ID “1” and the transmission source ID “0” is transmitted. When the release request message is received from the terminal device 120, the record including the stream ID “1” is deleted from the stream table 320. As a result, the association between the stream ID “1”, the retransmission flag “0”, the storage flag “1”, the measurement ID “M001”, the transmission source ID “EAAA”, and the transfer destination ID “EBBB” is released. . In this case, the stream ID “1” is released and can be used as an identifier of another stream. Thereby, even when a large number of streams are transmitted, it is possible to prevent the available stream IDs from being exhausted.

2-2. Transfer Process Next, a process in which the transfer device 130 transfers moving image data to another terminal device 120 will be described. Here, it is assumed that the moving image data transmitted from the terminal device 120A is transferred from the transfer device 130 to the terminal device 120B. In the following description, the configuration of the terminal device 120B will be described with “B” appended to the end of the reference numeral.

FIG. 13 is a sequence chart showing an example of the transfer process. The terminal device 120B negotiates with the transfer device 130 before receiving the moving image data.

In step 501, in the terminal device 120B, the connection control unit 201B establishes a connection with the transfer device 130. In step 502, negotiation is performed between the terminal device 120 </ b> B and the transfer device 130. Specifically, the message transmission unit 202B transmits a creation request message 230 as illustrated in FIG. 5 to the transfer apparatus 130 via the connection being connected.

For example, the creation request message 230 includes a request ID “2”, the number of streams “1”, a stream ID “3”, a transmission source ID “EBBB”, and a transfer source ID “EAAA”. This is because the creation request message 230 with the request ID “2” includes the attribute information of one stream with the stream ID “3”, and the transfer source of the moving image data included in this stream is the terminal device 120A. It shows that there is.

In addition, after the creation request message 230 is transmitted, the message transmission unit 202B transmits an information transmission message regarding the stream with the stream ID “3” to the transfer apparatus 130 via the connected connection. For example, when the terminal device 120B supports MPEG-4, the information transmission message may include information indicating MPEG-4. For example, when the resolution of the display device 125B is R1, the information transmission message may include the resolution R1. For example, when the maximum frame rate that can be displayed by the display device 125B is F1, the frame rate F1 may be included in the information transmission message.

When the information transmission message is received by the message reception unit 301, in the transfer device 130, the determination unit 303 determines the transmission condition of the stream based on the information included in the information transmission message. For example, when information indicating MPEG-4 is included in the information transmission message, a compression method called MPEG-4 may be determined. For example, when the information transmission message includes the resolution R1, the resolution R1 may be determined. For example, when the frame rate F1 is included in the information transmission message, the frame rate F1 may be determined.

In step 503, the transfer apparatus 130 stores the attribute information included in the creation request message 230 and the transmission condition determined by the determination unit 303 in the stream table 320. For example, the stream table 320 corresponds to the stream ID “3”, the transmission source ID “EBBB”, the transfer source ID “EAAA”, and the transmission conditions determined by the determination unit 303, as shown in FIG. Stored with. Note that the retransmission flag, the storage flag, and the measurement ID are not stored because they are not included in the creation request message 230.

In step 504, the message transmission unit 302 transmits the response message 310 shown in FIG. 9 to the terminal device 120B via the connection being connected. In the example, the response message 310 includes a request ID “2” and result information “0”. When the terminal device 120B receives the response message, the negotiation between the terminal device 120B and the transfer device 130 ends. When the negotiation is completed, the processing of the transfer device 130 proceeds to step 505 shown in FIG.

In step 505, in the transfer device 130, when the data reception unit 304 starts receiving the stream from the terminal device 120A, the determination unit 305 determines whether or not to transfer the stream in real time. Specifically, when the SOS placed at the head of the section is received, the stream ID “1” is extracted from the SOS. As shown in FIG. 10, the stream table 320 stores a stream ID “1” and a retransmission flag “0” in association with each other. This indicates that the stream with the stream ID “1” is transferred in real time. Therefore, in this case, it is determined that the stream is transferred in real time. If it is determined that the stream is transferred in real time (YES in step 505), the processing of the transfer device 130 proceeds to step 506 shown in FIG.

In step 506, the processing unit 306 processes the moving image data included in the received stream based on the transmission condition stored in the stream table 320 in association with the stream ID “3”. The processing of the moving image data is started before the section EOS is received, and sequentially performed while the section is received. For example, when MPEG-4 is included in the transmission condition, the moving image data may be compressed according to MPEG-4. For example, when the resolution R1 is included in the transmission condition, when the resolution of the moving image data is higher than R1, the resolution of the moving image data may be changed to R1. For example, when the frame rate F1 is included in the transmission condition and the frame rate of the moving image data is higher than F1, the frame rate of the moving image data may be changed to F1, for example, by thinning out the moving image data.

In step 507, the data transmission unit 307 sequentially transmits the stream including the processed moving image data to the terminal device 120B. Transmission of this stream is started before the section's EOS is received, and is performed sequentially while the section is being received.

In step 508, when the data receiving unit 304 receives the EOS of the section in the transfer device 130, the frame included in the section is duplicated and stored in the database 330 shown in FIG. In this example, the database 330 stores the first to nth frames (n is a natural number) included in a section and the times T1 to Tn (n is a natural number) at which these frames are acquired in association with each other. The These times T1 to Tn may be absolute times obtained from the relative time indicated by the time stamp added to each frame. For example, in the case where the video recording start time is 12:00, when the time stamp added to the first frame indicates an elapsed time of 12 minutes, the time T1 is 12:12 Good. The processing in step 508 is performed every time reception of the section is completed. When the end flag added to the last section included in the stream is received, the determination unit 305 determines that reception of the stream is completed. In this case, “1” is stored in the database 330 as a completion flag as shown in FIG.

Returning to FIG. 13, in step 509, when the terminal 120B receives the stream at the data receiving unit 209B, the processing unit 210B displays the moving image according to the moving image data included in the stream on the display device 125B. The This moving image display is started before reception of the entire stream is completed, and is sequentially performed while the stream is being received. When the moving image data is compressed according to MPEG-4, the moving image data is used after being decoded according to MPEG-4. Thereby, the user of the terminal device 120B can browse the moving image. When the reception of the stream is completed, the terminal device 120B may perform the same processing as in steps 407 and 408 described above.

On the other hand, when the stream ID included in the SOS placed at the head of the section is associated with the retransmission flag “1” in the stream table 320 in step 505 described above, for example, the determination unit 305 causes the stream to be transmitted in real time. It is determined not to transfer (NO in step 505). In this case, the processing of the transfer device 130 proceeds to step 508 without performing the processing of steps 506 and 507 described above. That is, in this case, the moving image data is not transferred in real time. However, when the terminal device 120B requests transmission of moving image data after the moving image data is stored in the database 330, the above-described processing in steps 506 and 507 may be performed in response to this request. In this case, the moving image data is transferred to the terminal device 120B in non-real time.

2-3. Retransmission process In the transmission process described above, if there is a missing section, a retransmission process may be performed.

FIG. 14 is a sequence chart showing an example of retransmission processing. This retransmission processing may be performed every time reception of the section transmitted from the terminal device 120A is completed in the transfer device 130, for example.

In step 601, in the transfer apparatus 130, the number of units included in the section received by the data reception unit 304 is counted by the retransmission request unit 308. In step 602, based on the difference between the number of units counted by the retransmission request unit 308 and the number of units indicated by the unit count included in the EOS of the received section, whether or not there is a missing section. Is determined. If the numbers of these units match, it is determined that there is no missing section (NO in step 602). In this case, for example, an ACK (acknowledgement) message is transmitted from the transfer apparatus 130 to the terminal apparatus 120A, and then this process ends. This ACK message may include SOS and EOS included in the received section. On the other hand, if the number of these units is different, it is determined that there is a section defect (YES in step 602). In this case, the processing of the transfer device 130 proceeds to step 603.

In step 603, the message transmission unit 302 transmits a message requesting retransmission of the missing section (hereinafter referred to as “retransmission request message”) to the terminal device 120A. The retransmission request message may include information for identifying a missing section, for example. In step 604, when the terminal apparatus 120A receives the retransmission request message, the data retransmission unit 208A transmits a stream including the requested section to the transfer apparatus 130. Since this section is retransmitted, for example, a stream ID “2” is added to the SOS, the plurality of units, and the EOS included in this section, respectively.

When the stream is received from the terminal device 120A, the processing of the transfer device 130 proceeds to step 505 shown in FIG. In this case, for example, the stream ID “2” is included in the SOS of the retransmitted section included in the stream. As illustrated in FIG. 10, the stream table 320 stores a stream ID “2” and a retransmission flag “1” in association with each other. This indicates that the stream with the stream ID “2” is not transferred in real time. Therefore, in this case, the determination unit 305 determines not to transfer the stream in real time (NO in step 505). In this case, the processing of the transfer device 130 proceeds to step 508 described above. In step 508, the frame included in the retransmitted section is stored in the database 330 by the complementing unit 309. For example, when the first frame is included in the retransmitted section, the first frame stored in the database 330 may be overwritten by the newly received first frame. This complements the missing section.

According to the embodiment described above, the moving image data transmitted from the terminal device 120A to the transfer device 130 is processed according to the transmission condition determined by the negotiation between the transfer device 130 and the terminal device 120B, and then the terminal device. 120B. Therefore, even when the moving image data received by the transfer device 130 does not meet the request of the terminal device 120B, the moving image data according to the request can be provided to the terminal device 120B. In addition, since the transfer device 130 starts to transfer the moving image data in response to receiving the SOS of the section from the terminal device 120A, compared to the case where the transfer of the moving image data is started after reception of the entire section is completed. Thus, the transfer delay of moving image data is reduced.

3. The present invention is not limited to the above-described embodiment. The embodiment described above may be modified as follows. Moreover, you may implement combining the following two or more modifications.

In the embodiment described above, a plurality of imaging devices 110 may be connected to the terminal device 120A. In this case, when there are a plurality of moving image data having the terminal device 120B as a transfer destination, the transmission condition may include information for identifying the moving image data requested by the terminal device 120B. For example, when there is first to tenth moving image data from the first moving image data with the terminal device 120B as the transfer destination, when the terminal device 120B uses only the first moving image data and the third moving image data, the transmission condition includes the first Information for identifying the moving image data and the third moving image data may be included. In this case, the data transmission unit 307 may transmit only the first moving image data and the third moving image data among the first moving image data to the tenth moving image data to the terminal device 120B.

In the embodiment described above, a plurality of data may be transferred in parallel. For example, when the imaging device 110 and the measurement device are connected to the terminal device 120A, the moving image data output from the imaging device 110 and the measurement data output from the measurement device may be transferred in parallel. The measurement data includes a plurality of measurement values measured along the time series. In this case, when receiving the moving image data and the measurement data, the terminal device 120B may output these data along the same time axis.

FIG. 15 is a diagram showing an output example of moving image data and measurement data. In this example, a moving image corresponding to moving image data and a measured value corresponding to measurement data are displayed side by side along the same time axis on the display device 125B of the terminal device 120B. At this time, a moving image and a measured value corresponding to the same time are displayed simultaneously. For example, a moving image shot at time T10 and a measured value measured at time T10 are displayed simultaneously. Thereby, the user of the terminal device 120 </ b> B can browse the moving image shot at a certain time and the measured value measured at the time.

In the above-described embodiment, the display device 125 may display data on a time scale specified by a user operation, for example. Here, it is assumed that the data includes a plurality of measurement values measured along the time series. For example, when a time scale of 1 second intervals is designated, the measurement values may be displayed on the display device 125 at 1 second intervals. In this case, the transmission condition may include a designated time scale. In this case, the processing unit 306 may thin out data according to a designated time scale. For example, when the time scale is 1 second, data may be thinned out so that only one measurement value is included in one second. Thereby, it is possible to prevent unnecessary data from being transmitted in the terminal device 120 on the receiving side.

In the above-described embodiment, the processing unit 210 performs processing for displaying data, but the processing performed by the processing unit 210 is not limited to data display. For example, the processing unit 210 may perform processing for controlling or operating another device using data.

In the embodiment described above, the processing unit 306 may process the data before the data is stored in the database 330. In this case, the database 330 stores processed data.

In the above-described embodiment, the information included in the transmission condition is not limited to the compression method supported by the terminal device 120, the resolution supported by the display device 125, and the frame rate supported by the display device 125. For example, the transmission condition may include a bit rate supported by the terminal device 120 and an image size supported by the display device 125.

In the above-described embodiment, the information transmission message may include information indicating the status of communication between the transfer device 130 and the terminal device 120B. The communication status may include a communication type, a communication method, a communication band, or a communication speed. In this case, the processing unit 306 may thin out the data according to the communication status. For example, when the current communication band is smaller than a predetermined band, data may be thinned out according to the current communication band. Thereby, even when the communication band is small, the data transfer delay is reduced.

In the above-described embodiment, the transfer device 130 may transmit an error message to the terminal device 120B when the transmission condition received from the terminal device 120A does not match the transmission condition determined by the determination unit 303. For example, when the resolution of the imaging device 110 is lower than the resolution of the display device 125B of the terminal device 120B, the transmission condition determined by the determination unit 303 cannot be satisfied, and therefore an error message is transmitted to the terminal device 120B. Good.

In the above-described embodiment, data used in the transfer system 100 is not limited to moving image data. For example, sensor data output from the sensor may be used. This sensor detects a detection amount from the sensing target, and outputs sensor data indicating the detected detection amount. In this sensor, in addition to the imaging device 110 described above, for example, when the sensing target is a car, the sensor includes a tachometer that measures the number of revolutions of the engine, a speedometer that measures the speed of the car, and a steering angle. Various measuring devices provided in the vehicle such as an angle meter, a fuel injection amount meter, a thermometer, and a GPS (Global Positioning System) that measures the position of the vehicle may be used.

In the above-described embodiment, as illustrated in FIG. 11, when the completion flag of the database 330 is “1”, the data transmission unit 307 transmits all the data stored in the database 330 to a predetermined processing unit. May be. The predetermined processing unit may perform predetermined later-stage processing such as analysis processing and batch processing using data, for example.

In the above-described embodiment, when serial numbers are assigned to each section in chronological order, this number may be stored in the database 330 instead of the information indicating the time.

In the above-described embodiment, the stream ID is included in the SOS, but the retransmission flag itself may be included in the SOS.

In the embodiment described above, the length of each section may be an arbitrary length. Furthermore, the number of units included in each section need not necessarily be the same. The number of units included in one section may vary from section to section.

In the embodiment described above, the stream ID may be added to only a part of the SOS, the plurality of units, and the EOS included in each section of the stream. For example, the stream ID may be added only to the SOS, or may be added only to the EOS. Thereby, the data amount regarding the attribute of data can further be reduced.

In the embodiment described above, the stream does not necessarily have to be composed of a plurality of sections. For example, a stream may include only a plurality of units. In this case, the stream does not need to include SOS and EOS.

In the embodiment described above, the data transmitted from the terminal device 120 to the transfer device 130 is not necessarily limited to data output from an external device. For example, this data may be data generated by the terminal device 120.

The steps of processing performed in the transfer system 100 are not limited to the example described in the above-described embodiment. The steps of this process may be interchanged as long as there is no contradiction. The present invention may be provided as a transfer method including steps of processing performed in the transfer system 100 or the transfer apparatus 130.

The present invention may be provided as a program executed in the terminal device 120 or the transfer device 130. This program may be downloaded via a communication line such as the Internet. These programs are provided in a state where they are recorded on a computer-readable recording medium such as a magnetic recording medium (magnetic tape, magnetic disk, etc.), an optical recording medium (optical disk, etc.), a magneto-optical recording medium, or a semiconductor memory. May be.

As used herein, the terms “first” and “second” are used only to distinguish one element from another, and do not limit the amount or order of the elements. . Therefore, these terms may be used, for example, by exchanging numbers.

100: transfer system 110: imaging device 120: terminal device 130: transfer device 201: connection control unit 202: message transmission unit 203: message reception unit 204: acquisition unit 205: generation unit 206: Addition unit, 207: Data transmission unit, 208: Data retransmission unit, 209: Data reception unit, 210: Processing unit, 301: Message reception unit, 302: Message transmission unit, 303: Determination unit, 304: Data reception unit, 305 : Determination unit, 306: processing unit, 307: data transmission unit, 308: retransmission request unit, 309: complementing unit

Claims (13)

1. A receiving unit for receiving data in time series from the transmitting device;
   A determination unit that determines a transmission condition of the data by exchanging information with a receiving device;
   A processing unit that processes the data according to the transmission condition determined by the determination unit;
   When transferring the data in real time, a transmission unit that transmits the data processed by the processing unit to the receiving device;
   And a storage unit that stores the data along the time series when the data is not transferred in real time.
2. The transfer device according to claim 1, wherein when the data is transferred in real time, the storage unit stores a copy of the data after the data is transmitted to the receiving device.
3. Real time information indicating whether or not to transfer the data in real time is added to the top of the data,
   When the head of the data is received by the receiving unit, further comprising a determination unit for determining whether to transfer the data in real time based on the real time information,
   When the determination unit determines that the data is to be transferred in real time, the processing unit starts processing the received data before the reception of the data is completed, and the received data Are processed sequentially,
   The transmission unit starts transmission of the processed data before the reception of the data is completed when the determination unit determines to transfer the data in real time, and the processed data The transfer device according to claim 1 or 2, wherein
4. The storage unit stores the real-time information and the data identifier in association with each other,
   The identifier is added to the head of the data instead of the real-time information,
   The determination unit determines whether or not to transfer the data in real time based on the real-time information stored in the storage unit in association with the identifier added to the head of the data. 4. The transfer device according to 3.
5. The data is video data indicating a video,
   The receiving device has a display device that displays the moving image indicated by the moving image data,
   The information includes a resolution or a frame rate corresponding to the display device,
   The transfer device according to claim 1, wherein the processing unit changes a resolution or a frame rate of the moving image data according to the resolution or the frame rate.
6. The receiving device has a display device that displays the data on a specified time scale,
   The information includes the time scale;
   The transfer device according to claim 1, wherein the processing unit thins out the data according to the time scale.
7. The information includes a status of communication with the receiving device,
   The transfer device according to claim 1, wherein the processing unit thins out the data in accordance with the communication status.
8. At the end of the data transmitted from the transmission device, last information indicating the end is added,
   When the last information is received, further comprising a determination unit that determines that the reception of the data is completed,
   The transmission unit transmits the data stored in the storage unit to a predetermined processing unit when the determination unit determines that the reception of the data is completed. The transfer device described.
9. The data transmitted by the transmission device includes a plurality of data blocks each including at least one of a plurality of detection amounts detected by a sensor, and each of the plurality of data blocks includes a data block. Number information indicating the number of the at least one detection amount included is added,
   It is determined whether or not the data block is missing according to a difference between the number of the at least one detection amount included in the data block and the number indicated by the number information added to the data block. A determination unit to perform,
   When the determination unit determines that the data block is missing, a retransmission request unit that requests the transmission device to retransmit the missing data block;
   The supplementary part which complements the said missing data block using the said data block retransmitted from the said transmission apparatus according to the request | requirement of the said resending request part is further provided. Transfer device.
10. A transfer device, a transmission device, and a reception device;
    The transfer device is
    A first receiver for receiving data in time series from the transmitter;
    A determining unit that determines a transmission condition of the data by exchanging information with the receiving device;
    A processing unit that processes the data according to the transmission condition determined by the determination unit;
    When transferring the data in real time, a first transmission unit that transmits the data processed by the processing unit to the receiving device;
    In the case where the data is not transferred in real time, the storage unit stores the data along the time series,
    The transmitter is
    An acquisition unit for acquiring the data;
    A second transmission unit that transmits the data acquired by the acquisition unit to the transfer device;
    The receiving device is:
    A third transmitter for transmitting the information to the transfer device;
    A second receiver for receiving the data transmitted from the transfer device;
    And a processing unit that performs processing using the data received by the second receiving unit.
11. The data includes moving image data indicating a moving image taken by the imaging device, and measurement data output from the measurement device,
    The transfer system according to claim 10, wherein the processing unit outputs the moving image data and the measurement data along the same time axis.
12. Receiving data in time series from the transmitting device;
    Determining transmission conditions for the data by exchanging information with a receiving device;
    Processing the data according to the determined transmission conditions;
    When transferring the data in real time, transmitting the processed data to the receiving device;
    And a step of storing the data along the time series when the data is not transferred in real time.
13. On the computer,
    Receiving data in time series from the transmitting device;
    Determining transmission conditions for the data by exchanging information with a receiving device;
    Processing the data according to the determined transmission conditions;
    When transferring the data in real time, transmitting the processed data to the receiving device;
    A program for executing the step of storing the data along the time series when the data is not transferred in real time.

Images