WO2023162130A1 - Système, procédé et programme de collecte de données - Google Patents

Système, procédé et programme de collecte de données Download PDF

Info

Publication number
WO2023162130A1
WO2023162130A1 PCT/JP2022/007865 JP2022007865W WO2023162130A1 WO 2023162130 A1 WO2023162130 A1 WO 2023162130A1 JP 2022007865 W JP2022007865 W JP 2022007865W WO 2023162130 A1 WO2023162130 A1 WO 2023162130A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
communication path
server
data source
connection request
Prior art date
Application number
PCT/JP2022/007865
Other languages
English (en)
Japanese (ja)
Inventor
諒平 津上
達也 福井
Original Assignee
日本電信電話株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 日本電信電話株式会社 filed Critical 日本電信電話株式会社
Priority to PCT/JP2022/007865 priority Critical patent/WO2023162130A1/fr
Publication of WO2023162130A1 publication Critical patent/WO2023162130A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/1066—Session management
    • H04L65/1069—Session establishment or de-establishment

Definitions

  • This disclosure relates to communication control for data collection.
  • RDMA Remote Direct Memory Access
  • RDMA performs DMA transfer of data from the memory of a local computer to the memory of a different remote computer (direct data transfer between peripheral devices, main memory (RAM), etc. without CPU intervention). Therefore, since RDMA does not require CPU processing for data transfer, it is possible to avoid the aforementioned bottleneck on the receiving side.
  • Non-Patent Document 1 A remote transfer technology using this RDMA has been proposed (see, for example, Non-Patent Document 1).
  • Non-Patent Document 1 in order to transfer data by RDMA in a wide area communication network, it is necessary to set a communication path for each flow in advance. Therefore, collecting data from many data sources requires a large amount of network resources.
  • the present disclosure aims to provide a system that does not require a large amount of network resources even when collecting data from many data sources.
  • the systems and methods of the present disclosure include: A system for collecting data from multiple data sources on a server and a method performed by the system, comprising: a network device for transferring data from the plurality of data sources to the server is determined for each data source; When data to be transmitted to the server is generated, each of the plurality of data sources transmits a communication path connection request to a network device determined in the own device.
  • the data source apparatus and method of the present disclosure are A method executed by the data source device and the data source provided in a system that collects data from a plurality of data source devices on a server, the data source device When data to be transmitted to the server is generated, a communication path connection request is transmitted to the network device determined in the self-device.
  • Devices such as the controller and data source of the present disclosure can also be realized by a computer and a program, and the program can be recorded on a recording medium or provided through a communication network.
  • the program of the present disclosure is a program for realizing a computer as each functional unit provided in the apparatus according to the present disclosure, and is a program for causing the computer to execute each step included in the method executed by the apparatus according to the present disclosure. .
  • 1 shows a system configuration example of the present disclosure
  • 1 shows a configuration example of a communication network
  • a configuration example of a server is shown.
  • a configuration example of a data source is shown.
  • An example of information stored in the requirement table of each data source is shown.
  • 4 shows a configuration example of a transmission control controller; It is an example of a transmission policy decision sequence in the present disclosure.
  • An example of the operation in the transmission policy determination function is shown.
  • 4 shows an example of information stored in a transmission policy table;
  • An example of the Attribute ID area is shown.
  • An example of a sequence for creating and releasing a communication path is shown.
  • An example of a sequence for creating and releasing a communication path is shown.
  • 4 shows an example of transmission of sensing data from data source B to the server.
  • 4 shows an example of transmission of sensing data from data source A to the server.
  • An example of transmission of sensing data from data source C to the server is shown.
  • Fig. 1 shows a system configuration example of the present disclosure.
  • the system of the present disclosure comprises multiple data sources 10 , a transmission controller 20 and a server 40 .
  • a plurality of data sources 10 , transmission controllers 20 and servers 40 are connected by a communication network 80 .
  • a data source 10 is a terminal having user-side sensing data.
  • the server 40 is a terminal that collects data from data sources.
  • the data source 10, transmission controller 20, and server 40 of the present disclosure can also be realized by a computer and a program, and the program can be recorded on a recording medium or provided through a communication network.
  • FIG. 2 shows a configuration example of the communication network 80.
  • a communication network 80 is composed of a plurality of network devices 81 .
  • a network device 81 for transferring data from the data source 10 to the server 40 is determined for each data source 10 .
  • the data source 10A and server 40 are connected by network devices 81A and 81B
  • the data source 10B and server 40 are connected by network devices 81A and 81C
  • the data source 10C and server 40 are connected by network devices 81D and 81E.
  • each of the network devices 81A-81E When each of the network devices 81A-81E receives data addressed to the server 40 from the data sources 10A-10C, it knows to which network device it should be transferred. Therefore, even if the data sources 10A to 10C do not know which network device 81 to use, if they specify the server 40 and send data, the data will be sent to the server 40 . Thus, the present disclosure can transfer data losslessly and broadband when a communication path is being created.
  • This disclosure proposes a method of allocating lossless and broadband communication paths to required data sources 10 at required timings in order to realize data transfer by RDMA.
  • each of the plurality of data sources 10 when data to be transmitted to the server 40 is generated, transmits the network device 81 determined in the device at the transmission timing determined in the transmission policy of the device itself. to generate a communication path.
  • each data source 10 can autonomously transmit sensing data to the server 40 based on its own transmission policy.
  • Transmission policies can be exemplified by the following, for example.
  • First transmission policy Immediately after generation of data, a communication path is generated and data is transferred, and immediately after completion of data transfer, the communication path is released.
  • Second transmission policy After a certain amount of data is accumulated, a communication path is generated and data is transferred, and the communication path is released immediately after the completion of data transfer.
  • Third transmission policy A communication path is generated immediately after data is generated, data is transferred, and the communication path is maintained for a certain period of time after data transfer is completed.
  • Fourth transmission policy After a certain amount of data is accumulated, a communication path is generated and data is transferred, and the communication path is maintained for a certain period of time even after the data transfer is completed.
  • a transmission policy that meets the data transfer requirements of the data source 10 is distributed. Then, each data source 10 autonomously transmits sensing data according to a transmission policy that considers sensor characteristics and data transfer requirements for each application. As a result, the present embodiment controls the trade-off between immediacy of data transmission and shortening of communication path usage time.
  • the data source 10 directly transmits communication path connection requests to the network devices 81A to 81E.
  • the network devices 81A to 81E By reflecting the settings received by the network devices 81A to 81E, it is possible to save time and effort until the communication paths are generated. Thereby, the present disclosure can set up a communication path in a shorter time. A detailed description will be given below.
  • FIG. 3 shows a configuration example of the server 40.
  • the server 40 has a data reception function 41 , a memory 42 and an application 43 .
  • the data reception function 41 receives sensing data from each data source 10 .
  • Memory 42 stores sensing data from sensor 50 .
  • Application 43 collects sensing data from sensor 50 .
  • the application 43 is any application that collects any sensing data detected or generated by a user terminal, sensor, in-vehicle system, or the like.
  • FIG. 4 shows a configuration example of the data source 10.
  • the data source 10 includes a requirement notification function 11, a transmission policy reception function 12, a data transmission timing control function 13, a communication path setting function 14, a data transmission function 15, a communication path release function 16, a generated data storage function 17, a requirement table 18, A transmission policy table 19 is provided.
  • the generated data storage function 17 stores sensing data from the sensor 50 .
  • the requirement notification function 11 reads data transfer requirements from the requirement table 18 and notifies the transmission controller 20 of the read data transfer requirements.
  • the transmission policy reception function 12 receives transmission policies from the transmission control controller 20 .
  • Data transmission timing control function 13 controls communication path setting function 14 , data transmission function 15 and communication path release function 16 according to transmission policy table 19 .
  • the communication path setting function 14 transmits a connection request for creating a communication path with the server 40 to the network device 81 .
  • the data transmission function 15 transmits sensing data stored in the generated data storage function 17 to the server 40 .
  • the communication path release function 16 transmits a release request for releasing the communication path with the server 40 to the network device 81 .
  • the requirement table 18 stores data transfer requirements for each data source 10 .
  • the transmission policy table 19 stores policies for transmitting sensing data from the data source 10 to the server 40 .
  • FIG. 5 shows an example of information stored in the requirement table 18 of each data source.
  • the permissible delay time is the permissible delay time from when data is generated until it reaches the server.
  • the amount of data generated is the amount of data generated in the sensor 50 at one time.
  • FIG. 6 shows a configuration example of the transmission controller 20.
  • the transmission controller 20 has a requirement reception function 21 , a transmission policy determination function 22 , a transmission policy delivery function 23 and a transmission policy determination rule 24 .
  • FIG. 7 shows an example of the operation of the transmission controller 20.
  • Each data source 10 notifies the transmission controller 20 of data transfer requirements based on the information in the requirement table 18 (S101).
  • the data transfer requirements include the allowable delay time and data generation frequency of each data source 10, as shown in FIG.
  • a requirement reception function 21 of the transmission control controller 20 receives data transfer requirements from the data source 10 .
  • the transmission policy determination function 22 determines the transmission policy for each data source 10 according to the data transfer requirements according to the transmission policy determination rule 24 (S102).
  • the transmission policy distribution function 23 of the transmission controller 20 distributes the determined transmission policy to each data source 10 (S103).
  • FIG. 8 shows an example of the operation of the transmission policy determination function 22.
  • the transmission controller 20 acquires the data transfer requirements from the data source 10 (S11), it determines the allowable delay time requirements (S12). If the allowable delay time requirement is 100 ms or less (Yes in S12), a method of generating and transmitting a path immediately after data generation is determined (S13). On the other hand, if the allowable delay time requirement is more than 100 ms (No in S12), a method of storing a certain amount of sensing data and transmitting it is determined (S14). Next, the transmission controller 20 determines the data generation frequency (S15).
  • the method of releasing the path immediately after data transmission is determined (S16). On the other hand, if the data generation frequency is more than 4 times/s (No in S15), the method of waiting for a certain period of time after completion of data transmission and releasing the path is determined (S17).
  • the transmission policy determination function 22 determines as follows. ⁇ Data source A: Since the allowable delay time is set to 1000 ms and the data generation frequency is set to 4, the transmission controller 20 determines the second transmission policy as the transmission policy. ⁇ Data source B: Since the allowable delay time is set to 200 ms and the data generation frequency is set to 10 times, the transmission controller 20 determines the fourth transmission policy as the transmission policy. ⁇ Data source C: Since the set time of the allowable delay time is 100 ms and the set value of the data generation frequency is 5 times, the transmission controller 20 determines the third transmission policy as the transmission policy. As a result, transmission policies as shown in FIG. 9 are given to data sources A to C. FIG.
  • the data transfer requirements may include application information.
  • the transmission controller 20 in steps S12 and S15, sets the threshold considering the requirements contained in the application information.
  • the data source 10 When the data source 10 receives the transmission policy from the transmission controller 20, it stores it in the transmission policy table 19. As a result, the transmission policy suitable for each data source 10 is stored in the transmission policy table 19 of each data source 10, as shown in FIG.
  • the data source 10 transmits a communication path connection request to the network device 81 when sensing data is generated.
  • the received device 81 reflects the setting. For example, when the data source 10A transmits sensing data, the data source 10A transmits a communication path connection request to the network device 81B. The network device 81B then creates a communication path with the data source 10A.
  • the connection destination of the sensing data communication path is set in advance.
  • predetermined network devices 81B and 81A connecting between data source 10A and server 40 create communication paths between data source 10A and server 40.
  • the communication path can use any means capable of transmitting sensing data, and may be virtual such as VLAN or physical such as optical path.
  • Data in any format readable by the network device 81 can be used for the communication path connection request.
  • RDMA Remote Direct Memory Access
  • Fig. 10 shows an example of the Attribute ID area.
  • RoCE RDMA over Converged Ethernet
  • UDP User Datagram Protocol
  • the network device 81 can identify the negotiation based on the description of the Attribute ID field in the MAD Header.
  • the MAD Header is stored in the DATH Header in the Base Transport Header.
  • Attribute ID is determined for each negotiation. For example, if Attribute ID is 0x0010, it can be used as a trigger for path generation in network device 81B.
  • Fig. 11 shows an example of a sequence for creating and releasing a communication path.
  • the data source 10A When using RDMA, the data source 10A generates a communication path to the server 40 with ConnectRequest as a trigger. At this time, the network device 81B transfers the ConnectRequest to the next network device 81A. The network device 81A also transfers the ConnectRequest to the server 40 after completing the generation of the communication path.
  • the server 40 uses the generated communication path to send ConnectReply to the data source 10A.
  • the data source 10A transmits ReadyToUse to the server 40 upon receiving the ConnectReply from the server 40 . This enables RDMA communication from the data source 10A to the server 40.
  • FIG. 11 shows an example of a sequence for creating and releasing a communication path.
  • the data source 10A releases the communication path to the server 40 with the Disconnect Request as a trigger.
  • the network device 81B transfers the Disconnect Request to the network device 81A.
  • the server 40 receives the Disconnect Request, it sends a Disconnect Reply to the data source 10A.
  • the timing for transferring the ConnectRequest in the network devices 81B and 81A is, for example, after the completion of the generation of the communication path.
  • the present disclosure is not so limited.
  • the network devices 81B and 81A may transfer the ConnectRequest without waiting for the completion of communication path generation.
  • Fig. 12 shows an example of a sequence for creating and releasing a communication path.
  • the network devices 81B and 81A forward the ConnectRequest without waiting for the completion of communication path generation.
  • the network devices 81B and 81A have a function of transmitting a path setting completion notification to the data source 10A, which is the source of the ConnectRequest.
  • the data source 10A since the transmission route of the sensing data is determined in advance, the data source 10A counts the number of path setting completion notifications transmitted from the network devices 81B and 81A so that all the network devices 81B on the transmission route and 81A, it can be confirmed that the setting of the communication path has been completed. After this confirmation, the data source 10A transmits ReadyToUse to the server 40 . This enables RDMA communication from the data source 10A to the server 40.
  • the data source 10A When releasing the communication path, the data source 10A sends a Disconnect Request to the server 40. Server 40 releases the communication path and sends a DisconnectReply to data source 10A. Since the communication paths are released in the server 40, the communication paths are also released in the network devices 81A and 81B.
  • FIG. 13 shows an example of transmission of sensing data from the data source 10B to the server 40.
  • the data source 10B transmits sensing data according to the fourth transmission policy.
  • the data source 10B accumulates the sensing data generated by the sensor 50, and when the data reaches 20 MB, it creates a communication path with the server 40 (S202).
  • the data source 10B transmits sensing data to the server 40 (S204-1, S204-2).
  • the data source 10B waits for the next 50 ms before transmitting a communication path release request.
  • the sensing data is generated during that time, the sensing data is transmitted each time (S204-3).
  • the data source 10B releases the communication path (S205) when 50 ms have passed since the last data transmission (step S204-3).
  • FIG. 14 shows an example of transmission of sensing data from the data source 10A to the server 40.
  • the data source 10A transmits sensing data according to the second transmission policy.
  • the data source 10A accumulates sensing data generated by the sensor 50, and when the data reaches 15 MB, it creates a communication path with the server 40 (S302).
  • the data source 10A collectively transmits the accumulated sensing data to the server 40 (S304-1, S304-2, S304-3).
  • the data source 10A releases the communication path (S305).
  • FIG. 15 shows an example of transmission of sensing data from the data source 10C to the server 40.
  • the data source 10C transmits sensing data according to the third transmission policy.
  • the data source 10C creates a communication path with the server 40 (S402).
  • the data source 10C transmits sensing data to the server 40 (S404-1), and then waits for transmission of a communication path release request for 20 ms.
  • the sensing data is transmitted each time (S404-2, S404-3).
  • the data source 10C releases the communication path (S405) when 20 ms have passed since the last data transmission (step S404-3).
  • this embodiment prevents exhaustion of network resources and realizes RDMA communication with a large amount of data sources by time-divisionally allocating communication paths at necessary timings. Since a communication path is allocated to the data source 10 to which sensing data is to be transmitted only for the required period, the utilization rate of the path can be improved and network resources can be reduced.
  • the present disclosure determines a transmission policy for each data source 10 and generates communication paths for each data source 10 according to the transmission policy, so communication paths can be allocated in a time division manner at required timings. . For this reason, the present disclosure shortens the communication path utilization time from each data source 10 to the server 40, facilitates reuse of the communication path, and reduces the trade-off between data transmission immediacy and communication path utilization time reduction. It can be controlled to reduce the number of required communication paths for the communication network 80 as a whole.
  • the present disclosure can realize a lossless and broadband communication network 80 with communication paths, and enable RDMA communication with multiple data sources without requiring a large amount of network resources. Furthermore, the present disclosure can prevent data loss in the communication network 80, so that data transfer using reliable RDMA can be realized.
  • the data collected by the server 40 is sensing data
  • the present disclosure can be applied to any data that is required to be collected, such as user terminals, sensors, and in-vehicle systems.
  • This disclosure can be applied to the information and communications industry.

Landscapes

  • Engineering & Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • General Business, Economics & Management (AREA)
  • Multimedia (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Computer And Data Communications (AREA)

Abstract

L'objectif de cette divulgation est de fournir un système qui ne nécessite pas une grande quantité de ressources de réseau même lors de la collecte de données à partir d'un grand nombre de sources de données.　Cette divulgation concerne un système destiné à collecter, dans un serveur, des données provenant de multiples sources de données, les sources de données respectives transmettant une exigence de transfert de données à un contrôleur, le contrôleur déterminant une politique de transmission pour chaque source de données sur la base de l'exigence de transfert de données, une source de données transmettant de manière autonome une demande de connexion de voie de communication sur la base de la politique de transmission, le contrôleur générant une voie de communication sur la base de la demande de connexion de voie de communication provenant de la source de données, et la source de données transmettant des données à l'aide de la voie de communication générée.
PCT/JP2022/007865 2022-02-25 2022-02-25 Système, procédé et programme de collecte de données WO2023162130A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/JP2022/007865 WO2023162130A1 (fr) 2022-02-25 2022-02-25 Système, procédé et programme de collecte de données

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2022/007865 WO2023162130A1 (fr) 2022-02-25 2022-02-25 Système, procédé et programme de collecte de données

Publications (1)

Publication Number Publication Date
WO2023162130A1 true WO2023162130A1 (fr) 2023-08-31

Family

ID=87765027

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2022/007865 WO2023162130A1 (fr) 2022-02-25 2022-02-25 Système, procédé et programme de collecte de données

Country Status (1)

Country Link
WO (1) WO2023162130A1 (fr)

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2020017821A (ja) * 2018-07-24 2020-01-30 富士通フロンテック株式会社 外部情報受配信装置、データ送信方法、及びプログラム

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2020017821A (ja) * 2018-07-24 2020-01-30 富士通フロンテック株式会社 外部情報受配信装置、データ送信方法、及びプログラム

Similar Documents

Publication Publication Date Title
US7930446B2 (en) Methods and apparatuses for wireless network communication wherein a universal serial bus request block (URB) is generated that will vary parameters that controls wireless transmission commands between devices
CN106411767B (zh) 通过远程直接存储器访问的传输操作的方法、系统和介质
EP3547580B1 (fr) Procédé et appareil d'envoi de données, et procédé et appareil de réception de données
KR101021566B1 (ko) 우선 순위와 예약 대역폭 프로토콜을 활용하여 네트워크에서비스 품질(ＱｏＳ)을 제공하는 메커니즘
EP2193634B1 (fr) Réservation de bande passante pour des flux de données dans des réseaux d'interconnexion
US10484472B2 (en) Methods and systems for efficiently moving data between nodes in a cluster
KR20040012876A (ko) 호스트 컴퓨터 시스템과 이더넷 어댑터 사이의 데이터 이송
US11513985B2 (en) RoCE over wireless
KR101426217B1 (ko) 예약된 스트림의 최대 레이턴시를 감소시키는 방법
JP2023511889A (ja) サービスレベル構成方法および装置
US7580410B2 (en) Extensible protocol processing system
US11177969B2 (en) Interface device and data communication method
CN112398754B (zh) 数据传输方法、装置、介质、电子设备及网络接入设备
WO2023162130A1 (fr) Système, procédé et programme de collecte de données
WO2023162128A1 (fr) Système, procédé, et programme de collecte de données
WO2023162127A1 (fr) Système, procédé et programme de collecte de données
WO2014010189A1 (fr) Dispositif mandataire, système de communication et programme
WO2023012878A1 (fr) Système, procédé et programme de collecte de données
WO2018142866A1 (fr) Dispositif de transfert, procédé de transfert et programme
WO2023162134A1 (fr) Système de collecte de données, procédé et programme
JP6488557B2 (ja) 通信制御システム、通信システム、通信制御方法および通信制御プログラム
JPH10190734A (ja) パケット転送装置及びパケット転送方法
EP3955115B1 (fr) Relance flexible de niveau de liaison pour commutateurs de mémoire partagée
JP2002183014A (ja) コンテンツ配信システム及び方法
JP5058758B2 (ja) 通信管理装置、その制御方法及びプログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22928657

Country of ref document: EP

Kind code of ref document: A1