WO2019167348A1 - Communication device, communication method, recording medium, and management device - Google Patents

Communication device, communication method, recording medium, and management device Download PDF

Info

Publication number
WO2019167348A1
WO2019167348A1 PCT/JP2018/041044 JP2018041044W WO2019167348A1 WO 2019167348 A1 WO2019167348 A1 WO 2019167348A1 JP 2018041044 W JP2018041044 W JP 2018041044W WO 2019167348 A1 WO2019167348 A1 WO 2019167348A1
Authority
WO
WIPO (PCT)
Prior art keywords
area information
communication device
display area
video
unit
Prior art date
Application number
PCT/JP2018/041044
Other languages
French (fr)
Japanese (ja)
Inventor
中川　聰
Original Assignee
沖電気工業株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 沖電気工業株式会社 filed Critical 沖電気工業株式会社
Publication of WO2019167348A1 publication Critical patent/WO2019167348A1/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
    • H04N21/64—Addressing
    • H04N21/6405—Multicasting
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/65—Transmission of management data between client and server
    • H04N21/658—Transmission by the client directed to the server
    • H04N21/6587—Control parameters, e.g. trick play commands, viewpoint selection

Definitions

  • the present invention relates to a communication apparatus, a communication method, a recording medium, and a management apparatus, and can be applied to, for example, a distribution system that distributes multimedia data including video information and sound information from a distribution source to a plurality of users.
  • IPs When distributing data such as video and audio to multiple users, IPs are provided to users who request these data from video sources such as video cameras and multimedia sources from audio sources such as microphones.
  • a distribution method using unicast communication using a packet or the like is used.
  • the distribution server replicates and distributes the same data as many as the number of unicast channels according to the number of users, so the network bandwidth is reduced. There is a problem that it is wasted and cannot be distributed to a sufficient number of users (see FIG. 2).
  • a method called application level multicast is known. This is a method of realizing multicast to a large number of users only by the user's application by the user receiving the distribution becoming a relay node and redistributing to another user (see FIG. 4).
  • Patent Document 1 discloses a method for switching a distribution tree according to the situation by measuring a distribution path throughput of a distribution tree that performs distribution using application level multicast.
  • each user displays only a selected portion of the distributed video.
  • a video that gives a bird's-eye view of the entire office is distributed, and each user displays only the part of the department or person in which they are interested, or a video of the entire sports stadium.
  • Each user is displaying the play or player's part of interest, or 360 degree omnidirectional video is distributed, and each user displays only the angle of view in their respective line of sight. (See FIG. 5).
  • the conventional application-level multicast technology distributes the entire distribution video to all users at a high resolution, and is not displayed to the user at the end of the distribution tree, for example.
  • the network is distributed with high resolution including the portion, the network bandwidth is wasted, and depending on the network quality, the image quality must be lowered and distributed.
  • the present invention has been made to solve the above-described problems, and employs the following configuration.
  • a communication apparatus is a communication apparatus that multicasts video data through a tree-structured distribution path.
  • An area information receiving unit that receives area information including area information, (2) managing area information including display area information of each lower-level communication device received by the area information receiving unit, and display area information of the own device
  • the area information management unit and (3) area information including the display area information of each lower-level communication device managed by the area information management unit and the area information including the display area information of its own device are positioned above the distribution route.
  • a region information transmission unit that transmits to the upper communication device, (4) a reception unit that receives the video data from the upper communication device, and (5) display from the received video data based on the display region information of the own device.
  • Territory A display area extraction unit that extracts display data and outputs display data, and (6) a video that processes video data based on area information including display area information of each lower-level communication device managed by the area information management unit A processing unit, and (7) a retransmission unit that transmits post-processed video data processed by the video processing unit.
  • a communication device is a distribution source communication device that distributes video data by multicast distribution using a tree-structured distribution route, and (1) one or a plurality of lower-order communication devices that are positioned below the distribution route.
  • An area information receiving unit that receives area information including each display area information, and (2) an area information management unit that manages area information including display area information of each lower-level communication device received by the area information receiving unit; (3) A video processing unit that processes video data to be distributed based on region information including display region information of each lower-level communication device managed by the region information management unit, and (4) video processing by the video processing unit.
  • a transmission unit that transmits the processed video data.
  • a communication method is the communication method for multicast distribution of video data through a tree-structured distribution path, wherein (1) one or a plurality of subordinates in which the area information receiving unit is located below the distribution path (2) the area information management unit receives area information including the display area information of each lower-level communication apparatus received by the area information receiving unit; (3) The area information transmission unit includes area information including display area information of each lower-level communication device managed by the area information management unit, and display area information of the own device. (4) The reception unit receives video data from the higher-level communication device, and (5) the display area extraction unit displays the display area information of the own apparatus. Based on the received video The display area is extracted from the data and the display data is output.
  • the video processing unit converts the video data based on the area information including the display area information of each lower-level communication device managed by the area information management unit.
  • the video is processed, and (7) the retransmitting unit transmits the processed video data processed by the video processing unit.
  • a communication method is a communication method of a distribution source for multicast distribution of video data through a distribution path having a tree structure, wherein (1) the area information receiving unit is located at a lower position of the distribution path 1 or (2) The area information management unit manages area information including the display area information of each lower communication apparatus received by the area information receiving unit. And (3) the video processing unit processes the video data to be distributed based on the region information including the display region information of each lower-level communication device managed by the region information management unit, and (4) the transmission unit The post-processing video data processed by the video processing unit is transmitted.
  • a recording medium for multicast distribution of video data through a tree-structured distribution path wherein (1) one or a plurality of lower-level communication devices positioned at the lower level of the distribution path is used.
  • An area information receiving unit that receives area information including each display area information; (2) area information including display area information of each lower-level communication device received by the area information receiving unit; (3) region information including display region information of each lower-level communication device managed by the region information management unit, and region information including the display region information of the own device.
  • An area information transmission unit that transmits to a higher-level communication device located at a higher level; (4) a reception unit that receives video data from the higher-level communication device; and (5) a reception based on display area information of the own device.
  • a display area extracting unit that extracts a display area from the video data and outputs the display data; and (6) video data based on area information including display area information of each lower-level communication device managed by the area information management unit.
  • a computer-readable recording medium recording a communication program that functions as a re-transmission unit that transmits post-processed video data processed by the video processing unit.
  • a recording medium in a communication program of a distribution source for multicast distribution of video data through a tree-structured distribution path.
  • An area information receiving unit for receiving area information including display area information of each communication device; and (2) area information management for managing area information including display area information of each lower-level communication device received by the area information receiving unit.
  • a video processing unit that processes video data to be distributed based on region information including display region information of each lower-level communication device managed by the region information management unit, and (4) a video processing unit It is a computer-readable recording medium which recorded the communication program which functions as a transmission part which transmits the image data after process by which image processing was carried out.
  • a management device is a management device that manages a communication device that multicasts video data via a tree-structured distribution path, and (1) receives one or more video data from a communication device that is a distribution source.
  • a communication path setting unit that determines a distribution path between the communication apparatuses and instructs each communication apparatus that configures the distribution path to set a communication path of the distribution path; and (2) display area information between the communication apparatuses.
  • a relay unit that relays transmission / reception of the region information including the region information management unit, and (3) a region information management unit that manages region information including the display region information of each communication device. Only the area information including the display area information of the apparatus is used, and the area information management unit notifies the area information including the display area information of the lower communication apparatus below the child node notified to each communication apparatus.
  • the generated code amount by the video coding technique used in the transmission unit and the re-transmission unit is applied only to an area that is not a display area in any of the child node and its descendant nodes. Reduces the amount of transmission data and retransmission data sent to child nodes without adversely affecting the video displayed on each receiving node because video processing is effective. be able to.
  • the network bandwidth can be afforded, it is possible to perform video encoding with higher image quality, and it is possible to improve the image quality of the video data in the display area output from each receiving node.
  • FIG. 6 is an overall configuration diagram illustrating an example of the overall configuration of the video distribution system according to the embodiment.
  • the video distribution system 1 includes a video distribution management server 10, a distribution source node 20, and reception nodes 30 (30-1 to 30-5).
  • the distribution system 1 forms a tree-structured network, and distributes media data using application level multicast.
  • the data flow from the distribution source node 20 to the reception node 30 is assumed to be downstream, and conversely, the data flow from the reception node 30 to the distribution source node 20 is assumed to be upstream.
  • a node (distribution source node 20 or other receiving node 30) that is located upstream from a certain receiving node 30 and directly communicates is called a “parent node” and is located downstream from a certain receiving node 30.
  • Nodes that directly communicate with each other (other receiving nodes 30) are called “child nodes”.
  • parent nodes located further upstream of the parent node of the own device 30 are collectively referred to as “ancestor nodes”, and child nodes located further downstream of the child nodes of the own device 30 are referred to as “descendant”. Also called “node”.
  • Application level multicast is one of the methods for distributing data to multiple nodes.
  • the distribution technique of application level multicast is a method of distributing to a plurality of receiving nodes as distribution destinations while reducing the load on the distribution source node (or server).
  • a wide variety of data including video and audio can be supported as data types, and real-time live distribution and storage-type distribution can be supported.
  • the IP multicast technique is a technique for performing distribution data routing control and replication processing in the IP layer.
  • a router on a distribution path performs data duplication processing, so even if the number of distribution destination receiving nodes increases, the load on the distribution source node does not increase.
  • a protocol a protocol called IGMP in IPv4 and a protocol called MDLv2 in IPv6 are used.
  • the disadvantage is that all routers on the path from the distribution source node to the distribution destination receiving node need to be able to handle these special protocols.
  • the infrastructure that enables IP multicasting is not yet widespread, and is not particularly widespread in corporate networks.
  • Application level multicast technology is a method of dealing with an increase in the number of receiving nodes of a distribution destination by suppressing an increase in the load of a distribution source terminal by duplicating data by a device on a route, like IP multicast.
  • the difference between the application level multicast technology and the IP multicast technology is that the receiving node of the distribution destination performs data replication.
  • the media data distributed by the distribution source node 20 exemplifies a case where video information is included in this embodiment, but may include sound information (information regarding sound including sound and sound).
  • the video distribution management server 10 manages the configuration of the video distribution tree between the distribution source node 20 and the reception node 30.
  • the video distribution management server 10 receives the distribution source node 20 and each receiving node. 30 is instructed to reconstruct the distribution tree. Further, transmission / reception of area information between the receiving node 30 and the distribution source node 20 to be described later may be performed directly between nodes, but the video distribution management server 10 is configured to mediate transmission / reception of area information. Also good.
  • the video distribution management server 10 determines a distribution tree between the distribution source node 20 and the reception node 30 configuring the distribution tree, and configures a distribution tree for each node (distribution source node 20 and reception node 30).
  • Area for managing area information including display area information of each receiving node 30 and a communication path setting section for instructing setting of a communication path for a distribution path between the nodes, a relay section for relaying transmission / reception of area information between nodes And an information management unit.
  • the video distribution management server 10 acquires area information from each receiving node 30, but the area information transmitted from each receiving node 30 at that time includes only the display area information of the receiving node 30.
  • the video distribution management server 10 notifies the area information below the child node notified to the receiving node 30 and the distribution source node 20. In other words, the video distribution management server 10 notifies the reception node 30 and the distribution source node 20 of the area information, but the areas of the descendant nodes such as the child nodes and grandchild nodes of the reception node 30 and the distribution source node 20. The video distribution management server 10 may notify the information to the receiving node 30 or the distribution source node 20.
  • the distribution source node 20 distributes video information such as video content data to be distributed and video data from a video source such as a video camera to each receiving node 30.
  • the distribution source node 20 In response to an instruction from the video distribution management server 10, the distribution source node 20 notifies each of the reception nodes 30 that are direct distribution destinations of the plurality of reception nodes 30 of the video information desired to be distributed from the reception node 30.
  • the video information that has been processed according to the region information to be transmitted is transmitted.
  • the receiving nodes 30 (30-1 to 30-5) receive video information from nodes (for example, the distribution source node 20 and other receiving nodes 30) located on the upstream side of the own device via a communication line. Received, extracts the display area in the frame of the received video according to the display area information of its own device, and displays the video of the extracted display area. Also, the receiving node 30 responds to the received video information to each of the other receiving nodes 30 having the own device as a parent node (that is, each child node of the own device) according to an instruction from the video distribution management server 10. Then, the processing according to the area information notified from the other receiving node 30 is performed, and the video information after the processing is redistributed. As the receiving node 30, for example, a smartphone, a tablet terminal, a personal computer, a wearable terminal, or the like can be applied.
  • the video distribution management server 10 manages the distribution tree for each distribution source (for each video content to be distributed).
  • the distribution source node 20 may be arranged at each of a plurality of bases, and communication may be realized by distributing information bidirectionally between the plurality of bases.
  • FIG. 1 is an internal configuration diagram illustrating an internal configuration of the receiving node 30 according to the embodiment.
  • the reception node 30 includes a reception unit 301, a display region extraction unit 302, a video processing unit 303, a retransmission unit 304, a region information reception unit 305, a region information management unit 306, and a region information transmission unit 307. Is provided.
  • the reception node 30 includes, for example, a CPU, a ROM, a RAM, an EEPROM, an input / output interface unit, a communication device, and the like, and a processing program (communication program) stored in the ROM or downloaded from a server. By executing the above, various functions as the receiving node 30 described later are realized.
  • a computer-readable recording medium that records the processing program (communication program) may also be provided.
  • the receiving unit 301 receives distribution data including video from the parent node (distribution source node 20 or other receiving node 30) of the own device 30.
  • a distribution source node 20 or a node close to the distribution source node 20 is set as a parent node.
  • the display area extraction unit 302 has acquired display area information instructed by the user of the device 30 and extracts display data from the video received by the receiving unit 301 based on the display area information instructed by the user.
  • the display data is output.
  • the video processing unit 303 When the image processing unit 303 functions as a relay node that performs redistribution to a child node (another receiving node 30), the video processing unit 303 displays a child node and its descendant nodes held in the area information management unit 306.
  • the area information including the area information is read out, the video is processed based on the area information including the display area information of the child node and the descendant node, and the distribution video is transmitted to the child node. To give.
  • the re-transmission unit 304 distributes the distribution video to the child node processed by the video processing unit 303 to the child node.
  • the area information receiving unit 305 includes area information including display area information from a child node of its own device 30 and its descendant nodes (that is, other receiving nodes 30 constituting a tree below the child node in the tree structure video distribution system 1). Is received, and region information including the display region information is provided to the region information management unit 306.
  • the area information management unit 306 manages the display area information of the own device 30 instructed by the user and the area information received by the area information receiving unit 305.
  • the area information transmission unit 307 obtains area information including the display area information of the own device 30 and area information including the display area information of the child node or its descendant node from the area information held in the area information management unit 306. Send to parent node.
  • FIG. 7 is an internal configuration diagram illustrating an internal configuration of the distribution source node 20 according to the embodiment.
  • the distribution source node 20 includes a video processing unit 203, a transmission unit 204, a region information reception unit 205, and a region information management unit 206.
  • the distribution source node 20 includes, for example, a CPU, a ROM, a RAM, an EEPROM, an input / output interface unit, a communication device, and the like, and a processing program (communication program) stored in the ROM or downloaded from a server. ), Various functions as the distribution source node 20 described later are realized.
  • a computer-readable recording medium that records the processing program (communication program) may also be provided.
  • the video processing unit 203 acquires distribution data including the video to be distributed, processes and distributes the video based on the area information including the display area information of the child node and its descendant nodes held in the area information management unit 206.
  • a video for distribution is generated, and a video for distribution to a child node (receiving node 30) that receives distribution directly from the own device 20 is given to the transmission unit 204.
  • the transmission unit 204 transmits transmission data including the distribution video processed by the video processing unit 203.
  • the area information receiving unit 205 receives area information including display area information of a child node and its descendant nodes (receiving nodes 30 constituting a tree below the child node on the distribution tree) from the child node, and the received area information Is given to the area information management unit 206.
  • the area information management unit 206 manages the area information received from the child node and its descendant nodes received by the area information receiving unit 205.
  • the distribution state of the distribution tree serving as the distribution route of the video information is managed by the video distribution management server 10, and node information related to each node and information regarding the configuration of the distribution tree with the distribution source node 20 and the reception node 30. By exchanging, the distribution tree is rebuilt according to the dynamic addition and removal of nodes. Arbitrary methods can be used for the contents of information related to the configuration of a specific distribution tree, signaling for setting a communication path as a specific distribution path, and the like.
  • the video distribution management server 10 may be configured to also serve as a signaling server that relays signaling for setting a communication path between nodes.
  • the video distribution management server 10 also serves as a signaling server for setting a communication path for transmission / reception, or relays transmission / reception of area information between nodes without directly transmitting / receiving area information between nodes. It is good also as a structure which serves concurrently.
  • the distribution source node 20 inputs distribution data that is a video to be distributed, and notifies the video distribution management server 10 of node information as a distribution source node.
  • the receiving node 30 notifies the video distribution management server 10 of node information as a receiving node including, for example, information for specifying video data to be distributed.
  • the video distribution management server 10 determines the configuration of a distribution tree composed of the distribution source node 20 and (one or more) reception nodes 30, and distributes to each node (the distribution source node 20 and the reception node 30). Notify information related to the connection partner of the path.
  • Each node of the distribution source node 20 and the reception node 30 sets a communication path with a parent node for receiving distribution data and a communication path to a child node for transmitting distribution data, and transmits / receives distribution data. Do.
  • Each node of the distribution source node 20 and the reception node 30 secures a communication path with the connection partner according to the distribution path, and the distribution source node 20 receives the transmission of the video data after processing the video information that the distribution source node 20 has.
  • the node 30 receives the distribution data from the parent node, and retransmits the video data after processing the received video data when the node 30 has a child node.
  • the video distribution management server 10 follows the node information notified from the new reception node 30 to the parent node (distribution source node 20 or already distributed).
  • the receiving node 30) is allocated, and the assigned parent node starts distribution to the new receiving node 30, thereby constructing a distribution tree.
  • the distribution tree When one node device 30 among the receiving nodes 30 participating in the distribution tree leaves the distribution tree, distribution to the receiving node 30 whose parent node (or ancestor node) is the stopped node is stopped. Therefore, a new parent node is assigned to the receiving node 30 whose distribution is stopped.
  • the reassignment of the parent node may be performed only for the direct child node of the leaving node, or the reassignment may be performed not only for the direct child node but also for the descendant node of the child node. By reallocating parent nodes to these receiving nodes 30, the distribution tree is reconstructed.
  • display area information which is information indicating an area to be displayed on each receiving node 30 is input. This may be, for example, area information set for each receiving node 30 in advance, or may be area information based on an instruction from the user via a user interface such as a mouse or a touch operation.
  • the receiving node 30 receives the video data from the parent node (distribution source node 20 or other receiving node 30) that is the direct distribution source of the video information by the reception unit 301. A part of the video data is extracted from the received video data in accordance with the input display area information by the display area extraction unit 302, output as display data, and displayed on a display or the like.
  • the video processing unit 303 processes the video for distribution to the child node, and the processed video information is displayed. Transmission is performed from the retransmission unit 304 to each child node. When there is no child node, the processing process and the retransmission process are not performed.
  • the child node is notified of region information from the child node including the display region information of the other receiving nodes 30 that are child nodes and the other receiving nodes 30 that are descendants in the distribution tree, and the region information receiving unit 305. Receive at.
  • the received region information from the child node and the display region information of the receiving node 30 itself are input to the region information management unit 306, and the region information management unit 306 displays the video processing target region information according to these region information as video. This is supplied to the processing unit 303.
  • the area information management unit 306 supplies the area information from the child node and the area information including its own display area information to the area information transmission unit 307 in order to notify the parent node. Notify the node.
  • Distribution data that is a video to be distributed is input to the distribution source node 20, but this is not transmitted as it is, but is transmitted after performing a video processing process according to area information notified from the child node. Do.
  • the video data that is the input distribution data is processed by the video processing unit 203 for processing the video for distribution to the child node (receiving node 30) that receives the direct distribution from the distribution source node 20.
  • the processed video data is transmitted from the transmission unit 204 to each child node.
  • the region information receiving unit 205 receives the region information from the child node including the display region information of the child node and its descendant node (the receiving node 30 constituting the tree below the child node on the distribution tree). .
  • the region information received by the region information receiving unit 205 is supplied to the region information management unit 206, and the video processing unit 203 follows the region information including the display region information of the child node and its descendant nodes held by the region information management unit 206. Processing to process the video.
  • FIG. 8 is an explanatory diagram illustrating the transmission / reception process of the area information from the child node to the parent node according to the embodiment.
  • the notation “user (30-1)” indicates a frame of a video distributed from the distribution source node 20 to the reception node 30-1, and D1 indicates a reception
  • the display area selected in the video frame by the user of the node 30-1 is shown.
  • what is surrounded by a dotted line is an image of region information exchanged between nodes.
  • each of the receiving nodes 30-1 to 30-5 that receives video distribution from the distribution source node 20 display areas D1 to D5, which are areas to be displayed, are designated by the respective users.
  • Each receiving node 30 adds the display area information instructed by the user in its own device 30 to the area information notified from the child node, and notifies it as area information to the parent node.
  • the display of the own device 30-3 is displayed in the display area D2.
  • the information including the area D3 is transmitted as area information to the user (30-5) as the parent node.
  • the display area information instructed by the user at each receiving node 30 is described as a rectangular area, but the display area information is not limited to the rectangular area.
  • the display area information can be information for specifying the display area in the video frame. For example, in the case of a rectangular area, the upper left coordinate value of the rectangular area, the value indicating the width of the rectangular area, the value indicating the height, etc.
  • the area information may be expressed by this information.
  • the area information management unit 306 of each receiving node 30 may manage area information collected from child nodes and descendant nodes in a list form. Specifically, region information including display region information collected from child nodes and descendant nodes is associated with identification information of child nodes and descendant nodes (for example, identifier information assigned to each node from the video distribution management server). Manage it. Thereby, area information including display area information can be managed for each child node and descendant node. Alternatively, for each direct child node, it can be managed as a list of all area information below the child node (individual nodes are not specified).
  • the area information management unit 306 may integrate the areas in the inclusion relationship based on the display area information of the own device 30 and the area information including the display area information of the child node and the descendant node. For example, in FIG. 8, in the user (30-3), the display area D2 of the user (30-2), which is a child node, and the display area D3 of the receiving node 30-3 partially overlap. In such a case, the area information management unit 306 of the receiving node 30-3 deletes the overlapping area for either the display area D2 of the user (30-2) or the display area D3 of the receiving node 30-3, You may make it integrate an overlapping area
  • the area information management unit 306 when a plurality of display areas are overlapped based on the display area information of the own device 30 and the area information including the display area information of the child node and the descendant node, You may make it integrate in a big area
  • region information is good also as a structure which provides and communicates by providing a direct communication path between nodes (between the receiving node 30 and the delivery origin node 20), it passes via servers, such as the video delivery management server 10.
  • a configuration may be adopted in which information is exchanged by relay. Further, for example, when relaying by the video distribution management server 10, only the display area information of its own device is transmitted from the area information transmission unit 307, and the video distribution management server 10 follows the configuration state of the distribution tree. The information may be transmitted to the region information receiving unit 305 and the region information receiving unit 205 after the region information is aggregated. That is, the area information transmission unit 307 transmits only the display area information of its own device to the video distribution management server 10.
  • the video distribution management server 10 identifies the receiving node 30 and the distribution source node 20 that are the parent node and the ancestor node that should receive the area information according to the configuration state of the distribution tree, and transmits the area information to these nodes.
  • the area information of all the nodes of the subtree of the distribution tree including the receiving node 30 (the other receiving nodes 30 constituting the tree below the child node on the distribution tree) is aggregated and transmitted.
  • the area information receiving unit 305 and the area information receiving unit 205 receive the area information of the subtree of the distribution tree below each child node from the video distribution management server 10.
  • the image distribution management server 10 may also be configured to integrate region information due to region overlap and the like.
  • the video data to be distributed is imaged for ease of explanation.
  • the notation “user (30-1)” indicates a frame of video distributed from the distribution source node 20 to the reception node 30-1, and D1 is selected by the user of the reception node 30-1 within the video frame.
  • the displayed area is shown.
  • what is surrounded by a dotted line indicates distribution data, a hatched portion indicates a portion subjected to video processing, and a white portion indicates a portion where video processing is not performed.
  • the child node is notified of area information obtained by collecting display area information of the child node and its descendant nodes (other receiving nodes 30 constituting the tree below the child node on the distribution tree). Yes.
  • an area that is not a display area in any of the reception nodes 30 below the distribution path (an area expressed by hatching in FIG. 9). ).
  • An area that is a display area in any of the receiving nodes 30 below the distribution path (an area expressed by a white portion in FIG. 9) is not a target of video processing.
  • the child node and its child node are displayed in accordance with the region information including the display region information of the child node and its descendant nodes held by the region information management units 206 and 306.
  • region information including the display region information of the child node and its descendant nodes held by the region information management units 206 and 306.
  • video processing that is effective in reducing the amount of generated code by the video encoding technique used in the transmission unit 204 and the retransmission unit 304 is performed.
  • the image processing units 203 and 303 may simply paint the region to be processed in a single color (gray).
  • the video processing units 203 and 303 may perform a blur filter process such as a Gaussian filter.
  • discrete cosine transform and entropy coding are used in the video coding technique used in the transmission unit 204 and the retransmission unit 304, and a code amount is generated in a complex texture region or a region with a sharp edge.
  • the generation of the code amount can be reduced in the monochromatic region or the region subjected to the low-pass filter processing.
  • the receiving node 30 performs processing only on an area that is the first processing target area (an area that is its own display area but is not included in the area information from the child node). It is good also as a structure which reduces this.
  • the user (30-3) receives the video data that has been processed from the parent node user (30-5). At this time, the user (30-3) is the child node.
  • the area information including the display area information of a certain user (30-2) only the area other than the display area of the user (30-2) is processed.
  • each receiving node 30 only needs to perform video processing on its own display area other than the display area of the child nodes and descendant nodes (the other areas are already processed by the parent node). The amount of processing can be reduced.
  • An area that is not to be processed may be enlarged over a certain range in advance, or a process that performs a weak filter process may be performed on the area to be processed around the area that is not to be processed.
  • each of the child nodes has been described as performing video processing processing according to the area information in the distribution tree below the child nodes.
  • it may be configured to perform a processing process that generates only one type of processed image obtained by integrating the region information of all the child nodes (in this case, the child node may be used).
  • the child node may be used.
  • an area that does not become a display area may be transmitted without being processed).
  • it can be used for various data distribution such as simultaneously distributing information such as documents and presentations in addition to video and audio data.
  • the present invention can be implemented as a system or device having the above-described configuration, or as a program for realizing the processing of each device constituting the system.
  • a downloadable program for example, a JavaScript (registered trademark) program
  • a server Web server or video distribution management server
  • It can also be configured as a server device (server program) that configures the system (downloaded to a Web browser).
  • Video distribution system 10 ... Video distribution management server, 20 ... Distribution source node, 203 ... Video processing unit, 204 ... Transmission unit, 205 ... Area information reception unit, 206 ... Area information management unit, 30 (30-1 to 30-5) ... receiving node, 301 ... receiving unit, 302 ... display area extracting unit, 303 ... video processing unit, 304 ... retransmitting unit, 305 ... area information receiving unit, 306 ... area information management Part, 307... Area information transmission part.

Abstract

[Purpose] To reduce the data amount of video data of a display area outputted by each of a plurality of nodes and enable high picture quality delivery. [Solution] According to the invention, a communication device for multicast delivering video data via delivery paths having a tree configuration receives area information including display area information of each of one or more lower communication devices; manages the area information including the display area information of each lower communication device and also manages the display area information of the local device; transmits area information that includes the area information including the display area information of each lower communication device and that also includes the display area information of the local device to an upper communication device; receives video data from the upper communication device; extracts a display area from the video data on the basis of the display area information of the local device; and transmits processed video data obtained by video processing the video data on the basis of the area information including the display area information of each lower communication device.

Description

通信装置、通信方法、記録媒体及び管理装置COMMUNICATION DEVICE, COMMUNICATION METHOD, RECORDING MEDIUM, AND MANAGEMENT DEVICE
　本発明は、通信装置、通信方法、記録媒体及び管理装置に関し、例えば、映像情報や音情報等を含むマルチメディアデータを配信元から複数ユーザーへ配信を行う配信システムに適用し得る。 The present invention relates to a communication apparatus, a communication method, a recording medium, and a management apparatus, and can be applied to, for example, a distribution system that distributes multimedia data including video information and sound information from a distribution source to a plurality of users.
　近年、情報通信技術の発達により、遠隔の離れた場所にいる者同士があたかも同じ場所にいるかのようにコミュニケーションしたり、共同作業を行ったりする技術が注目されている。複数の映像や音声等の情報を、複数の場所や複数のユーザーに配信することによりこれら遠隔の複数者間のコミュニケーションを実現している。 In recent years, with the development of information and communication technology, technology that allows people at distant places to communicate as if they were in the same place, or to perform collaborative work, has attracted attention. By distributing information such as a plurality of images and sounds to a plurality of locations and a plurality of users, communication between these remote multiple persons is realized.
　映像や音声等のデータを複数のユーザーに配信する場合、ビデオカメラ等の映像ソースや、マイク等の音声ソースからのマルチメディアデータを持つ配信元から、これらのデータを要求するユーザーに対してＩＰパケット等を用いたユニキャスト通信で配信する方法が用いられる。 When distributing data such as video and audio to multiple users, IPs are provided to users who request these data from video sources such as video cameras and multimedia sources from audio sources such as microphones. A distribution method using unicast communication using a packet or the like is used.
　このようなユニキャスト通信では、同じデータを要求するユーザー数が増加すると、配信サーバーでは、ユーザー数に応じたユニキャスト通信路の数だけ同じデータを複製して配信するため、ネットワークの帯域幅を浪費して十分な数のユーザーに配信できないという問題がある（図２参照）。また、ユニキャスト通信ではなくマルチキャスト通信を用いてデータの複製を配信経路の途中のルーターで行うことによりネットワーク帯域の浪費を防止する方法もあるが、経路上のすべてのルーターがマルチキャスト通信に対応する必要があり、一般には普及していない（図３参照）。 In such unicast communication, if the number of users requesting the same data increases, the distribution server replicates and distributes the same data as many as the number of unicast channels according to the number of users, so the network bandwidth is reduced. There is a problem that it is wasted and cannot be distributed to a sufficient number of users (see FIG. 2). In addition, there is a method to prevent waste of network bandwidth by using multicast communication instead of unicast communication to copy data with a router in the middle of the distribution route, but all routers on the route support multicast communication. It is necessary and is not widely used (see FIG. 3).
　このような問題への対策として、アプリケーションレベルマルチキャストと呼ばれる方法が知られている。これは、配信を受けているユーザーが中継ノードとなり、さらに別のユーザーに対して再配信することにより、ユーザーのアプリケーションのみで多数のユーザーへのマルチキャストを実現する方法である（図４参照）。 As a countermeasure against such a problem, a method called application level multicast is known. This is a method of realizing multicast to a large number of users only by the user's application by the user receiving the distribution becoming a relay node and redistributing to another user (see FIG. 4).
　このようなアプリケーションレベルマルチキャストでは、配信経路をいかに構築するかがシステム全体の配信性能に大きく影響する。 In such application-level multicast, how the delivery route is constructed greatly affects the delivery performance of the entire system.
　特許文献１では、アプリケーションレベルマルチキャストを用いて配信を行う配信ツリーを、配信パスのスループット等を計測することによって、状況に応じて配信ツリーを切り替えてして方法などが開示されている。 Patent Document 1 discloses a method for switching a distribution tree according to the situation by measuring a distribution path throughput of a distribution tree that performs distribution using application level multicast.
特開２００７－２３５６８１号公報JP 2007-235681 A
　このようなアプリケーションレベルマルチキャストを用いて、多数のユーザーに高解像度の映像を配信する場合で、各ユーザーは配信される映像のうち、それぞれが選択した一部分のみを表示しているような用途を想定する。例えば、オフィス全体を俯瞰するような映像を配信し、各ユーザーはそれぞれが興味を持っている部署や人が映った部分のみを表示している場合、あるいは、スポーツの競技場全体の映像が配信されており、各ユーザーは興味のあるプレーや選手の部分を表示している場合、あるいは、３６０度全方向の映像が配信されており、各ユーザーはそれぞれの視線方向の画角のみを表示している場合などである（図５参照）。 In such a case where high-resolution video is distributed to a large number of users using application-level multicast, it is assumed that each user displays only a selected portion of the distributed video. To do. For example, a video that gives a bird's-eye view of the entire office is distributed, and each user displays only the part of the department or person in which they are interested, or a video of the entire sports stadium. Each user is displaying the play or player's part of interest, or 360 degree omnidirectional video is distributed, and each user displays only the angle of view in their respective line of sight. (See FIG. 5).
　このような用途を想定すると、従来のアプリケーションレベルマルチキャスト技術では、全ユーザーに対して、配信映像全体を高解像度に配信することとなり、例えは配信ツリーの末端のユーザーにも、そのユーザーでは表示されない部分を含めて高解像度に配信されることとなり、ネットワーク帯域が浪費されたり、また、ネットワーク品質によっては画質を低下させて配信せざるを得ない状況となるという課題があった。 Assuming such applications, the conventional application-level multicast technology distributes the entire distribution video to all users at a high resolution, and is not displayed to the user at the end of the distribution tree, for example. There is a problem that the network is distributed with high resolution including the portion, the network bandwidth is wasted, and depending on the network quality, the image quality must be lowered and distributed.
　本発明は、前記課題を解決するためになされたものであり、次のような構成を採用する。 The present invention has been made to solve the above-described problems, and employs the following configuration.
　第１の本発明に係る通信装置は、ツリー構成の配信経路で映像データをマルチキャスト配信する通信装置において、（１）配信経路の下位に位置している１又は複数の下位通信装置のそれぞれの表示領域情報を含む領域情報を受信する領域情報受信部と、（２）領域情報受信部により受信された各下位通信装置の表示領域情報を含む領域情報と、自装置の表示領域情報とを管理する領域情報管理部と、（３）領域情報管理部で管理される各下位通信装置の表示領域情報を含む領域情報と、自装置の表示領域情報とを含む領域情報を、配信経路の上位に位置している上位通信装置に送信する領域情報送信部と、（４）上位通信装置から映像データを受信する受信部と、（５）自装置の表示領域情報に基づいて、受信した映像データから表示領域を抽出して表示データを出力する表示領域抽出部と、（６）領域情報管理部に管理されている各下位通信装置の表示領域情報を含む領域情報に基づいて、映像データを映像加工する映像加工部と、（７）映像加工部により映像加工された加工後映像データを送信する再送信部とを備えることを特徴とする。 A communication apparatus according to a first aspect of the present invention is a communication apparatus that multicasts video data through a tree-structured distribution path. (1) Display of each of one or a plurality of lower-level communication apparatuses positioned at a lower level of a distribution path An area information receiving unit that receives area information including area information, (2) managing area information including display area information of each lower-level communication device received by the area information receiving unit, and display area information of the own device The area information management unit, and (3) area information including the display area information of each lower-level communication device managed by the area information management unit and the area information including the display area information of its own device are positioned above the distribution route. A region information transmission unit that transmits to the upper communication device, (4) a reception unit that receives the video data from the upper communication device, and (5) display from the received video data based on the display region information of the own device. Territory A display area extraction unit that extracts display data and outputs display data, and (6) a video that processes video data based on area information including display area information of each lower-level communication device managed by the area information management unit A processing unit, and (7) a retransmission unit that transmits post-processed video data processed by the video processing unit.
　第２の本発明に係る通信装置は、ツリー構成の配信経路で映像データをマルチキャスト配信する配信元の通信装置において、（１）配信経路の下位に位置している１又は複数の下位通信装置のそれぞれの表示領域情報を含む領域情報を受信する領域情報受信部と、（２）領域情報受信部により受信された各下位通信装置の表示領域情報を含む領域情報を管理する領域情報管理部と、（３）領域情報管理部に管理されている各下位通信装置の表示領域情報を含む領域情報に基づいて、配信する映像データを映像加工する映像加工部と、（４）映像加工部により映像加工された加工後映像データを送信する送信部とを備えることを特徴とする。 A communication device according to a second aspect of the present invention is a distribution source communication device that distributes video data by multicast distribution using a tree-structured distribution route, and (1) one or a plurality of lower-order communication devices that are positioned below the distribution route. An area information receiving unit that receives area information including each display area information, and (2) an area information management unit that manages area information including display area information of each lower-level communication device received by the area information receiving unit; (3) A video processing unit that processes video data to be distributed based on region information including display region information of each lower-level communication device managed by the region information management unit, and (4) video processing by the video processing unit. And a transmission unit that transmits the processed video data.
　第３の本発明に係る通信方法は、ツリー構成の配信経路で映像データをマルチキャスト配信する通信方法において、（１）領域情報受信部が、配信経路の下位に位置している１又は複数の下位通信装置のそれぞれの表示領域情報を含む領域情報を受信し、（２）領域情報管理部が、領域情報受信部により受信された各下位通信装置の表示領域情報を含む領域情報と、自装置の表示領域情報とを管理し、（３）領域情報送信部が、領域情報管理部で管理される各下位通信装置の表示領域情報を含む領域情報と、自装置の表示領域情報とを含む領域情報を、配信経路の上位に位置している上位通信装置に送信し、（４）受信部が、上位通信装置から映像データを受信し、（５）表示領域抽出部が、自装置の表示領域情報に基づいて、受信した映像データから表示領域を抽出して表示データを出力し、（６）映像加工部が、領域情報管理部に管理されている各下位通信装置の表示領域情報を含む領域情報に基づいて、映像データを映像加工し、（７）再送信部が、映像加工部により映像加工された加工後映像データを送信することを特徴とする。 A communication method according to a third aspect of the present invention is the communication method for multicast distribution of video data through a tree-structured distribution path, wherein (1) one or a plurality of subordinates in which the area information receiving unit is located below the distribution path (2) the area information management unit receives area information including the display area information of each lower-level communication apparatus received by the area information receiving unit; (3) The area information transmission unit includes area information including display area information of each lower-level communication device managed by the area information management unit, and display area information of the own device. (4) The reception unit receives video data from the higher-level communication device, and (5) the display area extraction unit displays the display area information of the own apparatus. Based on the received video The display area is extracted from the data and the display data is output. (6) The video processing unit converts the video data based on the area information including the display area information of each lower-level communication device managed by the area information management unit. The video is processed, and (7) the retransmitting unit transmits the processed video data processed by the video processing unit.
　第４の本発明に係る通信方法は、ツリー構成の配信経路で映像データをマルチキャスト配信する配信元の通信方法において、（１）領域情報受信部が、配信経路の下位に位置している１又は複数の下位通信装置のそれぞれの表示領域情報を含む領域情報を受信し、（２）領域情報管理部が、領域情報受信部により受信された各下位通信装置の表示領域情報を含む領域情報を管理し、（３）映像加工部が、領域情報管理部に管理されている各下位通信装置の表示領域情報を含む領域情報に基づいて、配信する映像データを映像加工し、（４）送信部が、映像加工部により映像加工された加工後映像データを送信することを特徴とする。 A communication method according to a fourth aspect of the present invention is a communication method of a distribution source for multicast distribution of video data through a distribution path having a tree structure, wherein (1) the area information receiving unit is located at a lower position of the distribution path 1 or (2) The area information management unit manages area information including the display area information of each lower communication apparatus received by the area information receiving unit. And (3) the video processing unit processes the video data to be distributed based on the region information including the display region information of each lower-level communication device managed by the region information management unit, and (4) the transmission unit The post-processing video data processed by the video processing unit is transmitted.
　第５の本発明に係る記録媒体は、ツリー構成の配信経路で映像データをマルチキャスト配信する通信プログラムにおいて、コンピュータを、（１）配信経路の下位に位置している１又は複数の下位通信装置のそれぞれの表示領域情報を含む領域情報を受信する領域情報受信部と、（２）領域情報受信部により受信された各下位通信装置の表示領域情報を含む領域情報と、自装置の表示領域情報とを管理する領域情報管理部と、（３）領域情報管理部で管理される各下位通信装置の表示領域情報を含む領域情報と、自装置の表示領域情報とを含む領域情報を、配信経路の上位に位置している上位通信装置に送信する領域情報送信部と、（４）上位通信装置から映像データを受信する受信部と、（５）自装置の表示領域情報に基づいて、受信した映像データから表示領域を抽出して表示データを出力する表示領域抽出部と、（６）領域情報管理部に管理されている各下位通信装置の表示領域情報を含む領域情報に基づいて、映像データを映像加工する映像加工部と、（７）映像加工部により映像加工された加工後映像データを送信する再送信部として機能させる通信プログラムを記録したコンピュータに読み取り可能な記録媒体である。 According to a fifth aspect of the present invention, there is provided a recording medium for multicast distribution of video data through a tree-structured distribution path, wherein (1) one or a plurality of lower-level communication devices positioned at the lower level of the distribution path is used. An area information receiving unit that receives area information including each display area information; (2) area information including display area information of each lower-level communication device received by the area information receiving unit; (3) region information including display region information of each lower-level communication device managed by the region information management unit, and region information including the display region information of the own device. An area information transmission unit that transmits to a higher-level communication device located at a higher level; (4) a reception unit that receives video data from the higher-level communication device; and (5) a reception based on display area information of the own device. A display area extracting unit that extracts a display area from the video data and outputs the display data; and (6) video data based on area information including display area information of each lower-level communication device managed by the area information management unit. And (7) a computer-readable recording medium recording a communication program that functions as a re-transmission unit that transmits post-processed video data processed by the video processing unit.
　第６の本発明に係る記録媒体は、ツリー構成の配信経路で映像データをマルチキャスト配信する配信元の通信プログラムにおいて、コンピュータを、（１）配信経路の下位に位置している１又は複数の下位通信装置のそれぞれの表示領域情報を含む領域情報を受信する領域情報受信部と、（２）領域情報受信部により受信された各下位通信装置の表示領域情報を含む領域情報を管理する領域情報管理部と、（３）領域情報管理部に管理されている各下位通信装置の表示領域情報を含む領域情報に基づいて、配信する映像データを映像加工する映像加工部と、（４）映像加工部により映像加工された加工後映像データを送信する送信部として機能させる通信プログラムを記録したコンピュータに読み取り可能な記録媒体である。 According to a sixth aspect of the present invention, there is provided a recording medium according to a sixth aspect of the present invention, in a communication program of a distribution source for multicast distribution of video data through a tree-structured distribution path. An area information receiving unit for receiving area information including display area information of each communication device; and (2) area information management for managing area information including display area information of each lower-level communication device received by the area information receiving unit. A video processing unit that processes video data to be distributed based on region information including display region information of each lower-level communication device managed by the region information management unit, and (4) a video processing unit It is a computer-readable recording medium which recorded the communication program which functions as a transmission part which transmits the image data after process by which image processing was carried out.
　第７の本発明に係る管理装置は、ツリー構成の配信経路で映像データをマルチキャスト配信する通信装置を管理する管理装置において、（１）配信元の通信装置と映像データを受信する１又は複数の通信装置との間の配信経路を決定し、当該配信経路を構成する各通信装置に配信パスの通信路の設定を指示する通信路設定部と、（２）各通信装置間の表示領域情報を含む領域情報の送受信を中継する中継部と、（３）各通信装置の表示領域情報を含む領域情報を管理する領域情報管理部とを備え、各通信装置から送信される領域情報は、当該通信装置の表示領域情報を含む領域情報のみとし、領域情報管理部が、各通信装置に通知する子ノード以下の下位の通信装置の表示領域情報を含む領域情報を通知することを特徴とする。 A management device according to a seventh aspect of the present invention is a management device that manages a communication device that multicasts video data via a tree-structured distribution path, and (1) receives one or more video data from a communication device that is a distribution source. A communication path setting unit that determines a distribution path between the communication apparatuses and instructs each communication apparatus that configures the distribution path to set a communication path of the distribution path; and (2) display area information between the communication apparatuses. A relay unit that relays transmission / reception of the region information including the region information management unit, and (3) a region information management unit that manages region information including the display region information of each communication device. Only the area information including the display area information of the apparatus is used, and the area information management unit notifies the area information including the display area information of the lower communication apparatus below the child node notified to each communication apparatus.
　本発明によれば、以下のような効果が得られる。 According to the present invention, the following effects can be obtained.
　配信元ノードおよび受信ノードの映像加工部において、子ノードやその子孫ノードのいずれにおいても表示領域とならない領域に対してのみ、送信部や再送信部で用いられる映像符号化技術による発生符号量を低減する効果のある映像加工を行うので、各受信ノードで表示される映像については映像加工による悪影響を与えることなく、子ノードに対して送信される送信データや再送信データの情報量を削減することができる。また、ネットワーク帯域に余裕ができるため、より高画質の映像符号化を施すことが可能となり、各受信ノードで出力される表示領域の映像データを高画質化することが可能となる。 In the video processing unit of the distribution source node and the reception node, the generated code amount by the video coding technique used in the transmission unit and the re-transmission unit is applied only to an area that is not a display area in any of the child node and its descendant nodes. Reduces the amount of transmission data and retransmission data sent to child nodes without adversely affecting the video displayed on each receiving node because video processing is effective. be able to. In addition, since the network bandwidth can be afforded, it is possible to perform video encoding with higher image quality, and it is possible to improve the image quality of the video data in the display area output from each receiving node.
実施形態に係る受信ノードの内部構成を示す内部構成図である。It is an internal block diagram which shows the internal structure of the receiving node which concerns on embodiment. ユニキャスト通信を説明する説明図である。It is explanatory drawing explaining unicast communication. マルチキャスト通信を説明する説明図である。It is explanatory drawing explaining multicast communication. アプリケーションレベルマルチキャスト配信を説明する説明図である。It is explanatory drawing explaining application level multicast delivery. アプリケーションレベルマルチキャストによる映像の部分表示領域データの配信を説明する説明図である。It is explanatory drawing explaining delivery of the partial display area data of the image | video by application level multicast. 実施形態に係る映像配信システムの全体構成を示す全体構成図である。It is a whole lineblock diagram showing the whole video distribution system composition concerning an embodiment. 実施形態に係る配信元ノードの内部構成を示す内部構成図である。It is an internal block diagram which shows the internal structure of the delivery origin node which concerns on embodiment. 実施形態において子ノードから親ノードへの領域情報の送受信を説明する説明図である。It is explanatory drawing explaining transmission / reception of the area | region information from a child node to a parent node in embodiment. 実施形態において親ノードから子ノードへの配信データの配信を説明する説明図である。It is explanatory drawing explaining delivery of the delivery data from a parent node to a child node in embodiment.
　（Ａ）主たる実施形態
　以下では、本発明に係る通信装置、通信方法及び通信プログラム、管理装置の実施形態を、図面を参照しながら詳細に説明する。 (A) Main Embodiments Hereinafter, embodiments of a communication device, a communication method, a communication program, and a management device according to the present invention will be described in detail with reference to the drawings.
　（Ａ－１）実施形態の構成
　［全体構成］
　図６は、実施形態に係る映像配信システムの全体構成の一例を示す全体構成図である。 (A-1) Configuration of Embodiment [Overall Configuration]
FIG. 6 is an overall configuration diagram illustrating an example of the overall configuration of the video distribution system according to the embodiment.
　図６において、実施形態に係る映像配信システム１は、映像配信管理サーバー１０、配信元ノード２０、受信ノード３０（３０－１～３０－５）を有する。 In FIG. 6, the video distribution system 1 according to the embodiment includes a video distribution management server 10, a distribution source node 20, and reception nodes 30 (30-1 to 30-5).
　実施形態に係る配信システム１は、ツリー構造のネットワークを形成しており、アプリケーションレベルマルチキャストを用いてメディアデータを配信する。 The distribution system 1 according to the embodiment forms a tree-structured network, and distributes media data using application level multicast.
　以下では、配信ツリーにおいて、配信元ノード２０から受信ノード３０側へのデータの流れを下流とし、逆に受信ノード３０から配信元ノード２０側へのデータの流れを上流とする。 In the following, in the distribution tree, the data flow from the distribution source node 20 to the reception node 30 is assumed to be downstream, and conversely, the data flow from the reception node 30 to the distribution source node 20 is assumed to be upstream.
　配信ツリーにおいて、ある受信ノード３０から上流側に位置しており、直接通信するノード（配信元ノード２０あるいは他の受信ノード３０）を「親ノード」と呼び、ある受信ノード３０から下流側に位置しており直接通信するノード（他の受信ノード３０）を「子ノード」と呼ぶ。また、配信ツリーにおいて、自装置３０の親ノードのさらに上流側に位置する親ノードを総称して「祖先ノード」とも呼び、自装置３０の子ノードのさらに下流側に位置する子ノードを「子孫ノード」とも呼ぶ。 In the distribution tree, a node (distribution source node 20 or other receiving node 30) that is located upstream from a certain receiving node 30 and directly communicates is called a “parent node” and is located downstream from a certain receiving node 30. Nodes that directly communicate with each other (other receiving nodes 30) are called “child nodes”. In the distribution tree, parent nodes located further upstream of the parent node of the own device 30 are collectively referred to as “ancestor nodes”, and child nodes located further downstream of the child nodes of the own device 30 are referred to as “descendant”. Also called “node”.
　アプリケーションレベルマルチキャストは、複数のノードにデータを配信する手法の１つである。アプリケーションレベルマルチキャストの配信手法は、配信元ノード（もしくはサーバー）の負荷を低減しつつ複数の配信先である受信ノードに配信する方法である。一般にはデータの種類として映像音声を含む多種多様なものをサポートすることができ、またリアルタイムのライブ配信にも蓄積型の配信にも対応することが可能である。 Application level multicast is one of the methods for distributing data to multiple nodes. The distribution technique of application level multicast is a method of distributing to a plurality of receiving nodes as distribution destinations while reducing the load on the distribution source node (or server). In general, a wide variety of data including video and audio can be supported as data types, and real-time live distribution and storage-type distribution can be supported.
　また、複数のノードにデータを配信する手法として、ＩＰマルチキャスト技術がある。ＩＰマルチキャスト技術は、ＩＰ層において配信データの経路制御や複製処理を行う手法である。ＩＰマルチキャスト技術では、配信経路上のルーターがデータの複製処理を行うので、配信先の受信ノードが増えても、配信元ノードの負荷が増大しない。プロトコルとしては、ＩＰｖ４ではＩＧＭＰ、ＩＰｖ６ではＭＤＬｖ２と呼ばれるものが使われる。欠点としては、配信元ノードから配信先の受信ノードまでの経路上のルーターがすべてこれらの特別なプロトコルを処理できる必要が有ることである。ＩＰマルチキャストを可能にするインフラストラクチャはまだ普及しているとは言えず、特に企業内ネットワークでは普及していない。 Also, as a technique for distributing data to a plurality of nodes, there is an IP multicast technique. The IP multicast technique is a technique for performing distribution data routing control and replication processing in the IP layer. In the IP multicast technology, a router on a distribution path performs data duplication processing, so even if the number of distribution destination receiving nodes increases, the load on the distribution source node does not increase. As a protocol, a protocol called IGMP in IPv4 and a protocol called MDLv2 in IPv6 are used. The disadvantage is that all routers on the path from the distribution source node to the distribution destination receiving node need to be able to handle these special protocols. The infrastructure that enables IP multicasting is not yet widespread, and is not particularly widespread in corporate networks.
　アプリケーションレベルマルチキャスト技術は、ＩＰマルチキャストと同様に、経路上の装置がデータを複製することによって配信元端末の負荷増大を抑え、配信先の受信ノード数の増大に対処する手法である。アプリケーションレベルマルチキャスト技術とＩＰマルチキャスト技術との違いは、データ複製を配信先の受信ノードが行う点である。 Application level multicast technology is a method of dealing with an increase in the number of receiving nodes of a distribution destination by suppressing an increase in the load of a distribution source terminal by duplicating data by a device on a route, like IP multicast. The difference between the application level multicast technology and the IP multicast technology is that the receiving node of the distribution destination performs data replication.
　配信元ノード２０が配信するメディアデータは、この実施形態では映像情報を含む場合を例示するが、音情報（音声や音響を含む音に関する情報）を含むものであってもよい。 The media data distributed by the distribution source node 20 exemplifies a case where video information is included in this embodiment, but may include sound information (information regarding sound including sound and sound).
　映像配信管理サーバー１０は、配信元ノード２０と受信ノード３０の間の映像配信ツリーの構成を管理する。映像配信管理サーバー１０は、ネットワークに新規の受信ノード３０が参加するときや受信ノード３０が離脱するときなどのように、配信ツリーの再構成が必要なときに、配信元ノード２０や各受信ノード３０に対して配信ツリーの再構成を指示する。また、後述する受信ノード３０や配信元ノード２０の間の領域情報の送受信は、直接ノード間で送受信しても良いが、映像配信管理サーバー１０が領域情報の送受信の仲介をするような構成としても良い。 The video distribution management server 10 manages the configuration of the video distribution tree between the distribution source node 20 and the reception node 30. When the distribution tree needs to be reconfigured, such as when a new receiving node 30 joins the network or when the receiving node 30 leaves, the video distribution management server 10 receives the distribution source node 20 and each receiving node. 30 is instructed to reconstruct the distribution tree. Further, transmission / reception of area information between the receiving node 30 and the distribution source node 20 to be described later may be performed directly between nodes, but the video distribution management server 10 is configured to mediate transmission / reception of area information. Also good.
　映像配信管理サーバー１０は、配信ツリーを構成する配信元ノード２０と受信ノード３０との間の配信ツリーを決定し、各ノード（配信元ノード２０、受信ノード３０）に対して配信ツリーを構成する各ノード間の配信パスの通信路の設定を指示する通信路設定部と、ノード間の領域情報の送受信を中継する中継部と、各受信ノード３０の表示領域情報を含む領域情報を管理する領域情報管理部とを有する。映像配信管理サーバー１０は、各受信ノード３０から領域情報を取得するが、そのとき各受信ノード３０から送信される領域情報には、その受信ノード３０の表示領域情報のみが含まれる。また、受信ノード３０や配信元ノード２０に通知する子ノード以下の領域情報を、映像配信管理サーバー１０が通知する。つまり、映像配信管理サーバー１０は、受信ノード３０や配信元ノード２０に対して領域情報を通知するが、その受信ノード３０や配信元ノード２０の子ノードや孫ノードなどの子孫となるノードの領域情報を、映像配信管理サーバー１０が受信ノード３０や配信元ノード２０に対して通知するようにしてもよい。 The video distribution management server 10 determines a distribution tree between the distribution source node 20 and the reception node 30 configuring the distribution tree, and configures a distribution tree for each node (distribution source node 20 and reception node 30). Area for managing area information including display area information of each receiving node 30 and a communication path setting section for instructing setting of a communication path for a distribution path between the nodes, a relay section for relaying transmission / reception of area information between nodes And an information management unit. The video distribution management server 10 acquires area information from each receiving node 30, but the area information transmitted from each receiving node 30 at that time includes only the display area information of the receiving node 30. In addition, the video distribution management server 10 notifies the area information below the child node notified to the receiving node 30 and the distribution source node 20. In other words, the video distribution management server 10 notifies the reception node 30 and the distribution source node 20 of the area information, but the areas of the descendant nodes such as the child nodes and grandchild nodes of the reception node 30 and the distribution source node 20. The video distribution management server 10 may notify the information to the receiving node 30 or the distribution source node 20.
　配信元ノード２０は、配信したい映像のコンテンツデータや、ビデオカメラ等の映像ソースからの映像データ等の映像情報を、各受信ノード３０に配信する。配信元ノード２０は、映像配信管理サーバー１０の指示に応じて、複数の受信ノード３０のうち、直接の配信先となる受信ノード３０のそれぞれに、配信したい映像情報に対して受信ノード３０から通知される領域情報に応じた加工処理を行った映像情報の送信を行う。 The distribution source node 20 distributes video information such as video content data to be distributed and video data from a video source such as a video camera to each receiving node 30. In response to an instruction from the video distribution management server 10, the distribution source node 20 notifies each of the reception nodes 30 that are direct distribution destinations of the plurality of reception nodes 30 of the video information desired to be distributed from the reception node 30. The video information that has been processed according to the region information to be transmitted is transmitted.
　受信ノード３０（３０－１～３０－５）は、自装置よりも上流側に位置しているノード（例えば、配信元ノード２０、他の受信ノード３０）から、通信回線を介して映像情報を受信し、自装置の表示領域情報に従って、受信した映像のフレーム内の表示領域を抽出し、その抽出した表示領域の映像を表示する。また、受信ノード３０は、映像配信管理サーバー１０の指示に応じて、自装置を親ノードとしている他の受信ノード３０（すなわち、自装置の各子ノード）のそれぞれに、受信した映像情報に対して、他の受信ノード３０から通知される領域情報に応じた加工処理を行い、加工処理後の映像情報の再配信を行う。受信ノード３０は、例えば、スマートフォン、タブレット端末、パーソナルコンピュータ、ウェラブル端末等を適用することができる。 The receiving nodes 30 (30-1 to 30-5) receive video information from nodes (for example, the distribution source node 20 and other receiving nodes 30) located on the upstream side of the own device via a communication line. Received, extracts the display area in the frame of the received video according to the display area information of its own device, and displays the video of the extracted display area. Also, the receiving node 30 responds to the received video information to each of the other receiving nodes 30 having the own device as a parent node (that is, each child node of the own device) according to an instruction from the video distribution management server 10. Then, the processing according to the area information notified from the other receiving node 30 is performed, and the video information after the processing is redistributed. As the receiving node 30, for example, a smartphone, a tablet terminal, a personal computer, a wearable terminal, or the like can be applied.
　なお、図６において、配信元ノード２０は複数存在してもよく、映像配信管理サーバー１０では配信ツリーをそれぞれの配信元ごと（配信したい映像コンテンツごと）に管理する。例えば、複数の拠点にそれぞれ配信元ノード２０を配置し、複数拠点間で双方向に情報を配信しあうことでコミュニケーションを実現するような構成としてもよい。 In FIG. 6, there may be a plurality of distribution source nodes 20, and the video distribution management server 10 manages the distribution tree for each distribution source (for each video content to be distributed). For example, the distribution source node 20 may be arranged at each of a plurality of bases, and communication may be realized by distributing information bidirectionally between the plurality of bases.
　［受信ノード３０の内部構成］
　図１は、実施形態に係る受信ノード３０の内部構成を示す内部構成図である。 [Internal configuration of receiving node 30]
FIG. 1 is an internal configuration diagram illustrating an internal configuration of the receiving node 30 according to the embodiment.
　図１において、実施形態に係る受信ノード３０は、受信部３０１、表示領域抽出部３０２、映像加工部３０３、再送信部３０４、領域情報受信部３０５、領域情報管理部３０６、領域情報送信部３０７を備える。 In FIG. 1, the reception node 30 according to the embodiment includes a reception unit 301, a display region extraction unit 302, a video processing unit 303, a retransmission unit 304, a region information reception unit 305, a region information management unit 306, and a region information transmission unit 307. Is provided.
　受信ノード３０は、例えば、ＣＰＵ、ＲＯＭ、ＲＡＭ、ＥＥＰＲＯＭ、入出力インタフェース部、通信装置などを備えるものであり、ＣＰＵがＲＯＭに格納される、あるいは、サーバーからダウンロードされる処理プログラム（通信プログラム）を実行することにより、後述する受信ノード３０としての各種機能が実現される。また、当該処理プログラム（通信プログラム）を記録した、コンピュータに読み取り可能な記録媒体も提供され得る。 The reception node 30 includes, for example, a CPU, a ROM, a RAM, an EEPROM, an input / output interface unit, a communication device, and the like, and a processing program (communication program) stored in the ROM or downloaded from a server. By executing the above, various functions as the receiving node 30 described later are realized. A computer-readable recording medium that records the processing program (communication program) may also be provided.
　受信部３０１は、自装置３０の親ノード（配信元ノード２０若しくは他の受信ノード３０）から映像を含む配信データを受信するものである。ツリー構造を構築している映像配信システム１で、配信元ノード２０又は配信元ノード２０に近い位置にあるノードを親ノードとする。 The receiving unit 301 receives distribution data including video from the parent node (distribution source node 20 or other receiving node 30) of the own device 30. In the video distribution system 1 in which the tree structure is constructed, a distribution source node 20 or a node close to the distribution source node 20 is set as a parent node.
　表示領域抽出部３０２は、自装置３０のユーザーにより指示された表示領域情報が取得されており、受信部３０１により受信された映像から、ユーザーから指示された表示領域情報に基づいて表示データを抽出し、その表示データを出力する。 The display area extraction unit 302 has acquired display area information instructed by the user of the device 30 and extracts display data from the video received by the receiving unit 301 based on the display area information instructed by the user. The display data is output.
　映像加工部３０３は、自装置３０が子ノード（他の受信ノード３０）に再配信を行う中継ノードとして機能する場合に、領域情報管理部３０６に保持されている子ノードやその子孫ノードの表示領域情報を含む領域情報を読み出し、その子ノードやその子孫ノードの表示領域情報を含む領域情報に基づいて、映像を加工して子ノードへの配信用映像とし、その配信用映像を再送信部３０４に与える。 When the image processing unit 303 functions as a relay node that performs redistribution to a child node (another receiving node 30), the video processing unit 303 displays a child node and its descendant nodes held in the area information management unit 306. The area information including the area information is read out, the video is processed based on the area information including the display area information of the child node and the descendant node, and the distribution video is transmitted to the child node. To give.
　再送信部３０４は、映像加工部３０３により加工された子ノードへの配信用映像を、子ノードに配信するものである。 The re-transmission unit 304 distributes the distribution video to the child node processed by the video processing unit 303 to the child node.
　領域情報受信部３０５は、自装置３０の子ノードやその子孫ノード（すなわち、ツリー構造の映像配信システム１で子ノード以下のツリーを構成する他の受信ノード３０）から表示領域情報を含む領域情報を受信し、その表示領域情報を含む領域情報を領域情報管理部３０６に与える。 The area information receiving unit 305 includes area information including display area information from a child node of its own device 30 and its descendant nodes (that is, other receiving nodes 30 constituting a tree below the child node in the tree structure video distribution system 1). Is received, and region information including the display region information is provided to the region information management unit 306.
　領域情報管理部３０６は、ユーザーにより指示された自装置３０の表示領域情報と、領域情報受信部３０５により受信された領域情報とを管理する。 The area information management unit 306 manages the display area information of the own device 30 instructed by the user and the area information received by the area information receiving unit 305.
　領域情報送信部３０７は、領域情報管理部３０６で保持されている領域情報から、自装置３０の表示領域情報を含む領域情報や、子ノード若しくはその子孫ノードの表示領域情報を含む領域情報を、親ノードに送信する。 The area information transmission unit 307 obtains area information including the display area information of the own device 30 and area information including the display area information of the child node or its descendant node from the area information held in the area information management unit 306. Send to parent node.
　［配信元ノード２０の内部構成］
　図７は、実施形態に係る配信元ノード２０の内部構成を示す内部構成図である。 [Internal configuration of distribution source node 20]
FIG. 7 is an internal configuration diagram illustrating an internal configuration of the distribution source node 20 according to the embodiment.
　図７において、実施形態に係る配信元ノード２０は、映像加工部２０３、送信部２０４、領域情報受信部２０５、領域情報管理部２０６を有する。 7, the distribution source node 20 according to the embodiment includes a video processing unit 203, a transmission unit 204, a region information reception unit 205, and a region information management unit 206.
　配信元ノード２０は、例えば、ＣＰＵ、ＲＯＭ、ＲＡＭ、ＥＥＰＲＯＭ、入出力インタフェース部、通信装置などを備えるものであり、ＣＰＵがＲＯＭに格納される、あるいは、サーバーからダウンロードされる処理プログラム（通信プログラム）を実行することにより、後述する配信元ノード２０としての各種機能が実現される。また、当該処理プログラム（通信プログラム）を記録した、コンピュータに読み取り可能な記録媒体も提供され得る。 The distribution source node 20 includes, for example, a CPU, a ROM, a RAM, an EEPROM, an input / output interface unit, a communication device, and the like, and a processing program (communication program) stored in the ROM or downloaded from a server. ), Various functions as the distribution source node 20 described later are realized. A computer-readable recording medium that records the processing program (communication program) may also be provided.
　映像加工部２０３は、配信したい映像を含む配信データを取得し、領域情報管理部２０６に保持されている子ノードやその子孫ノードの表示領域情報を含む領域情報に基づいて映像を加工して配信用映像を生成し、自装置２０から直接配信を受ける子ノード（受信ノード３０）への配信用映像を送信部２０４に与える。 The video processing unit 203 acquires distribution data including the video to be distributed, processes and distributes the video based on the area information including the display area information of the child node and its descendant nodes held in the area information management unit 206. A video for distribution is generated, and a video for distribution to a child node (receiving node 30) that receives distribution directly from the own device 20 is given to the transmission unit 204.
　送信部２０４は、映像加工部２０３により加工された配信用映像を含む送信データを送信するものである。 The transmission unit 204 transmits transmission data including the distribution video processed by the video processing unit 203.
　領域情報受信部２０５は、子ノードやその子孫ノード（配信ツリー上の子ノード以下のツリーを構成する受信ノード３０）の表示領域情報を含む領域情報を子ノードから受信し、その受信した領域情報を領域情報管理部２０６に与える。 The area information receiving unit 205 receives area information including display area information of a child node and its descendant nodes (receiving nodes 30 constituting a tree below the child node on the distribution tree) from the child node, and the received area information Is given to the area information management unit 206.
　領域情報管理部２０６は、領域情報受信部２０５により受信された子ノードやその子孫ノードから受信した領域情報を管理する。 The area information management unit 206 manages the area information received from the child node and its descendant nodes received by the area information receiving unit 205.
　（Ａ－２）実施形態の動作
　次に、実施形態に係る映像配信システム１における映像を含む配信データの配信処理を、図面を参照しながら詳細に説明する。 (A-2) Operation of Embodiment Next, distribution processing of distribution data including video in the video distribution system 1 according to the embodiment will be described in detail with reference to the drawings.
　図６に示すように、配信したい映像情報の配信元である配信元ノード２０から、映像情報の受信者である（複数の）受信ノード３０に対してアプリケーションレベルマルチキャストを用いて映像情報の配信を行う。 As shown in FIG. 6, the distribution of the video information from the distribution source node 20 that is the distribution source of the video information to be distributed to the receiving nodes 30 that are the recipients of the video information using application level multicast. Do.
　映像情報の配信経路となる配信ツリーの構成状態は映像配信管理サーバー１０が管理しており、配信元ノード２０や受信ノード３０との間で、各ノードに関するノード情報や配信ツリーの構成に関わる情報をやり取りすることにより、動的なノードの追加や離脱に応じた配信ツリーの再構築を行う。具体的な配信ツリーの構成に関わる情報の内容や、具体的な配信パスとなる通信路を設定するためのシグナリング等については任意の方法を用いることができる。例えば映像配信管理サーバー１０がノード間の通信路設定のためのシグナリングを中継するシグナリングサーバーを兼務するような構成としても良いし、また例えば、受信ノード３０や配信元ノード２０の間の領域情報の送受信を行うための通信路の設定のためのシグナリングサーバーを兼務したり、あるいは、ノード間では領域情報を直接送受信せず、ノード間の領域情報の送受信を映像配信管理サーバー１０が中継する役割を兼務するような構成としても良い。 The distribution state of the distribution tree serving as the distribution route of the video information is managed by the video distribution management server 10, and node information related to each node and information regarding the configuration of the distribution tree with the distribution source node 20 and the reception node 30. By exchanging, the distribution tree is rebuilt according to the dynamic addition and removal of nodes. Arbitrary methods can be used for the contents of information related to the configuration of a specific distribution tree, signaling for setting a communication path as a specific distribution path, and the like. For example, the video distribution management server 10 may be configured to also serve as a signaling server that relays signaling for setting a communication path between nodes. For example, the region information between the receiving node 30 and the distribution source node 20 The video distribution management server 10 also serves as a signaling server for setting a communication path for transmission / reception, or relays transmission / reception of area information between nodes without directly transmitting / receiving area information between nodes. It is good also as a structure which serves concurrently.
　まず、配信元ノード２０は、配信したい映像である配信データを入力し、配信元ノードとしてのノード情報を映像配信管理サーバー１０に通知する。 First, the distribution source node 20 inputs distribution data that is a video to be distributed, and notifies the video distribution management server 10 of node information as a distribution source node.
　受信ノード３０は、例えば、配信を受けたい映像データを特定する情報などを含む、受信ノードとしてのノード情報を映像配信管理サーバー１０に通知する。 The receiving node 30 notifies the video distribution management server 10 of node information as a receiving node including, for example, information for specifying video data to be distributed.
　映像配信管理サーバー１０は、配信元ノード２０や、（１つ以上の）受信ノード３０で構成する配信ツリーの構成を決定し、それぞれのノード（配信元ノード２０と受信ノード３０）に対して配信パスの接続相手に関わる情報を通知する。 The video distribution management server 10 determines the configuration of a distribution tree composed of the distribution source node 20 and (one or more) reception nodes 30, and distributes to each node (the distribution source node 20 and the reception node 30). Notify information related to the connection partner of the path.
　配信元ノード２０や受信ノード３０の各ノードは、配信データを受信するための親ノードとの通信路や、配信データを送信するための子ノードへの通信路を設定し、配信データの送受信を行う。 Each node of the distribution source node 20 and the reception node 30 sets a communication path with a parent node for receiving distribution data and a communication path to a child node for transmitting distribution data, and transmits / receives distribution data. Do.
　配信元ノード２０や受信ノード３０の各ノードは、配信パスに従った接続相手との通信路を確保し、配信元ノード２０は自身が持つ映像情報の加工処理後の映像データの送信を、受信ノード３０は親ノードからの配信データの受信と、自身が子ノードを持つ場合は、受信した映像データの加工処理後の映像データの再送信を行う。 Each node of the distribution source node 20 and the reception node 30 secures a communication path with the connection partner according to the distribution path, and the distribution source node 20 receives the transmission of the video data after processing the video information that the distribution source node 20 has. The node 30 receives the distribution data from the parent node, and retransmits the video data after processing the received video data when the node 30 has a child node.
　新規に受信ノード３０が配信ツリーに参加する場合は、映像配信管理サーバー１０は、新規の受信ノード３０から通知されるノード情報にしたがって、配信元となる親ノード（配信元ノード２０もしくはすでに配信を受けている受信ノード３０）を割り当て、割り当てられた親ノードが、新規受信ノード３０への配信を開始することにより、配信ツリーの構築を行う。 When a new reception node 30 participates in the distribution tree, the video distribution management server 10 follows the node information notified from the new reception node 30 to the parent node (distribution source node 20 or already distributed). The receiving node 30) is allocated, and the assigned parent node starts distribution to the new receiving node 30, thereby constructing a distribution tree.
　配信ツリーに参加している受信ノード３０の内の１つのノード装置３０が配信ツリーから離脱した場合は、離脱したノードを親ノード（や祖先ノード）としていた受信ノード３０への配信が停止してしまうため、配信が停止してしまう受信ノード３０に対して新たな親ノードの割り当てを行う。親ノードの再割り当ては、離脱ノードの直接の子ノードに対してのみ行っても良いし、直接の子ノード以外にも、子ノードの子孫のノードに対しても再割り当てを行っても良い。これらの受信ノード３０に対して、親ノードの再割り当てを行うことにより、配信ツリーの再構築を行う。 When one node device 30 among the receiving nodes 30 participating in the distribution tree leaves the distribution tree, distribution to the receiving node 30 whose parent node (or ancestor node) is the stopped node is stopped. Therefore, a new parent node is assigned to the receiving node 30 whose distribution is stopped. The reassignment of the parent node may be performed only for the direct child node of the leaving node, or the reassignment may be performed not only for the direct child node but also for the descendant node of the child node. By reallocating parent nodes to these receiving nodes 30, the distribution tree is reconstructed.
　［各受信ノード３０における処理動作］
　実施形態に係る各受信ノード３０における処理動作を、図１を参照して説明する。 [Processing operation in each receiving node 30]
The processing operation in each receiving node 30 according to the embodiment will be described with reference to FIG.
　受信ノード３０では、それぞれの受信ノード３０で表示したい領域を示す情報である表示領域情報が入力される。これは、例えば、あらかじめ受信ノード３０ごとに設定された領域情報でもよく、また、例えばマウスやタッチ操作などのユーザーインターフェースを介したユーザーからの指示による領域情報でもよい。 In the receiving node 30, display area information which is information indicating an area to be displayed on each receiving node 30 is input. This may be, for example, area information set for each receiving node 30 in advance, or may be area information based on an instruction from the user via a user interface such as a mouse or a touch operation.
　受信ノード３０は、映像情報の直接の配信元である親ノード（配信元ノード２０もしくは他の受信ノード３０）からの映像データを受信部３０１で受信する。受信した映像データは入力される表示領域情報にしたがって、表示領域抽出部３０２で、映像データの一部分が抽出され、表示データとして出力され、ディスプレイ等に表示される。 The receiving node 30 receives the video data from the parent node (distribution source node 20 or other receiving node 30) that is the direct distribution source of the video information by the reception unit 301. A part of the video data is extracted from the received video data in accordance with the input display area information by the display area extraction unit 302, output as display data, and displayed on a display or the like.
　受信ノード３０が子ノード（他の受信ノード３０）への再配信を行う中継ノードである場合は、映像加工部３０３で、子ノードへの配信用に映像を加工し、加工された映像情報を再送信部３０４からそれぞれの子ノードに対して送信を行う。子ノードが存在しない場合は、加工処理及び再送信処理は行わない。 When the receiving node 30 is a relay node that performs re-distribution to a child node (another receiving node 30), the video processing unit 303 processes the video for distribution to the child node, and the processed video information is displayed. Transmission is performed from the retransmission unit 304 to each child node. When there is no child node, the processing process and the retransmission process are not performed.
　子ノードからは、子ノードである他の受信ノード３０および配信ツリーにおけるその子孫である他の受信ノード３０のそれぞれの表示領域情報を含む子ノードからの領域情報が通知され、領域情報受信部３０５で受信する。 The child node is notified of region information from the child node including the display region information of the other receiving nodes 30 that are child nodes and the other receiving nodes 30 that are descendants in the distribution tree, and the region information receiving unit 305. Receive at.
　受信した子ノードからの領域情報と、受信ノード３０自身の表示領域情報は、領域情報管理部３０６に入力され、領域情報管理部３０６は、これらの領域情報にしたがった映像加工対象領域情報を映像加工部３０３に供給する。 The received region information from the child node and the display region information of the receiving node 30 itself are input to the region information management unit 306, and the region information management unit 306 displays the video processing target region information according to these region information as video. This is supplied to the processing unit 303.
　また、領域情報管理部３０６は、子ノードからの領域情報と、自身の表示領域情報を含む領域情報を親ノードに通知するために領域情報送信部３０７に供給し、領域情報送信部３０７から親ノードに対して通知を行う。 The area information management unit 306 supplies the area information from the child node and the area information including its own display area information to the area information transmission unit 307 in order to notify the parent node. Notify the node.
　［配信元ノード２０における処理動作］
　配信元ノード２０の動作を、図７を参照しながら説明する。 [Processing Operation in Distribution Source Node 20]
The operation of the distribution source node 20 will be described with reference to FIG.
　配信元ノード２０には、配信したい映像である配信データが入力されるが、これをそのまま送信するのではなく、子ノードから通知される領域情報に従った映像加工処理を行ったうえで送信を行う。 Distribution data that is a video to be distributed is input to the distribution source node 20, but this is not transmitted as it is, but is transmitted after performing a video processing process according to area information notified from the child node. Do.
　入力された配信データである映像データは、映像加工部２０３で、配信元ノード２０から直接配信を受ける子ノード（受信ノード３０）への配信用に映像を加工する処理を行う。 The video data that is the input distribution data is processed by the video processing unit 203 for processing the video for distribution to the child node (receiving node 30) that receives the direct distribution from the distribution source node 20.
　加工された映像データは、送信部２０４からそれぞれの子ノードに対して送信される。 The processed video data is transmitted from the transmission unit 204 to each child node.
　子ノードからは、子ノードやその子孫ノード（配信ツリー上の子ノード以下のツリーを構成する受信ノード３０）の表示領域情報を含む子ノードからの領域情報を、領域情報受信部２０５で受信する。 From the child node, the region information receiving unit 205 receives the region information from the child node including the display region information of the child node and its descendant node (the receiving node 30 constituting the tree below the child node on the distribution tree). .
　領域情報受信部２０５で受信した、領域情報は領域情報管理部２０６に供給され、映像加工部２０３では領域情報管理部２０６で保持する子ノードやその子孫ノードの表示領域情報を含む領域情報にしたがって映像を加工する処理を行う。 The region information received by the region information receiving unit 205 is supplied to the region information management unit 206, and the video processing unit 203 follows the region information including the display region information of the child node and its descendant nodes held by the region information management unit 206. Processing to process the video.
　［領域情報の送受信処理］
　以下、受信ノード３０や配信元ノード２０の間でやり取りされる領域情報の様子を、図８を参照しながら説明する。 [Area information send / receive processing]
Hereinafter, the state of the area information exchanged between the receiving node 30 and the distribution source node 20 will be described with reference to FIG.
　図８は、実施形態の子ノードから親ノードへの領域情報の送受信処理を説明する説明図である。 FIG. 8 is an explanatory diagram illustrating the transmission / reception process of the area information from the child node to the parent node according to the embodiment.
　図８では、説明を容易にするために、例えば「ユーザー（３０－１）」の表記は配信元ノード２０から受信ノード３０－１に配信される映像のフレームを示しており、Ｄ１は、受信ノード３０－１のユーザーが映像フレーム内で選択した表示領域を示している。また、点線で囲んでいるものは、ノード間で授受される領域情報をイメージ化している。 In FIG. 8, for ease of explanation, for example, the notation “user (30-1)” indicates a frame of a video distributed from the distribution source node 20 to the reception node 30-1, and D1 indicates a reception The display area selected in the video frame by the user of the node 30-1 is shown. In addition, what is surrounded by a dotted line is an image of region information exchanged between nodes.
　図８において、配信元ノード２０から映像配信を受ける各受信ノード３０－１～３０－５では、各ユーザーによりそれぞれが表示したい領域である表示領域Ｄ１～Ｄ５が指示されている。それぞれの受信ノード３０は、子ノードから通知される領域情報に、自装置３０においてユーザーに指示された表示領域情報を追加したうえで、親ノードへの領域情報として通知する。 In FIG. 8, in each of the receiving nodes 30-1 to 30-5 that receives video distribution from the distribution source node 20, display areas D1 to D5, which are areas to be displayed, are designated by the respective users. Each receiving node 30 adds the display area information instructed by the user in its own device 30 to the area information notified from the child node, and notifies it as area information to the parent node.
　例えば、図８において、受信ノード３０－３のユーザー（３０－３）は、子ノードであるユーザ（３０－２）から表示領域Ｄ２を取得すると、その表示領域Ｄ２に自装置３０－３の表示領域Ｄ３を加えたものを領域情報として、親ノードであるユーザー（３０－５）に送信する。図８の例では、各受信ノード３０でユーザーにより指示された表示領域情報が矩形領域であるものとして説明しているが、表示領域情報は、矩形領域に限定されるものでない。表示領域情報は、映像フレームにおいて表示領域を特定する情報とすることができ、例えば、矩形領域である場合、矩形領域の左上座標値、矩形領域の幅を示す値、高さを示す値等などの情報で領域情報を表現しても良い。 For example, in FIG. 8, when the user (30-3) of the receiving node 30-3 acquires the display area D2 from the child node user (30-2), the display of the own device 30-3 is displayed in the display area D2. The information including the area D3 is transmitted as area information to the user (30-5) as the parent node. In the example of FIG. 8, the display area information instructed by the user at each receiving node 30 is described as a rectangular area, but the display area information is not limited to the rectangular area. The display area information can be information for specifying the display area in the video frame. For example, in the case of a rectangular area, the upper left coordinate value of the rectangular area, the value indicating the width of the rectangular area, the value indicating the height, etc. The area information may be expressed by this information.
　各受信ノード３０の領域情報管理部３０６は、子ノードや子孫ノードから収集した領域情報をリスト状で管理するようにしてもよい。具体的には、子ノードや子孫ノードの識別情報（例えば、映像配信管理サーバーから各ノードに付与された識別子情報等）に、子ノードや子孫ノードから収集した表示領域情報を含む領域情報を対応付けて管理する。これにより、子ノードや子孫ノード毎に、表示領域情報を含む領域情報を管理することができる。あるいは、直接の子ノード毎に、その子ノード以下の全ての領域情報のリスト（個々のノードは特定しない）として管理することができる。 The area information management unit 306 of each receiving node 30 may manage area information collected from child nodes and descendant nodes in a list form. Specifically, region information including display region information collected from child nodes and descendant nodes is associated with identification information of child nodes and descendant nodes (for example, identifier information assigned to each node from the video distribution management server). Manage it. Thereby, area information including display area information can be managed for each child node and descendant node. Alternatively, for each direct child node, it can be managed as a list of all area information below the child node (individual nodes are not specified).
　また、領域情報管理部３０６は、自装置３０の表示領域情報と、子ノードや子孫ノードの表示領域情報を含む領域情報とに基づいて、包含関係にある領域を統合するようにしてもよい。例えば、図８において、ユーザー（３０－３）では、子ノードであるユーザー（３０－２）の表示領域Ｄ２と、受信ノード３０－３の表示領域Ｄ３とが一部領域で重なっている。このような場合に、受信ノード３０－３の領域情報管理部３０６は、ユーザー（３０－２）の表示領域Ｄ２又は受信ノード３０－３の表示領域Ｄ３のいずれか一方について重複領域を削除し、重複領域を統合するようにしてもよい。これにより、重複領域を省いた領域情報を配信元ノード２０や受信ノード３０に指示することができるため、親ノードへ通知する領域情報のデータ量を削減することができる。 Further, the area information management unit 306 may integrate the areas in the inclusion relationship based on the display area information of the own device 30 and the area information including the display area information of the child node and the descendant node. For example, in FIG. 8, in the user (30-3), the display area D2 of the user (30-2), which is a child node, and the display area D3 of the receiving node 30-3 partially overlap. In such a case, the area information management unit 306 of the receiving node 30-3 deletes the overlapping area for either the display area D2 of the user (30-2) or the display area D3 of the receiving node 30-3, You may make it integrate an overlapping area | region. As a result, the area information excluding the overlapping area can be instructed to the distribution source node 20 and the receiving node 30, so that the data amount of the area information notified to the parent node can be reduced.
　さらに、領域情報管理部３０６は、自装置３０の表示領域情報と、子ノードや子孫ノードの表示領域情報を含む領域情報とに基づいて、複数の表示領域が重なっている場合に、その全ての領域を覆うような大きな領域に統合するようにしてもよい。例えば、図８において、ユーザー（３０－５）では、ユーザー（３０－２）の表示領域Ｄ２、ユーザー（３０－３）の表示領域Ｄ３、ユーザー（３０－４）の表示領域Ｄ４、ユーザー（３０－５）の表示領域Ｄ５が重なり合っているが、これらすべてを囲うようなより大きな１つの矩形領域に統合してしまい、この１つの領域情報を配信元ノード２０に送信する。このように、複数の表示領域が重なり合っており、比較的大きな表示領域の領域情報を送信する場合には、これら表示領域を統合した領域情報を形成したり、又はこれら領域を包含する大きな領域とする領域情報を形成するようにしてもよい。これにより、重複領域を包含する領域情報を配信元ノード２０や受信ノード３０に指示することができるため、親ノードに通知する領域情報のデータ量を削減することができる。 Furthermore, the area information management unit 306, when a plurality of display areas are overlapped based on the display area information of the own device 30 and the area information including the display area information of the child node and the descendant node, You may make it integrate in a big area | region which covers an area | region. For example, in FIG. 8, for the user (30-5), the display area D2 of the user (30-2), the display area D3 of the user (30-3), the display area D4 of the user (30-4), and the user (30 Although the display area D5 of −5) is overlapped, it is integrated into one larger rectangular area surrounding all of them, and this one area information is transmitted to the distribution source node 20. Thus, when a plurality of display areas are overlapped and the area information of a relatively large display area is transmitted, area information that integrates these display areas is formed, or a large area that includes these areas Area information to be formed may be formed. Thereby, since the area information including the overlapping area can be instructed to the distribution source node 20 and the receiving node 30, the data amount of the area information notified to the parent node can be reduced.
　これらの領域情報は順次経路上の受信ノード３０の表示領域が追加されながら親ノードに伝搬し、配信元ノード２０まで通知される。 These area information is sequentially propagated to the parent node while the display area of the receiving node 30 on the route is added, and is notified to the distribution source node 20.
　なお、これら領域情報の送受信は、ノード間（受信ノード３０や配信元ノード２０の間）で直接の通信路を設けてやり取りする構成としても良いが、映像配信管理サーバー１０などのサーバーを介した中継により情報交換する構成としても良い。さらに、例えば、映像配信管理サーバー１０で中継するような場合には、領域情報送信部３０７からは自装置の表示領域情報のみを送信し、映像配信管理サーバー１０で配信ツリーの構成状態に従った、領域情報の集約を行ったうえで、領域情報受信部３０５や領域情報受信部２０５に送信するような構成としても良い。すなわち、領域情報送信部３０７からは自装置の表示領域情報のみを、映像配信管理サーバー１０に送信する。映像配信管理サーバー１０はこの領域情報を受け取るべき親ノードや先祖ノードである受信ノード３０や配信元ノード２０を配信ツリーの構成状態にしたがって特定し、これらのノードに対して、領域情報を送信した受信ノード３０を含む配信ツリーのサブツリー（配信ツリー上の子ノード以下のツリーを構成する他の受信ノード３０）の全てのノードの領域情報を集約して送信する。領域情報受信部３０５や領域情報受信部２０５は、映像配信管理サーバー１０から各子ノード以下の配信ツリーのサブツリーの領域情報を受信する。領域の重なり等による領域情報の統合も映像配信管理サーバー１０で行うように構成しても良い。 In addition, although transmission / reception of these area | region information is good also as a structure which provides and communicates by providing a direct communication path between nodes (between the receiving node 30 and the delivery origin node 20), it passes via servers, such as the video delivery management server 10. A configuration may be adopted in which information is exchanged by relay. Further, for example, when relaying by the video distribution management server 10, only the display area information of its own device is transmitted from the area information transmission unit 307, and the video distribution management server 10 follows the configuration state of the distribution tree. The information may be transmitted to the region information receiving unit 305 and the region information receiving unit 205 after the region information is aggregated. That is, the area information transmission unit 307 transmits only the display area information of its own device to the video distribution management server 10. The video distribution management server 10 identifies the receiving node 30 and the distribution source node 20 that are the parent node and the ancestor node that should receive the area information according to the configuration state of the distribution tree, and transmits the area information to these nodes. The area information of all the nodes of the subtree of the distribution tree including the receiving node 30 (the other receiving nodes 30 constituting the tree below the child node on the distribution tree) is aggregated and transmitted. The area information receiving unit 305 and the area information receiving unit 205 receive the area information of the subtree of the distribution tree below each child node from the video distribution management server 10. The image distribution management server 10 may also be configured to integrate region information due to region overlap and the like.
　［映像配信処理］
　次に、図９を参照しながら、各ノード（配信ノード２０および受信ノード３０）での映像加工の様子を説明する。 [Video distribution processing]
Next, with reference to FIG. 9, the state of image processing at each node (distribution node 20 and reception node 30) will be described.
　図９も、図８と同様に、説明を容易にするために、配信する映像データをイメージ化している。例えば「ユーザー（３０－１）」の表記は配信元ノード２０から受信ノード３０－１に配信される映像のフレームを示しており、Ｄ１は、受信ノード３０－１のユーザーが映像フレーム内で選択した表示領域を示している。また、点線で囲んでいるものは配信データを示しており、ハッチがかかっている部分が映像加工を施した部分を示しており、白抜き部分が映像加工を施していない部分を示している。 In FIG. 9, as in FIG. 8, the video data to be distributed is imaged for ease of explanation. For example, the notation “user (30-1)” indicates a frame of video distributed from the distribution source node 20 to the reception node 30-1, and D1 is selected by the user of the reception node 30-1 within the video frame. The displayed area is shown. In addition, what is surrounded by a dotted line indicates distribution data, a hatched portion indicates a portion subjected to video processing, and a white portion indicates a portion where video processing is not performed.
　図８に示すように、子ノードからは、子ノードやその子孫ノード（配信ツリー上の子ノード以下のツリーを構成する他の受信ノード３０）の表示領域情報を集約した領域情報が通知されている。 As shown in FIG. 8, the child node is notified of area information obtained by collecting display area information of the child node and its descendant nodes (other receiving nodes 30 constituting the tree below the child node on the distribution tree). Yes.
　各ノード（配信ノード２０および受信ノード３０）における映像加工部２０３および映像加工部３０３では、その配信パス以下のいずれの受信ノード３０でも表示領域とならないような領域（図９のハッチで表現した領域）に対して映像加工を行う。配信パス以下のいずれかの受信ノード３０で表示領域となる領域（図９の白抜き部分で表現した領域）は映像加工の対象とはしない。 In the video processing unit 203 and the video processing unit 303 in each node (the distribution node 20 and the reception node 30), an area that is not a display area in any of the reception nodes 30 below the distribution path (an area expressed by hatching in FIG. 9). ). An area that is a display area in any of the receiving nodes 30 below the distribution path (an area expressed by a white portion in FIG. 9) is not a target of video processing.
　配信元ノード２０および受信ノード３０の映像加工部２０３、３０３では、それぞれの領域情報管理部２０６、３０６で保持する子ノードやその子孫ノードの表示領域情報を含む領域情報にしたがって、子ノードやその子孫ノードのいずれにおいても表示領域とならない領域（加工対象領域）に対して、送信部２０４や再送信部３０４で用いられる映像符号化技術による発生符号量を低減する効果のある映像加工を行う。 In the video processing units 203 and 303 of the distribution source node 20 and the reception node 30, the child node and its child node are displayed in accordance with the region information including the display region information of the child node and its descendant nodes held by the region information management units 206 and 306. For an area (processing area) that is not a display area in any of the descendant nodes, video processing that is effective in reducing the amount of generated code by the video encoding technique used in the transmission unit 204 and the retransmission unit 304 is performed.
　ここで、映像加工部２０３、３０３による映像加工の方法の一例を説明する。映像加工部２０３、３０３は、例えば、単純には加工対象の領域を単色（灰色など）に塗りつぶすようにしてもよい。あるいは、映像加工部２０３、３０３は、ガウスフィルタのようなぼかしフィルタ処理を行うようにしてもよい。送信部２０４や再送信部３０４で用いられる映像符号化技術では、例えば離散コサイン変換とエントロピー符号化が用いられるなどしており、複雑なテキスチャの領域や鮮明なエッジのある領域は符号量が発生しやすいが、単色領域や低域通過フィルタ処理された領域は符号量の発生が低減できる。 Here, an example of a video processing method by the video processing units 203 and 303 will be described. For example, the image processing units 203 and 303 may simply paint the region to be processed in a single color (gray). Alternatively, the video processing units 203 and 303 may perform a blur filter process such as a Gaussian filter. For example, discrete cosine transform and entropy coding are used in the video coding technique used in the transmission unit 204 and the retransmission unit 304, and a code amount is generated in a complex texture region or a region with a sharp edge. However, the generation of the code amount can be reduced in the monochromatic region or the region subjected to the low-pass filter processing.
　なお、受信ノード３０では、そのノードで初めて加工対象領域となる領域（自身の表示領域であるが子ノードからの領域情報には含まれない領域）に対してのみ加工処理を行うことにより処理量を削減するような構成としても良い。例えば、図８において、ユーザー（３０－３）で、親ノードであるユーザー（３０－５）から映像加工された映像データを受信するが、このとき、ユーザー（３０－３）では、子ノードであるユーザー（３０－２）の表示領域情報を含む領域情報に基づいて、ユーザー（３０－２）の表示領域以外の領域のみを映像加工することになる。これにより、各受信ノード３０は、子ノードや子孫ノードの表示領域以外の自身の表示領域のみ、映像加工を行えばよい（それ以外の領域はすでに親ノードで映像加工されている）ので、映像加工に係る処理量を削減することができる。 The receiving node 30 performs processing only on an area that is the first processing target area (an area that is its own display area but is not included in the area information from the child node). It is good also as a structure which reduces this. For example, in FIG. 8, the user (30-3) receives the video data that has been processed from the parent node user (30-5). At this time, the user (30-3) is the child node. Based on the area information including the display area information of a certain user (30-2), only the area other than the display area of the user (30-2) is processed. As a result, each receiving node 30 only needs to perform video processing on its own display area other than the display area of the child nodes and descendant nodes (the other areas are already processed by the parent node). The amount of processing can be reduced.
　また、表示領域が動的に変化するような場合（ユーザーのマウス操作など）で、かつ、領域情報の送受信による交換と、加工処理のタイミングの同期をとることが困難な場合は、加工対象としない領域をあらかじめある範囲にわたって拡大しておいたり、加工対象としない領域周辺の加工対象領域に対しては弱いフィルタ処理を施すような処理を行っても良い。 Also, if the display area changes dynamically (such as a user's mouse operation) and it is difficult to synchronize the processing timing with the exchange of area information transmission and reception, An area that is not to be processed may be enlarged over a certain range in advance, or a process that performs a weak filter process may be performed on the area to be processed around the area that is not to be processed.
　さらに、上記説明（図９の例）では、子ノードが複数ある場合はそれぞれの子ノードごとに、その子ノード以下の配信ツリーにおける領域情報に従った映像加工処理をそれぞれ行う場合について記載したが、例えば、映像加工処理の処理負荷を低減するため、すべての子ノードの領域情報を統合した１種類の加工画像のみを生成する加工処理を行うような構成としても良い（ただし、この場合は子ノード以下では表示領域とならないような領域が加工処理されずに送信される場合も起こり得る）。 Furthermore, in the above description (example of FIG. 9), when there are a plurality of child nodes, each of the child nodes has been described as performing video processing processing according to the area information in the distribution tree below the child nodes. For example, in order to reduce the processing load of the video processing process, it may be configured to perform a processing process that generates only one type of processed image obtained by integrating the region information of all the child nodes (in this case, the child node may be used). In the following, an area that does not become a display area may be transmitted without being processed).
　以上のような処理により、アプリケーションレベルマルチキャストを用いた映像情報の配信を行う。 Through the above processing, video information is distributed using application level multicast.
　（Ａ－３）実施形態の効果
　以上のように、本発明の実施形態によれば以下のような効果が得られる。 (A-3) Effects of Embodiment As described above, according to the embodiment of the present invention, the following effects can be obtained.
　アプリケーションレベルマルチキャストを用いた映像情報の配信システムにおいて、配信ツリーを構成する、配信元ノードおよび受信ノードの映像加工部において、配信ツリー上の子ノードやその子孫ノードのいずれにおいても表示領域とならない領域に対してのみ、送信部や再送信部で用いられる映像符号化技術による発生符号量を低減する効果のある映像加工を行うので、各受信ノードで表示される映像については映像加工による悪影響を与えることなく、子ノードに対して送信される送信データや再送信データの情報量を削減することができる。 In the video information distribution system using application level multicast, in the video processing unit of the distribution source node and the reception node constituting the distribution tree, an area that does not become a display area in any of the child node and its descendant nodes on the distribution tree Video processing that is effective in reducing the amount of code generated by the video encoding technique used in the transmission unit and the retransmission unit, the video displayed at each receiving node is adversely affected by the video processing. Therefore, it is possible to reduce the information amount of transmission data and retransmission data transmitted to the child node.
　また、ネットワーク帯域に余裕ができるため、より高画質の映像符号化を施すことが可能となり、各受信ノードで出力される表示領域の映像データを高画質化することが可能となる。 Also, since there is room in the network bandwidth, it is possible to perform video encoding with higher image quality, and it is possible to improve the image quality of the video data in the display area output from each receiving node.
　（Ｂ）他の実施形態
　本発明は上記実施形態に限定されず、その他のさまざまな映像配信システムに利用可能である。 (B) Other Embodiments The present invention is not limited to the above-described embodiments, and can be used for various other video distribution systems.
　例えば、映像や音声データ以外に文書やプレゼンテーション等の情報も同時に配信するなど様々なデータ配信に利用可能である。 For example, it can be used for various data distribution such as simultaneously distributing information such as documents and presentations in addition to video and audio data.
　本発明は、上記構成を有するようなシステム、装置として構成する場合や、上記システムを構成する各装置の処理を実現するようなプログラムとしても実施可能である。 The present invention can be implemented as a system or device having the above-described configuration, or as a program for realizing the processing of each device constituting the system.
　例えば、配信元ノードや受信ノードの機能を実現するようなダウンロード可能なプログラム（例えばＪａｖａＳｃｒｉｐｔ（登録商標）プログラム）をサーバー上（Ｗｅｂサーバーや映像配信管理サーバー）に配置し、ＰＣなどにダウンロードして（Ｗｅｂブラウザにダウンロードして）システムを構成するようなサーバー装置（サーバープログラム）として構成することも可能である。 For example, a downloadable program (for example, a JavaScript (registered trademark) program) that realizes the functions of a distribution source node and a reception node is arranged on a server (Web server or video distribution management server) and downloaded to a PC or the like. It can also be configured as a server device (server program) that configures the system (downloaded to a Web browser).
　１…映像配信システム、１０…映像配信管理サーバー、
　２０…配信元ノード、２０３…映像加工部、２０４…送信部、２０５…領域情報受信部、２０６…領域情報管理部、
　３０（３０－１～３０－５）…受信ノード、３０１…受信部、３０２…表示領域抽出部、３０３…映像加工部、３０４…再送信部、３０５…領域情報受信部、３０６…領域情報管理部、３０７…領域情報送信部。 1 ... Video distribution system, 10 ... Video distribution management server,
20 ... Distribution source node, 203 ... Video processing unit, 204 ... Transmission unit, 205 ... Area information reception unit, 206 ... Area information management unit,
30 (30-1 to 30-5) ... receiving node, 301 ... receiving unit, 302 ... display area extracting unit, 303 ... video processing unit, 304 ... retransmitting unit, 305 ... area information receiving unit, 306 ... area information management Part, 307... Area information transmission part.

Claims (16)

  1. 　ツリー構成の配信経路で映像データをマルチキャスト配信する通信装置において、
    　前記配信経路の下位に位置している１又は複数の下位通信装置のそれぞれの表示領域情報を含む領域情報を受信する領域情報受信部と、
    　前記領域情報受信部により受信された各下位通信装置の表示領域情報を含む領域情報と、自装置の表示領域情報とを管理する領域情報管理部と、
    　前記領域情報管理部で管理される前記各下位通信装置の表示領域情報を含む領域情報と、自装置の表示領域情報とを含む領域情報を、前記配信経路の上位に位置している上位通信装置に送信する領域情報送信部と、
    　前記上位通信装置から映像データを受信する受信部と、
    　自装置の表示領域情報に基づいて、受信した映像データから表示領域を抽出して表示データを出力する表示領域抽出部と、
    　前記領域情報管理部に管理されている各下位通信装置の表示領域情報を含む領域情報に基づいて、前記映像データを映像加工する映像加工部と、
    　前記映像加工部により映像加工された加工後映像データを送信する再送信部と
    　を備えることを特徴とする通信装置。     In a communication device that multicasts video data via a tree-structured distribution path,
    An area information receiving unit that receives area information including display area information of each of the one or more lower-level communication devices located in the lower level of the distribution path;
    A region information management unit that manages the region information including the display region information of each lower-level communication device received by the region information receiving unit, and the display region information of the own device;
    Upper-layer communication device in which region information including display region information of each lower-level communication device managed by the region-information management unit and region information including display region information of its own device is positioned above the distribution route An area information transmitter to transmit to
    A receiver for receiving video data from the higher-level communication device;
    A display area extraction unit that extracts a display area from received video data and outputs display data based on the display area information of the device itself;
    A video processing unit that processes the video data based on region information including display region information of each lower-level communication device managed by the region information management unit;
    A re-transmission unit that transmits post-processed video data processed by the video processing unit.
  2. 　前記上位通信装置から受信した映像データが、前記上位通信装置により映像加工された加工後映像データであり、
    　前記映像加工部が、前記上位通信装置から受信した前記加工後映像データのうち、当該通信装置で初めて加工対象となる領域について、前記各下位通信装置の表示領域以外の領域のみを映像加工する
    　ことを特徴とする請求項１に記載の通信装置。     The video data received from the host communication device is processed video data processed by the host communication device,
    The video processing unit processes only the area other than the display area of each of the lower-level communication devices in the processed video data received from the higher-level communication device for an area to be processed for the first time in the communication device. The communication apparatus according to claim 1.
  3. 　直接通信する複数の下位通信装置がある場合に、
    　前記映像加工部が、前記直接通信する各下位通信装置の表示領域情報を含む領域情報に基づいて、映像加工領域を統合して映像加工し、
    　前記再送信部が、前記映像加工部により映像加工領域を統合した加工後映像データを、前記直接通信する各下位通信装置に送信する
    　ことを特徴とする請求項１に記載の通信装置。     When there are multiple subordinate communication devices that communicate directly,
    The video processing unit integrates video processing areas and processes video based on area information including display area information of each lower-level communication device that communicates directly,
    The communication apparatus according to claim 1, wherein the re-transmission unit transmits post-processed video data obtained by integrating video processing regions by the video processing unit to each lower-level communication device that performs direct communication.
  4. 　ツリー構成の配信経路で映像データをマルチキャスト配信する配信元の通信装置において、
    　前記配信経路の下位に位置している１又は複数の下位通信装置のそれぞれの表示領域情報を含む領域情報を受信する領域情報受信部と、
    　前記領域情報受信部により受信された各下位通信装置の表示領域情報を含む領域情報を管理する領域情報管理部と、
    　前記領域情報管理部に管理されている各下位通信装置の表示領域情報を含む領域情報に基づいて、配信する映像データを映像加工する映像加工部と、
    　前記映像加工部により映像加工された加工後映像データを送信する送信部と
    　を備えることを特徴とする通信装置。     In the communication device of the distribution source that multicasts the video data via the distribution path of the tree configuration,
    An area information receiving unit that receives area information including display area information of each of the one or more lower-level communication devices located in the lower level of the distribution path;
    An area information management unit for managing area information including display area information of each lower-level communication device received by the area information receiving unit;
    A video processing unit that processes video data to be distributed based on region information including display region information of each lower-level communication device managed by the region information management unit;
    A communication unit comprising: a transmission unit configured to transmit post-processed video data processed by the video processing unit.
  5. 　前記映像加工部が、前記各下位通信装置の表示領域情報を含む領域情報に基づいて、前記下位通信装置のいずれにおいても表示領域とならない領域に対して、映像符号化技術の符号発生量を低減する映像加工処理を行なうことを特徴とする請求項１に記載の通信装置。 Based on the area information including the display area information of each of the lower-level communication devices, the video processing unit reduces the amount of code generated by the video encoding technique for an area that is not a display area in any of the lower-level communication apparatuses. The communication apparatus according to claim 1, wherein an image processing process is performed.
  6. 　前記映像加工部が、映像加工処理の対象とする領域に対して塗りつぶし処理を行うことを特徴とする請求項５に記載の通信装置。 6. The communication apparatus according to claim 5, wherein the video processing unit performs a fill process on an area to be processed.
  7. 　前記映像加工部が、映像加工処理の対象となる領域に対して低域通過画像フィルタ処理を行うことを特徴とする請求項５に記載の通信装置。 6. The communication apparatus according to claim 5, wherein the video processing unit performs low-pass image filter processing on a region to be subjected to video processing processing.
  8. 　前記映像加工部が、表示領域が動的に変化する場合に、映像加工処理の対象とならない領域を予め所定の範囲にわたって拡大し、拡大された領域以外の領域に対して映像加工処理を行うことを特徴とする請求項５に記載の通信装置。 When the display area dynamically changes, the video processing unit expands a region that is not a target of video processing over a predetermined range in advance, and performs video processing on a region other than the enlarged region The communication device according to claim 5.
  9. 　前記領域情報送信部が、前記上位通信装置に、映像加工処理の対象とならない領域を拡大した領域を含む領域情報を送信し、
    　前記映像加工部が、前記下位通信装置からの映像加工処理の対象とならない領域を拡大した領域を含む領域情報に基づいて、拡大した領域以外の領域に対して映像加工処理を行う
    　ことを特徴とする請求項５に記載の通信装置。     The area information transmission unit transmits area information including an area obtained by enlarging an area that is not a target of video processing processing to the host communication device,
    The video processing unit performs video processing on an area other than the enlarged area based on area information including an area obtained by enlarging an area that is not a target of video processing from the lower-level communication device. The communication device according to claim 5.
  10. 　前記領域情報管理部が、前記各下位通信装置の表示領域情報を含む領域情報と前記自装置の表示領域情報とに基づいて、包含関係にある領域を統合した領域情報を形成することを特徴とする請求項１に記載の通信装置。 The region information management unit forms region information that integrates regions in an inclusive relationship based on region information including display region information of each lower-level communication device and display region information of the own device. The communication device according to claim 1.
  11. 　前記領域情報管理部が、前記各下位通信装置の表示領域情報を含む領域情報と前記自装置の表示領域情報とに基づいて、重複する領域について当該重複領域を包含する別の大領域に統合した領域情報を形成することを特徴とする請求項１に記載の通信装置。 Based on the area information including the display area information of each of the lower communication devices and the display area information of the own device, the area information management unit has integrated the overlapping areas into another large area including the overlapping area. The communication apparatus according to claim 1, wherein area information is formed.
  12. 　ツリー構成の配信経路で映像データをマルチキャスト配信する通信方法において、
    　領域情報受信部が、前記配信経路の下位に位置している１又は複数の下位通信装置のそれぞれの表示領域情報を含む領域情報を受信し、
    　領域情報管理部が、前記領域情報受信部により受信された各下位通信装置の表示領域情報を含む領域情報と、自装置の表示領域情報とを管理し、
    　領域情報送信部が、前記領域情報管理部で管理される前記各下位通信装置の表示領域情報を含む領域情報と、自装置の表示領域情報とを含む領域情報を、前記配信経路の上位に位置している上位通信装置に送信し、
    　受信部が、前記上位通信装置から映像データを受信し、
    　表示領域抽出部が、自装置の表示領域情報に基づいて、受信した映像データから表示領域を抽出して表示データを出力し、
    　映像加工部が、前記領域情報管理部に管理されている各下位通信装置の表示領域情報を含む領域情報に基づいて、前記映像データを映像加工し、
    　再送信部が、前記映像加工部により映像加工された加工後映像データを送信する
    　ことを特徴とする通信方法。     In a communication method for multicast distribution of video data via a tree-structured distribution route,
    An area information receiving unit receives area information including display area information of each of one or a plurality of lower-level communication devices located at a lower level of the distribution path;
    The area information management unit manages the area information including the display area information of each lower-level communication device received by the area information receiving unit, and the display area information of the own device,
    The area information transmission unit positions area information including the display area information of each of the lower-level communication devices managed by the area information management unit and the display area information of the own apparatus at the upper level of the distribution route. To the higher-level communication device
    The receiving unit receives the video data from the host communication device,
    The display area extraction unit extracts the display area from the received video data based on the display area information of the own device, and outputs the display data.
    A video processing unit processes the video data based on region information including display region information of each lower-level communication device managed by the region information management unit,
    The retransmitting unit transmits post-processed video data processed by the video processing unit.
  13. 　ツリー構成の配信経路で映像データをマルチキャスト配信する配信元の通信方法において、
    　領域情報受信部が、前記配信経路の下位に位置している１又は複数の下位通信装置のそれぞれの表示領域情報を含む領域情報を受信し、
    　領域情報管理部が、前記領域情報受信部により受信された各下位通信装置の表示領域情報を含む領域情報を管理し、
    　映像加工部が、前記領域情報管理部に管理されている各下位通信装置の表示領域情報を含む領域情報に基づいて、配信する映像データを映像加工し、
    　送信部が、前記映像加工部により映像加工された加工後映像データを送信する
    　ことを特徴とする通信方法。     In the communication method of the distribution source that multicasts the video data through the distribution path of the tree configuration,
    An area information receiving unit receives area information including display area information of each of one or a plurality of lower-level communication devices located at a lower level of the distribution path;
    The area information management unit manages area information including display area information of each lower-level communication device received by the area information receiving unit,
    A video processing unit processes video data to be distributed based on region information including display region information of each lower-level communication device managed by the region information management unit,
    The communication method, wherein the transmission unit transmits the processed video data processed by the video processing unit.
  14. 　ツリー構成の配信経路で映像データをマルチキャスト配信する通信プログラムにおいて、
    　コンピュータを、
    　　前記配信経路の下位に位置している１又は複数の下位通信装置のそれぞれの表示領域情報を含む領域情報を受信する領域情報受信部と、
    　　前記領域情報受信部により受信された各下位通信装置の表示領域情報を含む領域情報と、自装置の表示領域情報とを管理する領域情報管理部と、
    　　前記領域情報管理部で管理される前記各下位通信装置の表示領域情報を含む領域情報と、自装置の表示領域情報とを含む領域情報を、前記配信経路の上位に位置している上位通信装置に送信する領域情報送信部と、
    　　前記上位通信装置から映像データを受信する受信部と、
    　　自装置の表示領域情報に基づいて、受信した映像データから表示領域を抽出して表示データを出力する表示領域抽出部と、
    　　前記領域情報管理部に管理されている各下位通信装置の表示領域情報を含む領域情報に基づいて、前記映像データを映像加工する映像加工部と、
    　　前記映像加工部により映像加工された加工後映像データを送信する再送信部と
    　して機能させることを特徴とする通信プログラムを記録したコンピュータに読み取り可能な記録媒体。     In a communication program that multicasts video data via a tree-structured distribution route,
    Computer
    An area information receiving unit that receives area information including display area information of each of the one or more lower-level communication devices located in the lower level of the distribution path;
    A region information management unit that manages the region information including the display region information of each lower-level communication device received by the region information receiving unit, and the display region information of the own device;
    Upper-layer communication device in which region information including display region information of each lower-level communication device managed by the region-information management unit and region information including display region information of its own device is positioned above the distribution route An area information transmitter to transmit to
    A receiver for receiving video data from the higher-level communication device;
    A display area extraction unit that extracts a display area from received video data and outputs display data based on the display area information of the device itself;
    A video processing unit that processes the video data based on region information including display region information of each lower-level communication device managed by the region information management unit;
    A computer-readable recording medium on which a communication program is recorded, which functions as a re-transmission unit for transmitting processed video data processed by the video processing unit.
  15. 　ツリー構成の配信経路で映像データをマルチキャスト配信する配信元の通信プログラムにおいて、
    　コンピュータを、
    　　前記配信経路の下位に位置している１又は複数の下位通信装置のそれぞれの表示領域情報を含む領域情報を受信する領域情報受信部と、
    　　前記領域情報受信部により受信された各下位通信装置の表示領域情報を含む領域情報を管理する領域情報管理部と、
    　　前記領域情報管理部に管理されている各下位通信装置の表示領域情報を含む領域情報に基づいて、配信する映像データを映像加工する映像加工部と、
    　　前記映像加工部により映像加工された加工後映像データを送信する送信部と
    　して機能させることを特徴とする通信プログラムを記録したコンピュータに読み取り可能な記録媒体。     In the communication program of the distribution source that multicasts the video data via the distribution path of the tree configuration,
    Computer
    An area information receiving unit that receives area information including display area information of each of the one or more lower-level communication devices located in the lower level of the distribution path;
    An area information management unit for managing area information including display area information of each lower-level communication device received by the area information receiving unit;
    A video processing unit that processes video data to be distributed based on region information including display region information of each lower-level communication device managed by the region information management unit;
    A computer-readable recording medium having a communication program recorded thereon, which functions as a transmission unit that transmits processed video data processed by the video processing unit.
  16. 　ツリー構成の配信経路で映像データをマルチキャスト配信する通信装置を管理する管理装置において、
    　配信元の通信装置と映像データを受信する１又は複数の通信装置との間の配信経路を決定し、当該配信経路を構成する各通信装置に配信パスの通信路の設定を指示する通信路設定部と、
    　前記各通信装置間の表示領域情報を含む領域情報の送受信を中継する中継部と、
    　前記各通信装置の表示領域情報を含む領域情報を管理する領域情報管理部と
    　を備え、
    　前記各通信装置から送信される領域情報は、当該通信装置の表示領域情報を含む領域情報のみとし、
    　前記領域情報管理部が、前記各通信装置に通知する子ノード以下の下位の通信装置の表示領域情報を含む領域情報を通知する
    　ことを特徴とする管理装置。     In a management device that manages a communication device that multicasts video data via a tree-structured distribution path,
    Communication path setting for determining a distribution path between a distribution source communication apparatus and one or a plurality of communication apparatuses that receive video data, and instructing each communication apparatus constituting the distribution path to set the communication path of the distribution path And
    A relay unit that relays transmission / reception of area information including display area information between the communication devices;
    An area information management unit that manages area information including display area information of each communication device;
    The region information transmitted from each communication device is only region information including the display region information of the communication device,
    The management apparatus characterized in that the area information management unit notifies area information including display area information of lower-level communication apparatuses below a child node to be notified to each communication apparatus.
PCT/JP2018/041044 2018-02-27 2018-11-05 Communication device, communication method, recording medium, and management device WO2019167348A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2018-033441 2018-02-27
JP2018033441A JP7114927B2 (en) 2018-02-27 2018-02-27 Communication device, communication method and communication program, management device

Publications (1)

Publication Number Publication Date
WO2019167348A1 true WO2019167348A1 (en) 2019-09-06

Family

ID=67804981

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2018/041044 WO2019167348A1 (en) 2018-02-27 2018-11-05 Communication device, communication method, recording medium, and management device

Country Status (2)

Country Link
JP (1) JP7114927B2 (en)
WO (1) WO2019167348A1 (en)

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
AHMADI, H. ET AL.: "Adaptive Multicast Streaming of Virtual Reality Content to Mobile Users", PROCEEDINGS OF THE THEMATIC WORKSHOPS, vol. 17, 27 October 2017 (2017-10-27), pages 170 - 178, XP055634624, ISBN: 978-1-4503-5416-5 *
YOKOZEKI, D. ET AL.: "Image transmission over ATM using a hierarchical coding scheme adaptive to arbitrary parts of an image", PROCEEDINGS OF THE 12TH PICTURE CODING SYMPOSIUM OF JAPAN (PCSJ97, 6 October 1997 (1997-10-06), pages 39 - 40 *

Also Published As

Publication number Publication date
JP7114927B2 (en) 2022-08-09
JP2019149700A (en) 2019-09-05

Similar Documents

Publication Publication Date Title
US8319816B1 (en) Methods, systems and program products for efficient communication of data between conference servers
JP5255654B2 (en) Multi-directional peer-to-peer media streaming
US9571793B2 (en) Methods, systems and program products for managing resource distribution among a plurality of server applications
CN101124566A (en) End-to-end publish/subscribe intermediate system structure
US20140071223A1 (en) Peer-to-peer video conferencing
CN109314760A (en) Video conferencing system, application server, equipment, video-meeting method and the non-transitory computer-readable medium for storing videoconference program
WO2015196361A1 (en) Multicast method, apparatus, and system for software defined network
CN101141397A (en) Single broadcast to multicast switching method
Civanlar et al. Peer-to-peer multipoint videoconferencing on the Internet
Wu et al. Towards multi-site collaboration in tele-immersive environments
Han et al. Design of multi-party meeting system for interactive collaboration
WO2019167348A1 (en) Communication device, communication method, recording medium, and management device
JP5399276B2 (en) Content distribution system and method and program
JP3759527B2 (en) Multicast data communication system and method, client-side gateway, server-side gateway, computer program, and recording medium recording the computer program
CN115529433A (en) Cross-domain polling method and device for cascading video conference and electronic equipment
JP3693978B2 (en) Multicast data communication method, multicast data communication system, relay device, relay method, relay program, and medium recording the relay program
US20140176666A1 (en) Method, computer program and device for managing media stream in video conferencing
KR20080065230A (en) System and method for communication in groups using a lot of data transferring server
JP5350084B2 (en) Data distribution apparatus, data distribution method, and program
Balikhina et al. Distributed virtual environments-An active future?
JP6690291B2 (en) Information distribution system, information distribution device, information distribution program, and information distribution method
JP7314545B2 (en) Communication node, communication program, communication method, distribution configuration management device, and information distribution system
JP7302289B2 (en) Communication device, communication control device, data delivery system, communication program and communication control program
JP2005295142A (en) Server and method for video information management
KR101535057B1 (en) Display system and method for advanced collaboration environment

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18907518

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18907518

Country of ref document: EP

Kind code of ref document: A1