JP2009017289A - Scheduling of negotiable and optimumly resource-dispersed multimedia service - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an opportunity driven and peer-to-peer negotiation type scheduling method based on an AV token in multimedia service. <P>SOLUTION: By using the AV token and a schedule information table, negotiation and scheduling of multimedia service for dispersed services are performed in a peer-to-peer communication mode. Also for renegotiation because of unusability of service, the AV token is used. An AV content is streamed only when it is requested by a service provider, so that just-in-time data transmission saving resources in service provision and a service receiver can be achieved. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates generally to multimedia service scheduling, and more particularly to opportunity-driven peer-to-peer negotiated scheduling based on AV tokens. More specifically, information on disparate service providing devices is dynamically exchanged with service consuming devices for optimized multimedia service scheduling. Related systems and methods are described.

  Streaming multimedia content between devices is becoming a common scenario for a variety of reasons, including multimedia content processing such as encryption, playback, and remote transcoding, distributed and collaborative processing. Current networking technologies enable communication capabilities in a variety of disparate devices for service consumption and service provision. Consumable AV services from service providing devices include remote transcoding, storage, transrating, and playback. Consumer electronic (CE) devices capable of handling digital content via wired or wireless connectivity can seamlessly provide or consume multimedia services between them.

  However, due to the nature of multimedia devices that are portable, ad hoc in terms of network presence, and heterogeneous in terms of capabilities, interfaces, etc., devices on the service provider side and devices on the service consumer side in a network environment It is difficult to perform service scheduling with a service provider without compromising service requirements such as service completion time and service quality. In order to strictly observe the service completion time, the AV service agreed in advance needs to be completed on time by the service providing apparatus. In addition, sample rate, number of bits per sample, bit rate, number of frames per second, height, width, aspect ratio, resolution, color depth, variable bit rate (VBR) / constant bit rate (CBR) The quality of AV services including features such as) must not be impaired. The opportunity-driven and negotiated service scheduling mechanism by both devices (service supply and consumption devices) is not available, leading to insufficient use of available multimedia resources in the network. . Opportunity-driven mechanisms search for available resources in the network and, even when service interruptions are observed, between service providing peers without compromising pre-agreed service completion time and service quality Negotiate task distribution. This is accomplished by utilizing AV token status as described herein during the process of negotiation between the service consumer and service provider.

  Network multimedia services (NMS) have different needs with respect to service time and service resources for devices connected to the network. These needs need to be based on service times such that a particular multimedia task is completed by a user application (immediately or delayed for a specified time) for a specified AV quality. Each task may have different resource configuration (eg, buffer size and CPU processing resources) requirements that need to be well coordinated. Generalized service configuration leads to streaming AV content without knowing the state of the service providing device, causing resource waste and inefficiency in local, intermediate and remote devices including interconnected links . Therefore, AV processing that is well negotiated and just-in-time streaming is important.

  In other areas, a multimedia service provider may receive multiple requests for multimedia service scheduling. In this case, efficient service scheduling based on the requirements of the service requesting device or taking into account the priority of the service requesting device is important for maximum user application satisfaction. In addition, the interconnect link between distributed multimedia services enables distributed multimedia services. Two or more devices provide distributed resources for the needs of a single user application that can be serviced in a collaborative environment by a service providing device. Resource allocation based on actual real-time usage is preferred for distributed multimedia services such as continuous playback. This requires an opportunity-driven scheduling mechanism that can be negotiated.

US Patent Application Publication No. 2003 / 0072563A1 US Pat. No. 6,771,626 International Publication No. WO2006 / 059343 International Publication No. WO2000 / 0667124 US Pat. No. 6,205,150 US Pat. No. 6,917,984 US Pat. No. 6,728,270 US Patent Application Publication No. 2005/0108414 International Publication No. WO2001 / 047249 US Patent Application Publication No. 2005/00005025

  The present invention relates generally to multimedia service scheduling, and more particularly to opportunity-driven peer-to-peer negotiated scheduling based on AV tokens. More specifically, information on disparate service providing devices is dynamically exchanged with service consuming devices for optimized multimedia service scheduling. Related systems and methods are described.

  Coarse multimedia service scheduling for distributed AV processing tasks faces the difficulty of rescheduling services in the event of service failure or unavailability. Service unavailability requires delivery of tasks to other devices, sharing of service load by some devices, or renegotiation for new service completion times. Rescheduling performed only when the service consumer notices a service failure is not timely for real-time AV content playback. Rescheduling also requires retransmission of all or part of the AV stream from the service consumer to the service provider without knowing the exact multimedia service completion point / range. Therefore, a better scheduling mechanism is needed to deal with the heterogeneous nature of service devices.

According to the present invention, in a multimedia service device environment, a method for scheduling a peer-to-peer negotiated multimedia service based on AV tokens, comprising:
a. Before actual multimedia / AV data processing is performed, an AV token for multimedia / AV processing service negotiation is generated, and the actual multimedia / AV data processing is performed based on the negotiated token status. Dynamically reassign
b. For optimal multimedia service negotiation, a dynamic and static service information table is generated by the service providing apparatus, and the service information table is serviced by the service providing apparatus prior to the multimedia service negotiation. Dynamically exchange to equipment,
c. Prior to the start of the negotiation, the multimedia service opportunity is identified by the service consumer based on the dynamic and static service information from the service provider,
d. Selecting, reserving, negotiating and managing multimedia services provided by a service providing device by the service consuming device;
e. Transmitting the streamed multimedia content from the service consuming device to the service providing device prior to the start of the multimedia service.

  According to the present invention, since actual multimedia / AV data processing is dynamically reassigned based on the negotiated token status, service failure does not occur when AV content is played back.

  Embodiments of the invention are supplemented by the drawings to illustrate the invention with respect to the elements of the specific drawings. Repeated cases are covered by specific cases that are covered by specific drawing elements.

  The present invention relates generally to multimedia service scheduling, and more particularly to opportunity-driven peer-to-peer negotiated scheduling based on AV tokens. More specifically, information on disparate service providing devices is dynamically exchanged with service consuming devices for optimized multimedia service scheduling. Related systems and methods are described.

  FIG. 1 shows service consumers, CE devices A and B (104 and 112) with multimedia content (106 and 114) and AV tokens (120 and 122), which are interconnected links (102). ) To the service providing device, that is, the CE devices X, Y, and Z (124, 130, and 136). Service consuming devices (104 and 112) have multimedia content (106 and 114) local to their respective devices, which are represented in groups of AV tokens (120 and 122) that have multimedia specific information. Has been. Further, the service consuming device has a service information table (SIT) (108 and 116) and an AV token table (110 and 118) in each device. The service information tables (108 and 116) contain information about service providing devices, while the AV token tables (110 and 118) contain scheduled AV token information. Multimedia content for all or partial content of the service consuming device is represented in terms of AV tokens and negotiated to one or more service providing devices for scheduling of multimedia services. While the successfully negotiated AV token is queued by the service providing device to be serviced, the service consuming device stores it locally on each device for service status checking. The selection of a service providing apparatus for AV token negotiation is performed based on the service information tables (108 and 116). The AV token table (110 and 118) of the service consuming device manages new tasks added by the user application on its side for service scheduling by negotiation with the service providing device. On the other hand, the AV token table and schedule (128, 134, and 140) of the service providing device manages the service negotiation request from the service consuming device and the service scheduling of the accepted AV service.

  Based on the local schedule, when the time scheduled by the service providing device (124 or 130 or 136) is reached, the multimedia content is transmitted from the service consuming device to the service providing device for the multimedia service and required. Optimize resource usage. Temporary content (126, 132, and 138) received from the service consumer or serviced by the service provider and streamed to the service consumer is sent to the negotiated service provider (124, 130, and 136). Saved. Successful negotiated and accepted multimedia services are locally scheduled and stored in the AV token table and schedule (128, 134, and 140) of the service provider. The AV token (120 and 122) initially constructed by the service consuming devices (104 and 112) is the AV token of the service providing device (124, 130, and 136) after successful multimedia service negotiation. Stored and scheduled in tables and schedules (128, 134, and 140). These AV tokens (142, 144, and 146) are distributed to various service providing devices based on the multimedia service negotiation (124, 130, and 136).

  FIG. 2 illustrates the process flow of the configuration in the service discovery and service information table of the invention. First, the service consuming devices are Universal Plug and Play (UPnP) (Simple Service Discovery Protocol (SSDP)), Web Service (W3C), Service Locating Protocol (SLP), etc. Using an existing service discovery protocol, an available service providing device is discovered (202). The service consumer selects one or more required service providing devices according to the service category (204). Next, the service consuming device registers the multimedia service for receiving the multimedia service related information to the selected service providing device using the existing service discovery protocol as described above. (206). The service providing device accepts service registration of the service consuming device (208), and transmits static multimedia service information (210) and dynamic multimedia service information (212) to the service consuming device. The received information is listed (214) in the service information table (108a and 108b) by the service consumer.

  Static and dynamic information transmitted statically or dynamically by the service providing device is presented in FIG. 3 in the form of a service information table (334) maintained by the service consuming device (108). A service provider list (302) includes a list of all registered multimedia service providers in the network. Each service providing device is associated with service information stored in the form of a storage location pointer (304) for a detailed information structure (336). Each service providing device is identified by a service providing device ID (306, 308, and 310) that is unique among them. This ID may be an IP address or service ID for uniquely identifying each possible / available service, or a combination of this information. A storage location pointer field (312) for each service providing device holds information about the location of the structure where detailed static information (314) and dynamic information (316) is stored.

  The static information includes supported media types (318), supported media services (320), and multimedia service details (322). This service information is used in an AV token for AV service negotiation that takes place between the service consumer and service provider for the best match. Supported media types (318) cover video and audio formats such as MPEG2-PS, MPEG2-TS, MPEG4-Part12, MP4-Part12, M4SYS-Part1, 3GPP, 3GPP2, and so on. Media Service (320) indicates the type of media service such as direct playback, transcoding, mixing, splitting, filtering, AV synchronization, video scaling, audio sampling, audio / video format conversion, etc. ing. Multimedia service details (322) include bit rate, sample rate, bits per sample, CBR / VBR, mono / stereo, channel frames per second, height, width, aspect ratio, resolution, Covers specific audio and video parameters including color depth, VBR / CBR, display aspect, etc.

  Dynamic information (316) includes AV frame / second processing speed (324), network path bandwidth (326) or access speed to the device, current status of available or unavailable device (328), And latency (330). The latency covers three different latencies: network latency, processing latency, and buffering latency in the service providing apparatus. The service providing apparatus collects these three latencies and transmits them dynamically or periodically to the service consuming apparatus. Network latency is the time required to stream data from a service consumer to a service provider and to the intended service consumer. This data is transmitted to the service consuming device for the first time when the service consuming device registers (206) with a specific service providing device. Processing latency is an estimated time for processing multimedia data. This data is calculated from the previous service reservation with the service providing apparatus. If the service providing device is new to the network, the estimated time for a particular multimedia data length is notified. Finally, the buffering latency is a time for transmitting and receiving the buffer latency on the service providing apparatus side, and can be calculated in advance by the service providing apparatus having dummy data for the first time. Periodic updates of these three latencies based on the latest service provided are calculated and notified to the registered service consumer. By using the contents of the service information table before participating in the negotiation process, it is possible to select an optimized service providing apparatus or multimedia service.

  FIG. 4 shows the overall process flow of multimedia service negotiation based on AV tokens. A service consumer receives a request from a user application for a network multimedia service (NMS) (402). Thereafter, the service consuming device constructs a request AV token (FIG. 7) and sends the request AV token to one or more service providing devices selected for multimedia service negotiation. A specific service providing apparatus is selected from the service information table (108) on the service consuming apparatus side (404). On the other hand, the service providing apparatus receives the requested AV token and decides to accept the negotiation request based on the local AV token table and the AV token queuing threshold (406). After the determination, the service providing device responds to the service consuming device with a response AV token regarding the status of the negotiation request. Depending on the status of the negotiation for the response AV token (410), the service consuming device makes a decision on the next operation. If the initial negotiation attempt is unsuccessful, the service consumer continues the negotiation process until it finds a suitable service provider (404). By identifying the success of the negotiated multimedia service, once the multimedia data request (412) is received from the service providing device, the multimedia to the service providing device by the service consuming device based on the AV token・ Data streaming is required. This ensures that multimedia data is only streamed when the service provider is ready for processing.

  FIG. 5 shows an AV token table (534) used by both the service consuming device and the service providing device to maintain AV token related information for multimedia service negotiation and scheduling. The AV token table includes an AV token ID (502) and a token storage location (510). Each service providing device or service consuming device stores in the table one or more AV token IDs (504, 506, and 508) that are uniquely configured by the service consuming device when generating a request AV token negotiation message. Can have. Each AV token ID has their respective token structure (536) that is uniquely pointed to by a storage location pointer (512a, 512b, and 512c).

  Each storage position pointer (510) specifies the position of the detailed information structure (536) of the AV token. Each AV token (514) has a plurality of fields representing multimedia content information. The service providing device ID or service consuming device ID (516) represents a unique ID of each service providing device or service consuming device. This ID may be a combination of an IP address, a service ID, and an application ID. This is followed by the storage location (518) of the AV capsule, where the storage location (518) is the storage location of the source of the complete or partial multimedia data as specified by the AV token, and the multimedia Contains the destination storage location where the data should be streamed. The source and destination addresses may be multimedia data area specific information (a byte area based on the http protocol) that can be used to access or write content together with the unique address of the storage location of the service device. The size (in bytes) of the multimedia data is stored in the next field (520). Specific multimedia data is also represented in the next field (522) in terms of presentation duration. Media type (524), media service (526), and service details (528) as shown in fields 318, 320, and 322 of static information (314) in FIG. 3 are also present in the AV token. is doing. The service start time (530) indicates the actual time (Network Time Protocol (NTP) based clock time) that the multimedia data is scheduled for the multimedia service. Finally, the estimated service end time (532) indicates the actual time at which the multimedia service should end. This time can also be expressed in terms of the estimated duration for multimedia processing. The sum of the duration and the service start time (530) gives the actual time.

  The steps of generating the AV token table in the service requesting device and the service providing device are different as shown in FIGS.

  Creation of the AV token table at the service consumer is triggered by an NMS request from the user application (1602). The service consumer then analyzes the details of the NMS request from the application. The application uses request information (1604) as the following fields: media type (318), media service (320), multimedia service details (322), AV capsule storage location (518), AV capsule As described by size (520), capsule length (522), service start time (530), estimated service end time (532), return capsule duration (710), and token status (720), one or Multiple types of information must be provided.

  Analyzing the conditions of the application request, the service consuming device selects the best service providing device based on the NMS request conditions and matching of the static and dynamic service information table (FIG. 3) (1606). The following fields (318, 320, 322, 324, 326, 328, and 330) are used in some cases for service provider selection. An AV token (502) is generated and inserted (1608) into the AV token table (534) at the service consuming device along with the relevant details (514-532) available. The service consumer is triggered (1610) to initiate NMS negotiation with the identified service provider. The status of NMS negotiation for each AV token is updated to the user application (1612).

  On the other hand, the construction of the AV table (534) in the service providing device begins (1702) when the service providing device accepts the service request for scheduling as indicated by 1114. The request AV token is used to extract the required information (fields 702-720) from the message body (1704). If necessary, additional information (512) or missing information (532) is configured (1706) by the service providing device. Finally, an entry to the AV token table (534) of the service providing apparatus is made based on the AV token ID (502) (1708). The AV token is serviced (1710) when a threshold trigger is received, as illustrated by FIG.

  FIG. 6 illustrates the response AV token. The response AV token has an AV token ID (602) field inherited from the request AV token (702). The service providing apparatus ID (604) is supplied by the service providing apparatus, and similarly to 516, uniquely identifies all services provided by the service providing apparatus. This ID represents unique information used to uniquely identify the service provider and the associated multimedia service / process. The token status field (606) represents the token status in the negotiation process between the service consuming device and the service providing device. The status of the negotiation may be successful, negotiated, failed, completed, in progress, changed, or forwarded. The service start time (608) and the estimated service end time are times proposed or accepted by the service providing apparatus after receiving the request from the service consuming apparatus. These two fields, (608) and (610), may be the same if no change is made by the service provider, otherwise the local time for the time when the service provider is newly proposed. Change them on a schedule.

  The steps required to construct a response AV token are shown in FIG. The service providing device accepts a valid service request for service scheduling and triggers the construction of the response AV token as shown at 1114 in FIG. 11 (1902). Next, the service providing apparatus constructs a response AV token having all relevant information / message fields as shown in FIG. 6 (1904). An AV token status that can be successful, negotiated, failed, completed, transferred, in progress, and changed needs to be determined (1906). If the negotiated AV service succeeds or fails as shown at 1116 in FIG. 11, a success or failure status is selected. If the service providing device fails to accept the negotiated condition (fields 708-720), it can respond with an optional condition based on the availability / capability of the service providing device resource. Negotiation is performed by selecting more appropriate fields than the requested conditions of 708-720. For example, the request is accompanied by [..., MPEG2TS, playback, 30 frames / second, ...] [..., media type, media service, service details, ...] If the providing device cannot support this request, but can transcode MPEGTS to MPEGPS for playback at a lower frame rate, the service providing device [MPEGTS, transcoding and playback, 25 frames / sec. ] Can respond. In this case, the negotiated status is used.

  When the request AV token refers to the completed AV token, the completion status is used, while the ongoing status is used for the AV service that is still in progress on the service providing apparatus side. The transfer status is used when the referenced AV token is transferred to another service providing apparatus. The status of the change includes the AV capsule storage location (destination) (706), AV capsule size (708), return capsule time (710), capsule length (712), media type (714), media service (716), and service details (718), and by the service provider to negotiate with the service consumer to change the status of the previously agreed AV service details including token status (emergency level) (720) Can start.

  This is followed by the AV token ID, service provider ID, service start time (1908), and estimated service end time (1910) fields determined by the service provider based on the local AV service resource. A fully configured response AV token is sent to each service consuming device (1912).

  FIG. 7 shows a requested AV token from the service consuming apparatus, and has an AV token ID (702) field that uniquely represents the token as already described with respect to the field 514. The service consuming device ID (704) is prepared by the service consuming device, and uniquely identifies the service consuming device itself and all services of the service consuming device. This ID represents unique information for identifying the service consuming device and the associated multimedia service / process. Fields 708, 712, 714, 716, and 718 have the same description in each field of FIG. 5 for 520, 522, 524, 526, and 528 fields. This information is derived from the user application. The return capsule duration field (710) describes the required return time for AV capsule / multimedia data by the service consumer. Finally, the token status (720) has the status of the negotiation of the AV token, which is the emergency level, negotiate, transfer, check, and change states.

  FIG. 18 shows the steps used to create the requested AV token. Initially, the service consuming device receives a trigger to initiate NMS negotiation (1802). This is typically the support request made by the user application in inserting the AV token into the AV token table of the service consumer (1610) or updating the negotiation status (1612) in case of a failed negotiation, for example. Done later. In this case, the user application can change the conditions (1604) of its own NMS request.

  Next, the service consumer creates a request AV token with all relevant information / message fields as shown in FIG. 7 (1804). The fields of AV token ID (702), service consumption device ID (704), and AV capsule storage location (706) are created by the service consumption device based on local information. Field information for return capsule duration (1808) and capsule length (1810) is obtained from the user application. Similarly, the media type (714), media service (716), and service details (718) fields are created based on user application information if provided and applicable, otherwise It is omitted (1812). The next step is the identification and insertion of AV token status (1814). As already mentioned, the request AV token can have the following states: urgency level, negotiate, transfer, check, and change status.

  The urgency level may be normal, intermediate, or rapid in order of increasing priority with respect to time units (hours / minutes / seconds). Rapid can be interpreted by the service consuming device as AV service is needed within seconds, the middle can be interpreted as needed within minutes, and usually can be interpreted as needed within hours . This provides room for service negotiation and rescheduling on the service providing device side if necessary. An intermediate emergency level is assigned by default. As already described, the negotiate status is used when an AV token is created to negotiate the AV service for the first time, or the AV capsule size (708), return capsule duration (710), capsule length (712) ), Media type (714), media service (716), service details (718), etc. are used when important fields are changed. The transfer status is used when the service consuming device wants to transfer the AV service from the currently scheduled service providing device to another service providing device. The status of the change is used if the currently scheduled task is changed in terms of AV service specifications for 706 fields or other fields that do not affect the return capsule duration (710). The status of the check is used to know the current state of the multimedia task in the service providing device. Finally, the created request AV token is transmitted to the service providing apparatus (1816).

  Specifically, the token status including success, negotiate, failure, completion, transfer, in progress, change, check, and emergency level of the request and response AV tokens, along with the SIT, the service providing device and the service consuming device Enables opportunity-driven AV service negotiations between It is mobile, ad hoc in terms of presence in the network, is heterogeneous in terms of capabilities, interfaces, etc., with service providing equipment in a network environment without compromising service requirements such as service completion time and service quality It is well suited to the nature of multimedia devices that make it difficult to schedule services with service consuming devices.

  A negotiation mechanism based on the AV token of the service consuming device is illustrated in the process flow of FIG. The service consuming device generates a request AV token for each multimedia data / AV capsule based on the given format of FIG. 7 (802). The AV token is created based on the local static and dynamic service information table as shown in FIG. 3 (804). During this process, the best service provider is selected based on static and dynamic service information (806). The best service provider can be determined based on a combination of supported media types, media services, and CPU utilization, or field information. All 336 fields may be for the purpose of selecting the best service provider.

  Next, the service consuming device transmits a request AV token to the selected one or more service providing devices, sets an urgent status as appropriate, and starts a service negotiation process. If the delivery of the AV token, which can be performed based on a reliable transport protocol, is successful, the service consumer registers this AV token in its own local AV token table as shown in FIG. Wait for the corresponding response AV token (808). The response AV token may be for the status of a new negotiation that has been initiated, or for an existing AV token in a service provider such as an AV token change or AV token transfer or AV token status update. (810).

  If the response AV token indicates that the negotiation was successful for the requested condition (814), the service consumer will schedule a schedule in its local AV token table for multimedia data streaming. Update (818). The fields 710, 714, 716, and 718 of the request AV token can be used as conditions that can be negotiated by the service consuming device and the service providing device based on the application requirements. Upon receipt of the multimedia data request message of the service providing device (820), the service consuming device streams the multimedia data to the specific service consuming device. The streaming of multimedia data may be based on a push or pull model by a service consumer or service provider (822).

  If the service provider presents a condition for negotiation that is different from the condition initially presented by the service consumer (812), the negotiation will fail as indicated in the token status (606) of the response AV token (814). sell. In this case, if the new negotiation condition is acceptable, the service consuming device may decide to accept the newly presented condition (816). The service consumer then schedules this AV token (818). Otherwise, the service consuming device starts the negotiation process again (806). For example, [..., current clock (NTP) + 300 seconds, 60 minutes, ..., quick / negotiate] field [..., return capsule time, capsule length, ..., token status] The service request AV token possessed can be received by the service providing apparatus. Based on the current schedule and resources, the service providing device can respond to the response AV token [..., Token status, service [..., negotiate, current clock (NTP) + 1200 seconds, current clock (NTP) + 1300 seconds]. Different conditions such as [start time, estimated service end time] can be presented. The negotiated response AV token states that the task can be completed in 100 seconds, but this is because the service start time is delayed by 600 seconds and the service consumer has a quick emergency level. Reject. A renegotiation with another service providing apparatus is started.

  The service consuming device performs periodic checks on the local AV token table, and scheduled AV tokens after successful negotiation with the service providing device are tracked for timely task completion. Is guaranteed (902). The process flow is shown in FIG. For AV capsule i, service start time (530)> current clock time and when streaming is not being performed, or for AV capsule i (service start time (530) + expected service end time (532))> If it is an acceptable delay time (1006), the service is not running as scheduled (904). Accordingly, the service consuming device transmits a request AV token for specific multimedia data to the service providing device corresponding to the multimedia service status based on the AV token information. The service provider is expected to respond to the service consumer with a token status (606) using the response AV token (FIG. 6). If the task is completed or in progress, no further action is required. On the other hand, the failure status of the response AV token requires resumption of the service negotiation process by the service consumer (912).

FIG. 10 illustrates the time boundary used by the service consumer (1010) to perform service negotiation using the response AV token for the return AV capsule duration (710). The start time (1002) indicates the time at which the first negotiation for the first AV token was performed for content from a list of all or all AV tokens. The presentation time (1012) indicates the time that the serviced / processed multimedia data is used by the service consuming device. The multimedia data needs to be streamed from the service consumer (1010) to the service provider (1016) and then to the service consumer (1014). The service consumption devices (1010) and (1014) may be the same entity or different entities. These two time intervals (1002 and 1012) provide an acceptable service delay (1004) for service negotiation between the service provider (1016) and the service consumer. L ₁ indicates an allowable delay time (1006) for the first multimedia data processing in which one multimedia data / AV capsule is provided from a group of related multimedia contents by the service providing apparatus. Yes. Subsequent multimedia data N in the group may have an acceptable delay time (1008) of L _{N in the} worst case.

  The return capsule duration (710) with an acceptable delay time for the AV token is calculated using the method described above. First, the presentation time of multimedia data / AV content is determined. Next, the number of AV tokens for a particular multimedia content having a small portion of AV data is calculated. For all AV tokens of multimedia content to be negotiated, the return capsule duration must be less than the combined latency of the specific service provider and presentation time.

  Latency (1016) is calculated by subtracting start time (1002) from presentation time (1012). The earliest presentation time can be calculated (1018) by adding the start time to the latency. However, a scheduling is proposed that has a time margin for the final multimedia data compared to the actual presentation time. The time negotiated by the service consumer in the requested AV token must take into account the method described above for calculating the time. For example, when creating a requested AV token, the service information table (334) may need to know a specific service to know the latency information (332) before estimating the required presentation time or return capsule time (710). Reference is made to the providing device (336). The estimated service end time (610) of the response AV token must be within or less than the required time (710). This time calculation can be performed with the help of FIG. In addition, latency information (332) from the service information table (336) is also used to minimize the available allowable service delay (1104). The overall latency is calculated (1020) by adding the network latency, processing latency, and buffering latency. Network latency is the time required to stream data from a service consumer to a service provider and to the intended service consumer. Processing latency is an estimated time to process multimedia data in the negotiation. Finally, the buffering latency is transmission and reception buffer latency on the service providing apparatus side as shown in FIG.

  FIG. 11 shows a process flow in the task of the service providing apparatus. The service providing apparatus receives a request AV token from the service consuming apparatus for each piece of multimedia data (1102). It is checked whether the field of the request AV token matches the service for the registered service consuming device (1104). The service providing device then checks three key fields: media type supported by the multimedia service, media service, and service details (1106). The return AV capsule duration (710) is checked (1108) against the local AV token table for the service opportunity. This is followed by a check for negotiation emergency level (1110) and token status (1112). Based on the service opportunity with respect to the local AV token table (534) and AV token queuing threshold (1204), the service provider decides to accept the service request for scheduling (1114). If the negotiation is unsuccessful (1116), the service provider responds to the service consumer with a failure message in the token status (606) field of the response AV token (1118).

  Otherwise, the service providing device responds to the service consuming device with a success message (1116) in the token status field of the response AV token. Alternatively, the token status field (606) can have a status of change if the currently scheduled AV token needs to be changed to a later time, and the AV token can be transferred to another device. If it needs to be transferred, it can have a status of transfer, and if it should propose a different time, it can have a status of negotiate (1120). Next, the service providing apparatus schedules successful AV capsule / multimedia data based on the AV token for multimedia content streaming in the local AV token table, and reserves the resource based on the AV token details. (1122). Upon obtaining the multimedia data processing threshold (1206) trigger from the local AV token schedule to request multimedia data (AV capsules) (1124), the service provider may stream the identified AV capsule. For this purpose, a multimedia data request (FIG. 15) message is sent to the service consuming device (1126). Finally, the service consuming apparatus streams the designated multimedia data / AV capsule to the service providing apparatus based on the AV token information (1128).

  FIG. 12 details the schedule threshold triggering method based on AV tokens. Assuming that token ID_1 (1202a) through token ID_N (1202f) from the full list of AV tokens (1208) are currently scheduled in the AV token queue, two thresholds used for triggering purposes Exists. The first threshold is a multimedia data processing threshold (1206), which indicates the time at which multimedia data referenced by the next AV token should be streamed from the service consuming device to the service providing device. The service providing apparatus periodically refers to its own local AV token table (534) and identifies the next AV service task. Fields such as AV capsule size (520), capsule length (522), service start time (530), and estimated service end time (532) are referenced to identify the next AV service task. These fields are used to roughly estimate when the next AV token will be serviced, which is referred to as the multimedia data processing threshold (1206). Once the threshold is reached, AV data needs to be fetched or streamed from a defined location (518). For example, the time required for processing the AV data can be calculated based on the fields 520 and 522 regarding the processing speed of the service providing apparatus. This information can be reconfirmed by 528 and 532 authorized service windows. Next, a further number of AV tokens from the AV token table are similarly evaluated to form an AV token queue. This mechanism provides triggering of services based on thresholds.

  The second threshold, the AV token queuing threshold (1204), indicates the acceptability of a new AV token for scheduling based on the current queue length. These two indicators are dynamic and independent of each other. Similarly, the fields 520, 522, 530, and 532 described above can be used for this purpose. In addition, 524, 526, and 528 can be similarly referenced due to the particulate AV processing delay in both threshold triggers. For example, an AV transcoding service may require a longer time than playback. Current local multimedia service progress, negotiated return capsule duration (710), and resource availability dictate this decision.

  The next two figures, FIGS. 13 and 14, describe a scenario utilizing the present invention. FIG. 13 illustrates the overall process of AV token based service negotiation for multimedia service scheduling. First, the service consuming apparatus (1302) finds an available service providing apparatus 1 (1304), service providing apparatus 2 (1306), and service providing apparatus 3 (1308) (1310). Next, the service consuming apparatus performs registration with the selected service providing apparatus (1312). The service provider continuously updates all registered service consumers with static and dynamic information (1314). This service information is stored in the service information table (1316). During service negotiation, the service information table is referenced (1318) for creation and transmission of the requested AV token.

  In this scenario, only the service providing device 1 and the service providing device 2 are selected for service negotiation. Accordingly, the service providing apparatus 1 (1322) and the service providing apparatus 2 (1324) receive the requested AV token and perform a local AV token schedule check. The service providing apparatus 1 and the service providing apparatus 2 respond with a response AV token about the negotiation status (1320). The service consuming device transmits a negotiation confirmation agreement message to each service providing device (1326). In an AV token schedule trigger for the multimedia data threshold (1328 and 1330), a multimedia data request message is sent to the service consumer (1332). This is followed by multimedia data streaming (1334) to the service provider by the service consumer for each AV data processing (1336 and 1338). The actual AV data is streamed only when the AV data is needed by the service providing device, saving resources on both devices.

  FIG. 14 illustrates a scenario in which the service providing apparatus 2 (1406) is unavailable and the previously scheduled AV token is renegotiated to the service providing apparatus 3. Items 1402 to 1430 in the figure are the same as 1302 to 1330 in FIG. Therefore, detailed description is omitted.

  The service consuming device discovers that the service providing device 2 cannot be used by its own check (sending a request AV token for task status check) or notification from the service providing device 2 (1432). This triggers generation (1436) of a request AV token for negotiation with the service providing apparatus 3 that refers to the local service information table (1434). The service providing apparatus 3 executes a local AV token schedule check (1446). Next, the service providing apparatus 3 responds with a response AV token about the negotiation status (1438). This is followed by the transmission of the negotiation confirmation consent message to the service providing device 3 by the service consuming device (1440). In an AV token schedule trigger based on the multimedia data threshold (1448), a multimedia data request message is sent to the service consumer (1442). This is followed by multimedia data streaming (1444) to the service provider 3 by the service consumer for AV data processing (1450). In this scenario, only the AV token is rescheduled in the actual AV data movement for final processing.

  FIG. 15 shows the format of the multimedia data request message (1502). It has an AV token ID (1504) indicating a specific AV token reference ID. This is followed by stream information (1506) including IP address, protocol type, port number and session ID information for AV data streaming. If the new destination is used for processed AV data, the service providing device needs to provide the destination storage location for the processed AV content to be written, otherwise The initial storage location at the source and destination (706) of the AV capsule as in the AV token is used.

  The present invention is applicable to a method and apparatus for scheduling a peer-to-peer negotiated multimedia service based on an AV token.

FIG. 6 illustrates the coexistence of a service consuming device and a service providing device having local AV multimedia content, which are interconnected by an interconnection link between the service providing device and the consuming device; Is distributed between. The flow of the process of exchanging static and dynamic service information is shown. Fig. 4 shows static and dynamic service information tables. The overall process flow of multimedia service negotiation based on AV tokens is shown. An AV token table is shown. Indicates a response AV token. Indicates the request AV token. A detailed process flow of multimedia service negotiation based on the AV token of the service consuming device is shown. Fig. 5 shows a local AV token table schedule check process of a service consumer. Fig. 4 illustrates how to create an acceptable service delay for service negotiation. A detailed process flow of multimedia service negotiation based on the AV token of the service providing apparatus is shown. A threshold trigger method based on an AV token is shown. Fig. 4 illustrates a scenario of using AV tokens for multimedia service negotiation. Fig. 4 illustrates a scenario of using AV tokens for multimedia service renegotiation when a service or device is unavailable. Indicates a multimedia data request message. An AV token table generation step in the service consuming apparatus is shown. An AV token table generation step in the service providing apparatus will be described. The steps for creating a request AV token are shown. The step of creating a response AV token is shown.

Claims (22)

A method for scheduling a peer-to-peer negotiated multimedia service based on AV tokens in a multimedia service device environment comprising:
a. Before actual multimedia / AV data processing is performed, an AV token for multimedia / AV processing service negotiation is generated, and the actual multimedia / AV data processing is performed based on the negotiated token status. Dynamically reassign
b. For optimal multimedia service negotiation, a dynamic and static service information table is generated by the service providing apparatus, and the service information table is serviced by the service providing apparatus prior to the multimedia service negotiation. Dynamically exchange to equipment,
c. Prior to the start of the negotiation, the multimedia service opportunity is identified by the service consumer based on the dynamic and static service information from the service provider,
d. Selecting, reserving, negotiating and managing multimedia services provided by a service providing device by the service consuming device;
e. Transmitting the streamed multimedia content from the service consuming device to the service providing device before the start of the multimedia service.   The method of claim 1, wherein the service device comprises a service consumption device and a service providing device that are independent and interconnected via an interconnection link. The service consuming device includes a service consuming device that is a multimedia device for multimedia service,
a. A multimedia task is represented by an AV token by a service consumer,
b. Served multimedia content can be stored locally or on other interconnected devices,
c. The request AV token is created based on the local service information table and the conditions requested by the user application,
d. The requested AV token with multimedia task information is sent to the best matching service provider for multimedia service negotiation;
e. AV tokens are scheduled and tracked for the status of multimedia services / tasks locally by the service consumer,
f. The method according to claim 1 or 2, wherein the service consuming device periodically checks a local AV token table for the status of the multimedia service and communicates with the service providing device for the multimedia service update. The service providing device includes a multimedia device that provides a multimedia service,
a. AV token-formatted multimedia services / tasks are received from the service consumer and evaluated based on the current multimedia service load with reference to the local AV token table and AV token queuing thresholds,
b. A response AV token is created as a result of the evaluation of the multimedia service and sent to the respective service consumer,
c. Successful accepted AV tokens are scheduled in the local AV token table for multimedia services,
d. AV tokens are scheduled, tracked locally by the service provider for the status of the multimedia service / task, and updated to the service consumer using the response AV token when needed,
e. The service providing device periodically checks a local AV token table and a multimedia data processing threshold for the status of the multimedia service, and communicates with the service consuming device for streaming multimedia data. The method described in 1. The service information table is created on the service consuming device side based on static and dynamic information transmitted statically or dynamically by the service providing device,
The creation method includes collecting service information selected from a service providing device in a structured manner and forming a service information table;
a. The service information table includes a service providing apparatus ID and static and dynamic service information,
b. Static information in the service information table includes supported media types, supported media services, and multimedia service details;
c. The dynamic information of the service information table includes the processing speed of AV frames / second, the speed of the network path to the service providing device, the current status of the service providing device, and latency information,
d. The method according to claim 1 or 3, wherein the latency information includes network latency, multimedia data processing latency, and multimedia data buffering latency. The step of creating the AV token table in the service consumption device includes:
a. A request from a user application for a network multimedia service is received by the service consumer;
b. Analyzing the details / conditions of the network multimedia service request brought by the user application by the service consumer;
c. Based on the conditions of the network multimedia service request and the static and dynamic information matching in the service information table, the service consuming device selects a service providing device,
d. Create a request AV token to be inserted into the local AV token table;
e. Initiating a network multimedia service negotiation with the service providing device by creating a trigger at the service consuming device and sending a request AV token;
f. 5. The method of claim 1, 3 or 4, comprising updating the negotiated network multimedia service status based on a response AV token to a user application. The step of creating the AV token table in the service providing device is as follows:
a. Accepting a multimedia service request that is effectively or successfully negotiated for scheduling by the service providing device;
b. Using the request AV token, extract the required multimedia service information from the message body,
c. Generating an additional information field by the service providing device;
d. Create an entry in the AV token table of the service providing device based on the AV token ID,
e. 6. The method of claim 1, 3, or 5, comprising processing an AV token when a multimedia data processing threshold trigger is received. The request AV token creation step includes:
a. A trigger for initiating negotiation of a network multimedia service is received by the service consuming device;
b. Request AV token process
i. Start for all of the message fields, which are the AV token ID, service consumer ID, AV capsule storage location, return capsule time, capsule length, media type, media service, service details field,
c. Identify and insert AV token status for a specific requested AV token,
d. The method according to claim 1, 3, or 4, comprising the step of requesting an AV token to be transmitted to the specified service providing apparatus. The step of creating the response AV token includes:
a. Accepting a valid service request for scheduling by the service providing device and triggering a process of creating a response AV token;
b. Response AV token process
ii. AV token status specified and inserted for a particular response AV token, and iii. Start for all of the message fields that are fields of AV token ID, service providing device ID, service start time, and estimated service end time;
c. 6. A method according to claim 1, 3 or 5, comprising the step of sending a response AV token to the identified service consuming device. And includes steps for opportunity-driven peer-to-peer multimedia service negotiation,
The steps for the multimedia service negotiation include:
a. Receiving a request for a user application for a network multimedia service by the service consumer;
b. A request AV token for multimedia service negotiation is created by the service consuming device based on a service information table and user application conditions;
c. Sending a request AV token to the service providing device;
d. Receiving a requested AV token by the service providing device and making an acceptance decision for providing a multimedia service based on a local AV token table and an AV token queuing threshold;
e. About the negotiation status, the service providing device responds to the service consuming device with a response AV token,
f. If the negotiation is successful, update the local AV token table by the service consuming device, and send multimedia data to the service providing device when requested;
g. The method of claim 1 including the step of creating a request AV token for multimedia service negotiation by the service consuming device when the negotiation fails. Further comprising assigning an AV token status to the requested AV token for negotiation;
The assigning step comprises:
a. Generating a request AV token for each multimedia data by the service consumer;
b. Multimedia data information including urgency, AV data storage location, AV capsule size, and capsule length, media type, media service, service details, and return capsule duration The application condition is specified by the service consumption device,
c. 9. The method of claim 8, comprising the step of assigning by the service consuming device a normal, intermediate or quick urgency level in order of increasing priority with respect to time units based on user application requirements. Further comprising assigning an AV token status from a plurality of statuses to a requested AV token for negotiation;
The assigning step comprises:
a. When the AV token is created to negotiate the AV service for the first time, or when a critical field is changed by the service providing device in the response AV token requesting service negotiation, the status of the negotiation is used.
b. When the service consuming device causes the AV service to be transferred from the currently scheduled service providing device to another service providing device, the transfer status is used,
c. When the currently scheduled task is changed with respect to the AV service specification that does not affect the return capsule time, use the status of the change,
d. 12. The method according to claim 8 or 11, comprising the step of using the status of the check to request the AV token status at the service providing device side. Further comprising assigning an AV token status to the requested AV token for negotiation;
The assigning step comprises:
a. Insert one or more token statuses based on your negotiation needs,
b. The AV token negotiation status is determined by the service providing device,
c. If the AV token negotiation is successful, schedule the AV token in the local AV token table;
d. 13. A method according to claim 11 or 12, comprising the step of inserting a new token status based on the response AV token status if the AV token negotiation is unsuccessful. Further comprising assigning an AV token status to the response AV token for negotiation;
The assigning step comprises:
a. A request AV token for each piece of multimedia data is received by the service providing device;
b. 10. The method of claim 9, comprising examining each of the fields of the requested AV token for multimedia service matches. Further comprising assigning an AV token status to a response AV token for negotiation with respect to confirmation of a multimedia service match;
The assigning step comprises:
a. The local multimedia service capability is checked by the service providing device for media type, media service, and service detail information,
b. By referring to the AV token queuing threshold, the service providing device checks the return AV capsule required time for a request match,
c. To support urgent service requests by checking the urgency level by the service providing device by referring to the AV token queuing threshold when a new AV token can be added, and otherwise To negotiate with existing AV tokens,
d. 15. A method according to claim 9 or 14, comprising the step of checking token status by the service providing device to determine the nature of the request. Further comprising assigning an AV token status to a response AV token for negotiation with respect to confirmation of multimedia service matching and proper operation;
The assigning step comprises:
a. Determining an associated AV token status for the received request AV token by the service providing device;
b. Detects whether or not the specified request can be handled. If the specified request cannot be handled, a failure response is sent.
c. Detects whether the specified request can be handled without changing the service condition, and if it can respond to the specified request without changing the service condition, it responds with success,
d. Detects whether a specified service request condition needs to be changed for response, and newly negotiates a response when the specified service request condition needs to be changed for response With the proposed service conditions,
e. Detects whether a specified request is for transfer or change, and changes the required condition, transfer or change status, if the specified request is for transfer or change 16. A method according to claim 14 or 15 comprising steps. Further comprising assigning an AV token status to the response AV token for negotiation regarding confirmation of a multimedia service match and for successful operation;
The assigning step comprises:
a. If the response AV token is successful, calculate the service start time and estimated service end time, insert this,
b. Adding a successful AV token to the local AV token table by the service providing device and allocating the required local AV resources to the AV service;
c. Receiving a trigger from a local AV token by the service providing device to request multimedia data from the service consuming device based on a multimedia data processing threshold;
d. A multimedia data request message is transmitted by the service providing device to the service consuming device for specific multimedia data;
e. The method according to claim 14, 15 or 16, comprising transmitting a stream of multimedia data by the service providing device based on AV token information. And periodically checking the AV token table,
The periodically checking step includes:
a. Periodically check the local AV token table by the service consumer for the service start time field;
b. For a given multimedia data, if the service start time is greater than the current time, or if the sum of the service start time and the estimated service end time is greater than the allowable service delay, the AV providing status from the service providing device Create a request AV token to get
c. Set the AV token status field to check, inform the service providing device of the AV token status,
d. The service providing apparatus responds with an AV token having a status field indicating any one of completion, failure, or ongoing,
e. If the response AV token status is unsuccessful, restart the service negotiation process by the service consumer;
f. 5. The method of claim 4, including the step of setting the periodicity of the check to seconds or minutes precision. A method for calculating a return capsule duration with an acceptable delay for an AV token comprising:
a. Specify the presentation time of multimedia data / AV content,
b. For a particular multimedia content having a small portion of AV data, identify the number of AV tokens;
c. Ensuring that for all AV tokens of multimedia content to be negotiated, the return capsule duration is less than the sum of the latency and presentation time of the specified service provider;
d. The latency includes a network, data processing, and buffering time for a service providing apparatus.   20. A method as claimed in any preceding claim, wherein the threshold for multimedia data processing and AV token queuing is specified by AV token information from the AV token table.   20. A method as claimed in any preceding claim, wherein opportunity driven negotiation is accomplished using a service information table, a local AV token table, and various AV token status options. An apparatus for scheduling a peer-to-peer negotiated multimedia service based on AV tokens in a multimedia service device environment, comprising:
a. Before actual multimedia / AV data processing is performed, an AV token for multimedia / AV processing service negotiation is generated, and the actual multimedia / AV data processing is performed based on the negotiated token status. A means of dynamically reassigning,
b. For optimal multimedia service negotiation, a dynamic and static service information table is generated by the service providing apparatus, and the service information table is serviced by the service providing apparatus prior to the multimedia service negotiation. Means for dynamically exchanging into the device;
c. Means for identifying a multimedia service opportunity by a service consumer based on dynamic and static service information from the service provider before the start of the negotiation;
d. Means for selecting, reserving, negotiating and managing multimedia services provided by a service providing device by the service consuming device;
e. Means for transmitting the streamed multimedia content from the service consuming device to the service providing device before the start of the multimedia service.

Images