NZ623020B2 - Systems and methods for computing applications - Google Patents
Systems and methods for computing applications Download PDFInfo
- Publication number
- NZ623020B2 NZ623020B2 NZ623020A NZ62302012A NZ623020B2 NZ 623020 B2 NZ623020 B2 NZ 623020B2 NZ 623020 A NZ623020 A NZ 623020A NZ 62302012 A NZ62302012 A NZ 62302012A NZ 623020 B2 NZ623020 B2 NZ 623020B2
- Authority
- NZ
- New Zealand
- Prior art keywords
- components
- component
- graph
- data
- computing
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 134
- 230000008569 process Effects 0.000 claims description 88
- 238000012545 processing Methods 0.000 claims description 72
- 230000007246 mechanism Effects 0.000 claims description 29
- 238000004891 communication Methods 0.000 claims description 26
- 238000009826 distribution Methods 0.000 claims description 10
- 238000003860 storage Methods 0.000 claims description 10
- 238000010606 normalization Methods 0.000 claims description 7
- 230000004044 response Effects 0.000 claims description 7
- 230000001131 transforming effect Effects 0.000 claims description 7
- 238000005192 partition Methods 0.000 claims description 5
- 230000000007 visual effect Effects 0.000 abstract description 76
- 238000013461 design Methods 0.000 abstract description 64
- 238000011161 development Methods 0.000 abstract description 55
- 239000000243 solution Substances 0.000 description 87
- 239000003795 chemical substances by application Substances 0.000 description 83
- 230000018109 developmental process Effects 0.000 description 66
- 150000001875 compounds Chemical class 0.000 description 43
- 238000010586 diagram Methods 0.000 description 32
- 238000007726 management method Methods 0.000 description 21
- 230000006870 function Effects 0.000 description 16
- 238000012360 testing method Methods 0.000 description 13
- 230000008859 change Effects 0.000 description 11
- 238000005516 engineering process Methods 0.000 description 10
- 230000004048 modification Effects 0.000 description 8
- 238000012986 modification Methods 0.000 description 8
- 230000000694 effects Effects 0.000 description 7
- 150000002500 ions Chemical class 0.000 description 7
- 230000003068 static effect Effects 0.000 description 6
- 230000006399 behavior Effects 0.000 description 5
- FFBHFFJDDLITSX-UHFFFAOYSA-N benzyl N-[2-hydroxy-4-(3-oxomorpholin-4-yl)phenyl]carbamate Chemical compound OC1=C(NC(=O)OCC2=CC=CC=C2)C=CC(=C1)N1CCOCC1=O FFBHFFJDDLITSX-UHFFFAOYSA-N 0.000 description 5
- 239000011159 matrix material Substances 0.000 description 5
- 230000001052 transient effect Effects 0.000 description 5
- 238000009434 installation Methods 0.000 description 4
- 238000002955 isolation Methods 0.000 description 4
- 230000008520 organization Effects 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 230000000903 blocking effect Effects 0.000 description 3
- 230000006835 compression Effects 0.000 description 3
- 238000007906 compression Methods 0.000 description 3
- 238000010276 construction Methods 0.000 description 3
- 238000007667 floating Methods 0.000 description 3
- 238000013507 mapping Methods 0.000 description 3
- 238000012805 post-processing Methods 0.000 description 3
- 238000013519 translation Methods 0.000 description 3
- 238000011144 upstream manufacturing Methods 0.000 description 3
- 244000228957 Ferula foetida Species 0.000 description 2
- 239000008186 active pharmaceutical agent Substances 0.000 description 2
- 238000013475 authorization Methods 0.000 description 2
- 230000009977 dual effect Effects 0.000 description 2
- 239000004615 ingredient Substances 0.000 description 2
- 230000010354 integration Effects 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000002085 persistent effect Effects 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 238000010200 validation analysis Methods 0.000 description 2
- 241000282465 Canis Species 0.000 description 1
- 241000282326 Felis catus Species 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 150000001768 cations Chemical class 0.000 description 1
- 230000008867 communication pathway Effects 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000007418 data mining Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 230000008676 import Effects 0.000 description 1
- 238000002347 injection Methods 0.000 description 1
- 239000007924 injection Substances 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000002688 persistence Effects 0.000 description 1
- 230000037452 priming Effects 0.000 description 1
- 238000012913 prioritisation Methods 0.000 description 1
- 239000012088 reference solution Substances 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
- 230000011664 signaling Effects 0.000 description 1
- 239000000344 soap Substances 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 238000001228 spectrum Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 239000011800 void material Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/10—Protecting distributed programs or content, e.g. vending or licensing of copyrighted material ; Digital rights management [DRM]
- G06F21/105—Arrangements for software license management or administration, e.g. for managing licenses at corporate level
-
- G06F2221/0773—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/20—Software design
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/30—Creation or generation of source code
- G06F8/34—Graphical or visual programming
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/30—Creation or generation of source code
- G06F8/36—Software reuse
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/60—Software deployment
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/70—Software maintenance or management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/445—Program loading or initiating
- G06F9/44521—Dynamic linking or loading; Link editing at or after load time, e.g. Java class loading
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5061—Partitioning or combining of resources
- G06F9/5066—Algorithms for mapping a plurality of inter-dependent sub-tasks onto a plurality of physical CPUs
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/08—Network architectures or network communication protocols for network security for authentication of entities
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/08—Network architectures or network communication protocols for network security for authentication of entities
- H04L63/0823—Network architectures or network communication protocols for network security for authentication of entities using certificates
Abstract
Systems and methods for dynamic development and deployment of computing applications including a development framework, a visual design subsystem, and a deployment subsystem, where at runtime the deployment subsystem is operable to dynamically deploy a computing application realized by a blueprint by sending a request at runtime for graphs and components instantiated by the blueprint. y sending a request at runtime for graphs and components instantiated by the blueprint.
Description
Title: Systems and s for computing applications
Field
The described embodiments relate to systems and methods for computing
applications, and in particular, to systems and methods for dynamic development
and deployment of computing applications.
Background
ing applications generally involve processing data, performing
ions on the data to carry out specific functions, completing tasks, controlling
components, and so on. An example computing application is a media
application. Media applications generally involve producing, transforming or
delivering media data, or a combination thereof. New devices and technology
se the use of computing applications and data. New k capabilities
and improved data access further increase the use of computing applications and
data. The availability of multiple computing languages, protocols and platforms
increase s available to computing application providers, developers, and
users but may make it ult to use a combination of multiple computing
applications or combine a new computing application with an existing system or
architecture due to ation, interoperability and connectivity problems. There
exists a need for improved methods and systems for the development and
deployment of ing applications, or at least alternatives.
Summary
[2a] Embodiments bed herein relate to a system for dynamic deployment
of computing applications comprising: one or more linked repositories storing
blueprints, graphs, and components; one or more processors to receive a
command to deploy at least one computing application to process at least one
input data stream, and in response, r deployment of the at least one
ing application over a plurality of disparate host systems; the at least one
computing application realized by a blueprint of the blueprints stored in the one
or more linked repositories; at least one host system of the ity of disparate
host systems comprising a cloud agent and one or more cloud engines
tiated by the cloud agent; wherein the cloud agent is configured to receive
the command to deploy the at least one computing application to process the at
least one input data stream and, in response, instantiate the one or more cloud
engines on the respective host system and provide a running environment for the
one or more cloud engines; wherein the one or more cloud s is configured
to cally construct the at least one computing application on the respective
host system by realizing requirements of the blueprints of the at least one
computing application, the requirements identifying at least one graph from the
graphs stored in the linked one or more repositories, a plurality of components
from the ents stored in the linked one or more repositories, and by
sending a request at e to the one or more linked repositories to load the
blueprint, the at least one graph and the plurality of components on the
respective host system; wherein the one or more cloud engines deploys the
cally constructed at least one computing application on the tive
host system by: tiating the at least one graph using the blueprint, the at
least one graph representing a workflow of the plurality of components, the
workflow defining an arrangement of the plurality of components; detecting that
the plurality of components comprise a first set of components written for a first
ecture and a second set of ents written for a second architecture;
representing at least a portion of the at least one graph as a first subgraph and a
second subgraph; using the first subgraph to define connections between the first
set of components using pins and the workflow of the at least one graph, and
using the second subgraph to define connections between the second set of
components using additional pins and the workflow of the at least one graph,
each component of the first and second sets of components having at least one
input pin for receiving at least one input data container and at least one output
pin for providing at least one output data container, the respective component
transforming the input data container into the output data container using a
computing processing mechanism, each component being a distribution plug-in
unit to provide a portable and isolated dependency set for the computing
processing mechanism of the respective component; connecting the second
ph to the first subgraph using at least one additional input pin and at least
one additional output pin to pass at least a portion of the data containers
between the first subgraph and the second subgraph, the connected first and
second subgraphs maintaining the workflow of the graph; instantiating a g
s ce of the first architecture on the respective host system for the
first set of components of the first subgraph; instantiating a running process
instance of the second architecture on another host system of the disparate host
systems for the second set of components of the second subgraph, the running
process instance of the first architecture being linked to the running process
instance of the second architecture by the one or more cloud engines at
deployment time to link the first subgraph and the second subgraph; and
sing the input data stream as a plurality of data containers using the
plurality of components to te an output data stream, the ity of data
containers flowing between the ity of components using the pins; using the
at least one additional input pin and the at least one additional output pin to pass
at least a portion of the data containers n the running process instance of
first architecture on the respective host system and the running process instance
of the second architecture of the other host system to generate the output data
stream.
[2b] Embodiments described herein relate to a method for dynamic
deployment of computing applications: providing one or more linked repositories
storing blueprints, graphs and components; configuring one or more processors
to receive a command to deploy at least one computing application to process at
least one input data stream, and in se, trigger deployment of the at least
one computing application over a plurality of ate host systems, the at least
one computing application realized by a blueprint of the blueprints in the one or
more linked repositories; configuring at least one host system of the plurality of
disparate host systems with a cloud agent and one or more cloud engines
~1b—
instantiated by the cloud agent; receiving, at the cloud agent, the command to
deploy the at least one computing application to process the at least one input
data stream and, in response, instantiating, using the cloud agent, the one or
more cloud engines on the tive host system and providing a running
environment for the one or more cloud engines; using the one or more cloud
engines to dynamically construct the ing application on the respective
host system by realizing requirements of the int of the at least one
computing application, the requirements fying at least one graph from the
graphs stored in the linked one or more repositories, a plurality of components
from the components stored in the linked one or more repositories, and by
sending a request to the one or more linked tories to load the blueprint,
the at least one graph and the plurality of components on the respective host
system; deploying using the one or more cloud engines the dynamically
constructed at least one ing application on the respective host system by:
instantiating the at least one graph using the blueprint, the at least one graph
representing a workflow of the plurality of components, the workflow defining an
ement of the plurality of components; detecting that the plurality of
components comprise a first set of components written for a first architecture and
a second set of components written for a second architecture; representing at
least a portion of the at least one graph as a first subgraph and a second
subgraph; using the first subgraph to define connections between the first set of
components using pins and the workflow of the at least one graph, and using the
second subgraph to define connections between the second set of components
using onal pins and the workflow of the graph, each component of the first
and second sets of components having at least one input pin for receiving at
least one input data container and at least one output pin for ing at least
one output data container, the respective component transforming the input data
container into the output data container using a ing processing
mechanism, each ent being a distribution plug-in unit to provide a
portable and isolated dependency set for the computing processing mechanism
of the respective component; connecting the second subgraph to the first
subgraph using at least one additional input pin and at least one additional output
pin to pass at least a portion of the data containers between the first subgraph
and the second subgraph, the connected first and second subgraphs maintaining
the workflow of the graph; instantiating a running process instance of the first
architecture on the respective host system for the first set of components of the
first subgraph; instantiating a running process instance of the second architecture
on another host system of the disparate host systems for the second set of
components of the second subgraph, the running process instance of the first
ecture being linked to the running process instance of the second
architecture by the one or more cloud engines at deployment time to link the first
subgraph and the second subgraph; and sing the input data stream as a
plurality of data containers using the plurality of ents to generate an
output data stream, the ity of data containers flowing between the plurality
of ents using the pins; using the at least one onal input pin and the
at least one additional output pin to pass at least a portion of the data containers
n the running process instance of first architecture on the respective host
system and the g process instance of the second architecture of the other
host system to generate the output data stream.
In a further aspect, embodiments described herein provide a system for
dynamic development of computing applications comprising a development
framework, one or more processors, and a memory coupled to the one or more
processor and configured to store instructions executable by the one or more
processors to configure the development framework to define components and
graphs, wherein each ent s a computing processing mechanism for
processing data containers of computing data at application runtime, wherein
each graph identifies components,
~1d-
connections between the components, and properties for the ents, wherein a
graph is an instantiation of a corresponding int at application runtime, wherein the
development ork enables components to be embedded within other
components. Additional and alternative functionality is described herein.
In accordance with some ments, a graph may deliver functionality defined
by the components identified by the graph, and wherein a blueprint connects the
functionality to a running environment. The blueprint may provide business logic for the
ponding graph.
in accordance with some embodiments, the system may further comprise a
visual design subsystem for realizing computing applications, wherein the visual design
subsystem is le to arrange components into functional blocks, define specific
orders of operation for the functional blocks, and define connections between the
functional blocks to instantiate the computing applications. Additional and alternative
functionality is described herein.
[6] In ance with some embodiments, each component may be ated
with one or more versions, wherein at least one of a graph and a blueprint comprises a
reference to a solution set of components, wherein the solution set identifies a version
for each component.
In accordance with some embodiments, at least one component may be
associated with one or more versions and n the development framework s
loading of an appropriate version of the at least one ent at ation runtime.
in accordance with some embodiments, a first component may be in a first
language and a second component may be in a second different language, wherein the
first and second components comprise data and are operable to access the memory
and data structures, and wherein the system further ses a translation module
operable to translate multiple languages into a common ge by translating the first
and second component data and how the first and second component re operable to
access the memory and the data structures.
In another aspect, embodiments described herein may provide a method for
dynamic development of computing applications: providing a dynamic development of
computing applications comprising a development framework, one or more processors,
._ 2 _.
and a memory coupled to the one or more sor and configured to store
ctions executable by the one or more processors to configure the development
framework to define ents and graphs, wherein each component defines a
computing processing mechanism for processing data containers of computing data at
application runtime, wherein each graph identifies components, connections between
the components, and properties for the components, wherein a graph is tiated by
a corresponding blueprint at application runtime; n the development framework
enables components to be embedded within other components; developing
components and graphs for a blueprint; and storing the components and the graphs for
the blueprint in the repository for loading at application runtime.
in another , embodiments described herein may e a system for
dynamic ment of computing applications comprising: a deployment subsystem
for deploying computing applications at runtime, one or more processors, and a
memory coupled to the one or more processor and configured to store instructions
executable by the one or more processors to configure the deployment subsystem with
a repository, cloud agent, cloud engine, wherein the computing applications are realized
by blueprints, wherein each blueprint may be used to instantiate a graph at application
runtime, wherein a graph identifies components, connections between the components,
and properties for the components, wherein each component s a ing
processing mechanism for processing data containers of computing data at application
e, wherein each graph identifies components, wherein the tory stores the
graphs and the components for loading at application runtime, wherein the cloud agent
controls at least one cloud , wherein the cloud engine provides a running
environment for the computing ation by using blueprints to instantiate graphs at
application e; wherein at runtime the ment subsystem dynamically
constructs and deploys a computing application by sending a request at runtime to the
repository for the graphs instantiate by corresponding blueprints and components
identified therein. Additional and alternative functionality is described herein.
In accordance with some embodiments, each component may be associated
with one or more versions, wherein at least one of a blueprint and a graph comprises
reference to a solution set of components, wherein the solution set identifies a version
for each component.
In accordance with some embodiments, the system may further comprise a
e , wherein the license server may dynamically manage licenses and
associates iicenses with components and graphs, wherein use of components and
graphs at application runtime requires the appropriate license.
In accordance with some embodiments, the system may further comprise a job
manager, wherein the job manager dispatches blueprints and graphs to cloud agents
based on available es d by the license server. The job manager may also
1O be configured to provide job and cloud engine dispatch, failover, tracking and reporting.
In accordance with some embodiments, the system may r comprise a
security manager, wherein the security manager provides for secure connections and
communications between system components. Additional and alternative functionality is
described .
[15] In accordance with some embodiments, each graph identifies components,
connections between the components, and properties for the components, wherein
components are connected by different types of pins.
In accordance with some embodiments, a data container defines a data type and
a data , wherein the data type is metadata describing the data container and the
data object maintains raw data.
In accordance with some ments, the repository manages ning of
components and graphs to keep track of updates made thereto, wherein the repository
serves the components and graphs at application e using appropriate versions of
the graphs and components. Additionai and alternative functionality is described herein.
[18] In accordance with some embodiments, the cloud agent is provided to each user
system to manage the local resources of the user system, wherein the cloud agents
interact with cloud engines to instantiate graphs using ints. Additional and
alternative functionality is described herein.
In accordance with some embodiments, the system may r se a
normalization module operable to receive input data files and convert and parse the
input data fiIes into data containers for processing by a graph.
_ 4 _
In accordance with some embodiments, the system may further se code
g module operable to digitally sign each component to associate a developer,
license, or both with at least one ent. Additional and alternative functionality is
described herein.
In accordance with some embodiments, the system may further comprise a
l certificate associated with a component provider tem, wherein the
component provider subsystem provides one or more components; a digital certificate
associated with a user computing subsystem, wherein the user computing subsystem is
associated with a computing application, wherein the computing application involves a
1O component provided by the component provider computing system; a license server
configured to digitally sign a component by g the component to the digital
certificate associated with the user computing subsystem and the digital certificate
associated with the component provider subsystem to te that the user computing
system and the ent provider tem accept performance of the digitaiiy
signed component; wherein at runtime prior to deploying each component the
deployment subsystem queries the license server to ine whether the component
is linked to the digital certificate associated with the user computing subsystem and the
digital certificate associated with the component er subsystem.
In accordance with some embodiments, the deployment subsystem may be
further ured to partition a graph into two or more subgraphs and handle
interprocess communications between the two or more subgraphs.
In another aspect, embodiments described herein may provide a method for
dynamic deployment of computing applications: providing a deployment subsystem for
deploying computing applications at runtime, one or more processors, and a memory
coupied to the one or more processor and configured to store instructions executable
by the one or more processors to configure the deployment subsystem to comprise a
repository, cloud agent, cloud engine, wherein the computing ations identify
blueprints, wherein each blueprint may be used to instantiate a graph at application
e, wherein a graph identifies components, connections between the components,
and properties for the ents, wherein each component defines a computing
processing mechanism for processing data containers of ing data at application
_ 5 _
runtime, wherein each graph identifies components; storing components and graphs in
the repository for loading at application runtime; providing, by the cloud engine, a
running environment for the computing application by using blueprints to instantiate
graphs at application runtime; controlling, by the cloud agent, the cloud engine; at
application runtime, dynamically deploying a computing application by sending a
t at runtime to the repository for the graphs and components identified in the
blueprint.
In accordance with some ments, the method may further se
providing a digital certificate associated with a component provider subsystem, wherein
the component provider subsystem provides one or more components; providing a
l certificate associated with a user computing tem, wherein the user
ing subsystem is associated with a computing application, wherein the
computing application es a component ed by the component provider
computing system; providing a license server configured to digitally sign a component
by linking the component to the digital certificate associated with the user computing
subsystem and the digital certificate associated with the component er subsystem
to indicate that the user computing system and the component provider subsystem
accept performance of the digitally signed component; receiving, at a license server,
acceptance of the component provided by the ent provider subsystem in the
computing application associated with user ing system by receiving the digital
certificate from the user computing subsystem and the digital certificate from the
component er computing system; linking, at the license server, the component
provided by the component provider subsystem in the ing application associated
with user computing system to the digital certificate from the user computing subsystem
and the digital certificate from the component provider ing system; and at
application runtime prior to deploying each component, querying the e server to
determine whether the component is linked to the digital icate associated with the
user computing subsystem and the digital certificate associated with the ent
provider subsystem.
[25] In another aspect, some embodiments described herein provide a system for
dynamic development and deployment of computing applications (such as eg. a media
_ 5 ._
application) comprising: a development framework comprising a software development
kit, components, data containers, pins, and graphs, n the re development
kit is used to define components, graphs, data containers, and blueprints. Each
component may define a computer processing mechanism for processing data
containers at application runtime. Each graph may be a template of a set of
components, and each int may be an embodiment of a graph; a visual design
subsystem configured to output graphs and blueprints to develop computing
applications using components, compound components and other graphs, wherein the
visual design subsystem is operable to e ents into functional blocks and
1O define specific orders of operation for the functional blocks; a deployment subsystem
for deploying computing applications at e comprising a repository, cloud agent,
and cloud engine. The computing applications identify graphs, blueprints compound
components, and components. The repository is configured to store graphs and
components for loading at application e. The cloud engine provides a running
environment for graphs and executes graphs at application runtime to instantiate
computing applications. The cloud agent may control and manage the cloud engine. At
runtime the deployment subsystem may dynamically construct and deploy a computing
ation by sending a t at runtime to the repository for the graphs, compound
components, and components identified in the computing application. The deployment
subsystem is le to deploy a computing application by, at runtime, retrieving,
transferring, downloading, and so on, the graphs, blueprints, etc. from the repository.
The components, graphs, blueprints may not be present in the computing application
and they may be pulled dynamically to create the computing application at runtime.
They may atso pre-exist y due to previous availability ) or through user
intent (eg. a job manager persists the availability because the job is going to repeat).
In accordance with some embodiments, the deployment subsystem may r
comprise a license server which may dynamically manage licenses and associate
licenses with components and graphs. Use of components and graphs identified in a
computing application es the appropriate license.
In accordance with some embodiments, the deployment subsystem may further
comprise a job manager, which dispatches cloud engines based on available licenses
managed by the license server.
In accordance with some embodiments, the deployment subsystem may r
comprise a ty manager which provides for secure connections and
communications between system components.
in accordance with some embodiments, the deployment tem may r
comprise a job manager configured to provide job and engine dispatch, failover,
tracking and reporting. The job manager may also be configured to provide the highest
level of access to the running cloud engines, and provide centralized access to the
cloud engines regardless of state (running or not). The job manager may further self-
extend interfaces (e.g. web services) based on the graph/blueprint that is ioaded on the
cloud engine to provide a namespace (similar to the web) which may allow the
developer to discover which graphs/components are used in that particular application,
query/set ters, and so on.
In accordance with some embodiments, a graph may define a set of
components, where components in the set are ted by different types of pins.
In accordance with some embodiments, a data container may define a data type
and a data , n the data type is metadata describing the container and the
data object maintains raw data.
in accordance with some ments, the repository is operable to manage
versioning of components and graphs to keep track of updates made thereto. The
repository serves the components and graphs at application runtime using appropriate
ns of the graphs and components.
[33] in accordance with some embodiments, the cloud agent is provided to each user
system to manage the local resources of the user system. The cloud agents interact
with cloud engines to e graphs in order to run computing applications.
In accordance with some embodiments, the system may further comprise a
normalization module operable to receive input files and convert and parse the input
files into data containers to be processed by a graph.
In accordance with some embodiments, the system may further comprise a code
signing module operable to digitally sign each ent to associate a
developer/license with each component.
in accordance with some embodiments, the system may further comprise a
translation module le to translate multiple languages into a common ge
for system.
In accordance with another aspect, embodiments may provide a method for
dynamic development and deployment of computing applications (such as media
ations, for example) comprising: providing a pment framework comprising a
software development kit, components, data containers, pins, and graphs. The software
development kit may be used to define the ents, graphs, and data ners.
Each component may define a computer processing mechanism for processing data
containers of media data at ation runtime. Each graph may define a set of
components, along with specific connections between the components and properties
for the components; providing a visual design subsystem to define and output graphs,
wherein graphs may be used to realize and create computing applications; using the
visual design subsystem to arrange ents into functional blocks and define
specific orders of operation for the functional blocks, including connections between the
onal blocks; providing a deployment subsystem for deploying ing
applications at e comprising a repository, cloud agent, and a cloud engine. The
computing applications may identify graphs, compound components, and ents.
The cloud engine may provide a running environment for graphs and executes graphs
at application runtime to instantiate ing applications. The cloud agent controls
the cloud engine; storing components and graphs output by the visual design
subsystem in the repository for loading at application runtime; and at application
runtime, dynamically ucting and deploying a computing application by sending a
request at runtime to the repository for the graphs, compound components, and
components identified in the computing applications.
In another aspect, embodiments described herein provide a system for dynamic
development and deployment of computing applications comprising: a development
framework comprising a software development kit, components, data containers, pins,
_ 9 _
and graphs. The software development kit may be used for defining the components,
graphs, blue prints, and data containers. Each component may define a media
processing mechanism for processing data containers of ing data at application
runtime. Each component may be associated with one or more versions. Each graph
may be a template fying components and used to generate a ponding
blueprint. A blueprint may be a final embodiment of the graph and may comprise a
reference to a solution set of components, where the solution set of components may
fy a version for each component; a visual design subsystem configured to define
and output graphs in order to realize and create ing applications. The visual
design subsystem may be operable to arrange components into functional blocks and
define specific orders of operation for the functional blocks, including connections
between the functional blocks. The visual design tem may be further configured
to define a solution set of components by identifying a n of each component; a
deployment subsystem for deploying computing applications at runtime comprising a
repository, one or more cloud agents, and one or more cloud engines. The computing
applications identify graphs, compound components, and components. The repository
may be configured to store graphs, blueprints and components for g at
application runtime. The cloud engine may provide a running environment for graphs
and may e blueprints of the graphs at application e to instantiate
computing applications. The cloud agent may be operable to l the cloud engine;
wherein at runtime the deployment subsystem dynamically constructs and deploys a
computing application by sending a request at runtime to the repository for the graphs,
blueprints, compound components, and components, ing appropriate versions
f, identified in the computing applications.
[39] In a further aspect, ments described herein provide method for dynamic
development and deployment of computing applications: providing a development
framework comprising a software development kit, components, data ners, pins,
and graphs. The software development kit may define components, graphs, blueprints,
and data containers. Each component may define a er processing mechanism
for processing data containers of computing data at application runtime. Each
component is associated with one or more versions. Each graph may be a template
_. 10 _
fying components and may be used to generate a corresponding blueprint. A
blueprint may be a final embodiment of the graph and may comprise a reference to a
solution set of components, where the solution set of components identifies a version
for each component; providing a visual design subsystem for defining and outputting
graphs in order to develop media ations, and for defining a solution set of
components by identifying a version of each component; using the visual design
subsystem to define a graph by arranging components into functional blocks and
ng specific orders of operation for the functional , including connections
between the functional blocks, where the graph references a solution set of
components; using the visual design subsystem to define a on set of components
referenced by the graph by receiving a selected version for each component in the
solution set of components; providing a deployment subsystem for deploying computing
applications at runtime comprising a tory, one or more cloud agents, and one or
more cloud engines. The media ations identify , compound components,
and components. The cloud engine provides a running environment for graph and
executes blueprints of the graphs at application e to tiate computing
ations The cloud agent may be operable to control the cloud engine; storing
components and graphs output by the visual design subsystem in the repository for
loading at application runtime; and at application runtime, dynamically ucting and
deploying a computing application by sending a request at runtime to the tory for
the graphs, compound components, and components, including appropriate versions
thereof identified in the computing application.
ln another aspect, embodiments described herein provide a system for dynamic
pment and deployment of computing applications comprising: a development
framework comprising a software development kit, components, data containers, pins,
and graphs. The software development kit may be for defining the components, graphs,
blue prints, data containers. Each ent may define a media processing
mechanism for processing data containers of computing data at application runtime.
Each graph may be a template identifying components and may be used to generate a
corresponding blueprint. A blueprint may be a final embodiment of a graph, and a graph
is the instantiation of a corresponding blueprint at application runtime; a digital
_ 11 _
certificate associated with a component provider subsystem, where the component
provider subsystem es one or more components of the development framework;
a digital certificate associated with user computing subsystem, where the user
computing subsystem is associated with a computing ation, where the computing
application involves a component provided by the component provider computing
system; a visual design subsystem configured to define and output graphs in order to
develop computing applications, where the visual design subsystem is operable to
e components into functionai blocks and define specific orders of operation
the functional ; a deployment subsystem for deploying computing applications at
1O runtime comprising a repository, one or more cloud agents, and one or more cloud
engines. The computing applications identify graphs, compound components, and
components. The repository may be configured to store graphs and components for
loading at application runtime. The cloud engine may provide a g environment for
graph and may execute blueprints of the graphs at application runtime to instantiate
computing applications. The cloud agent is operable to control one or more cloud
engines. The deployment subsystem may further se a license server configured
to digitally sign a component by linking the component to the digital certificate
associated with the user computing subsystem and the digital certificate associated with
the component provider subsystem to indicate that the user ing system and the
component er subsystem accept mance or conformity of the digitally signed
component, where performance may relate to runtime performance, service level
agreement, and other measures of mance; wherein at e the deployment
tem dynamically constructs and deploys a computing application by sending a
request at runtime to the repository for the graphs, blueprints, compound components,
and components identified in the computing applications. Prior to deploying each
component, the deployment subsystem may query the license server to determine
whether a component is linked to a digital certificate associated with the user ing
subsystem and the digital certificate ated with the component provider
subsystem.
[41] In a r aspect, embodiments described herein provide a method for dynamic
development and depioyment of computing applications comprising: providing a
_ 12 _
development framework comprising a software development kit, components, data
containers, pins, and graphs. The software pment kit may define components,
graphs, blueprints, and data containers. Each component may define a computing
processing mechanism for processing data containers of computing data at application
runtime. Each component may be ated with one or more ns and each
graph may be a template identifying components and may be used to generate a
corresponding blueprint. A blueprint may be a final embodiment of a graph, and a
int may be used to instantiate a graph at application runtime; providing a digital
certificate to a component provider computing system, where the component provider
1O computing system provides one or more components to the development framework;
providing a digital certificate to a user ing subsystem, where the user computing
subsystem is ated with a media application, and where the computing application
involves a component provided by the component provider computing system; providing
a visual design subsystem for defining and outputting graphs in order to p
computing applications, and for defining a solution set of ents by identifying a
version of each component; using the visual design tem to define a graph by
arranging components into functional blocks and ng specific orders of operation
for the functional blocks, where the graph may reference a solution set of components;
providing a deployment subsystem for deploying computing applications at runtime
comprising a repository, one or more cloud agents, and one or more cloud engines.
The computing applications may identify graphs, compound components, and
components. The cloud engine may provide a running environment for graphs and may
execute blueprints of the graphs at application runtime to instantiate computing
applications. The cloud agent is operable to control the cloud engine; receiving, at a
license server, ance of the component provided by the ent provider
subsystem in the computing application ated with the user computing system by
receiving the digital certificate from the user ing subsystem and the digital
cate from the component provider computing system; linking, at the license server,
the component provided by the component provider subsystem in the ing
application associated with user computing system to the digital certificate from the user
computing subsystem and the digital certificate from the component provider computing
_. 13 _
system; storing components and graphs output by the visual design subsystem in the
repository for loading at application runtime; and at application runtime, dynamically
constructing and deploying the computing application associated with the user
computing subsystem by sending a t at runtime to the repository for the graphs,
compound components, and components identified in the computing application; and
prior to deploying the ent provided by the component provider computing
, querying the license server to determine whether the component is linked to
the digital certificate associated with the user computing subsystem and the l
certificate associated with the component provider subsystem.
1O [42] Variations and combinations may also be provided by the embodiments
described herein. Additional aspects of various example embodiments are identified
and described in the following description.
Brief Description of the gs
For a better understanding of embodiments of the systems and methods
bed herein, and to show more clearly how they may be carried into effect,
reference will be made, by way of example, to the accompanying drawings in which:
illustrates a block diagram of the system for dynamic development and
deployment of computing ations, in accordance with an example embodiment;
illustrates a block m of the data flow of a system for dynamic
development and deployment of computing applications, in accordance with an
e embodiment;
illustrates another block diagram of the data flow of a system for
dynamic development and deployment of computing applications, in accordance with
an example ment;
illustrates a block diagram of example components in accordance with an
e embodiment;
illustrates a block diagram of example properties of an example
component in accordance with an example embodiment;
rates a block diagram of example data container and ents in
accordance with an example embodiment;
_ 14 _
illustrates a block diagram of an example graph in accordance with an
e embodiment;
illustrates a block diagram of an example ace for a visual design
subsystem in accordance with an example embodiment;
illustrates a block diagram of an e interface for a repository in
accordance with an example embodiment;
illustrates a block diagram of an example interface for a job manager in
ance with an example embodiment;
FIGS. 9 and 10 illustrate block diagrams of example web services
implementations in accordance with example embodiments;
FIGS. 11 and 12 illustrate block diagrams of example implementations of an
asset management and publishing system in accordance with example embodiments;
illustrates a block diagram of an example interface for defining a solution
set of components in accordance with example ments;
illustrates a block diagram of an example certification system in
accordance with example embodiments;
rates a block diagram of dynamic provisioning in accordance with
example embodiments;
illustrates a block diagram of partitioning mixed architectures in
accordance with example embodiments;
illustrates an example browser based console to access the license
server in accordance with example embodiments;
illustrates a block diagram of stand-alone deployment in ance with
example embodiments; and
illustrates a block diagram of network ment in accordance with
example ments.
The drawings, described below, are provided for purposes of illustration, and not
of limitation, of the s and features of various examples of embodiments
described herein. The drawings are not ed to limit the scope of the teachings in
any way. For simplicity and clarity of illustration, elements shown in the figures have not
-15“
necessarily been drawn to scale. The dimensions of some of the elements may be
exaggerated relative to other elements for clarity. Further, where considered
appropriate, reference numerals may be repeated among the figures to indicate
corresponding or analogous elements.
ption of Exemplary Embodiments
It will be appreciated that numerous specific details are set forth in order to
provide a thorough understanding of the exemplary embodiments described herein.
However, it will be understood by those of ordinary skill in the art that the ments
described herein may be ced without these specific details. ln other instances,
well-known methods, procedures and components have not been described in detail so
as not to obscure the embodiments described herein. rmore, this description is .
not to be considered as limiting the scope of the embodiments described herein in any
way, but rather as merely describing implementation of the various example
embodiments described herein.
[46] The embodiments of the systems and methods described herein may be
implemented in hardware or software, or a combination of both. However, these
embodiments may be implemented in computer ms executing on programmable
computers, each computer including at least one processor, a data storage system
(including volatile and non—volatile memory and/or storage elements), and at least one
communication interface. For example, the programmable ers may be a server,
network appliance, set-top box, embedded device, computer expansion module,
personal computer, , personal data assistant, cloud computing system or mobile
device. A cloud computing system is operable to deliver computing e through
shared resources, re and data over a network. Program code is applied to input
data to perform the functions described herein and to generate output ation. The
output information is applied to one or more output devices to generate a discernible
effect. In some embodiments, the ication interface may be a network
communication interface. in ments in which ts are combined, the
communication interface may be a software communication interface, such as those for
~15-
process communication. in still other embodiments, there may be a combination of
communication interfaces.
Each program may be implemented in a high level procedural or object oriented
programming or scripting language, or both, to communicate with a computer system.
However, alternatively the programs may be implemented in assembly or machine
language, if desired. In any case, the language may be a compiled or interpreted
language. Each such er program may be stored on a storage media or a device
(eg. ROM or magnetic diskette), readable by a general or special purpose
programmable computer, for uring and operating the er when the storage
1O media or device is read by the computer to perform the procedures described herein.
Embodiments of the system may also be considered to be implemented as a non-
transitory computer-readable storage , ured with a er program,
where the storage medium so configured causes a computer to operate in a specific
and predefined manner to perform the functions described herein.
[48] Furthermore, the system, processes and methods of the described embodiments
are capable of being buted in a computer program product including a physical
non-transitory computer readable medium that bears computer usable instructions for
one or more processors. The medium may be provided in various forms, including one
or more diskettes, compact disks, tapes, chips, magnetic and onic storage media,
and the like. The computer useable instructions may also be in s forms, including
compiled and non—compiled code.
Embodiments described herein may relate to various types of ing
applications, such as media applications, ce related applications, voting
applications, user registration applications, integrity management applications, and so
on. By way of illustrative example embodiments may be described herein in relation to
media applications.
Referring now to , there is shown a block diagram of a system 10 for
dynamic development and/or deployment of computing ations in accordance with
an example embodiment. By way of example, a computing application may be a media
application. A media application may be a computing application designed to m
specific tasks and activities for manipulating media data using a combination of
- 17 _.
hardware and software computing components. For example, the media ation
the data to carry out
may involve processing media data, performing operations on
specific functions, completing tasks, controlling components, producing, transforming or
delivering media data, or a combination thereof. The media application may generate a
deliverable or transform a rable for provision to output devices and for generation
of a discernable effect, such as by transforming received input media data into a
deliverable, for example. The media application may process, transform and manipuiate
input data streams to generate a complete media program for display, broadcasting,
distribution, and so on. For example, playback of the input data stream may be
discernably different from playback of the deliverable generated or ormed by the
media application.
The system 10 may scale from simple media applications run on a local
computer to complex media applications ed on a cloud ing system. A
cloud computing system is operable to deliver computing services through shared
resources, software and information over a network. The system 10 may be operable
for multipie platforms (e.g. Windows, Linux, OS X) and multiple languages (e.g. C++,
Java, ing), and may use standards based interfaces (e.g. SOAP, XML).
The system 10 may be implemented as a cloud computing system and may be
accessible to users through an externai interfaces layer 38 which may allow integration
with existing ses, applications and systems. The system 10 may include a
development framework 12 and a visual design subsystem 30 to define and output
graphs 28 in order to develop media applications. The system 10 may include a
deployment subsystem 14 for cally ing media applications at runtime. The
system 10 may provide a rm for building, developing and deploying professional
workflow ations for desktop, networked and cloud based systems.
By way overview, the development ork 12 may be used for the
development of component 24 and workflow (e.g. graphs 28, blueprints 28a)
technologies. The repository 32 may provide a centralized pool of component 24
logies and workflow blueprint328a and may act as both the warehouse and
supply chain (for syncing upstream/downstream repositories 32). The visual designer
may be used to design and test workflow graphs 28 and blueprints 28a. A license
._ 13 ._
server 42 may control authorization of component technologies. The system 10 may
e one or more of the following features: multi-platform support through the
ork SDK 20; multi-language support allows native pment in multiple
languages such as C++, Java, and scripting languages; support for fiexible workflow
models with inline, parallel, and staged execution of individual processes; consistent
management interfaces through standards based web services regardless of workflow
complexity or scope; dynamic scalability allows for simple and complex solutions to
easily scale to large volume processing capabilities with low provision and ment
costs. Other features may also be provided by system 10 as described herein.
[54] The system 10 may enable decomposition of hardware and software problems
into their core elements. These core elements may be referred to as components 24.
By breaking down multiple ms, a catalog of components 24 may be ped
that can be brought together in different ways (eg. by graphs 28, blueprints 28a) to
solve new problems. For example, a user may want to perform video compression and
send email notification upon completion. These two problems are very different but by
combining elements of the video compression problems, that is, components 24 for
video codec, multiplexer and file writer; and the email problem, that is, components 24
for database lookup, report generator and email engine; system 10 can combine the
two into a te solution that not only performs the core video compression, but
may also sends out notification emails to the users who need to be notified of the
project completion.
The system 10 may enable registration of the components 24 into a repository
32 of technology, used to store, manage, and access these technologies in a controlled
nment that may be centralized or distributed. The system 10 may allow these
repositories 32 to be d together into d supply chains, where downstream
repositories 32 can be synced with upstream tories 32.
The system 10 may control access to components 24 using a floating license
server 42 which may check out licenses when components 24 are being used.
The system 10 may provide a ication/management bridge between a
higher level application and the cloud engines 36a which run jobs. The cloud agents 34
_1g_
may provide that bridge. The cloud agents 34 may provide a consistent web services
integration and management point less of the complexity of the solution.
The system 10 may provide a method for creating workflow solutions s 28,
blueprints 28a) from ents 24. The visual designer 30 may be implemented as a
visual design tool which allows new ons to be created, tested, and saved as
graphs 28 or blueprints 28a that can be nced by an application. Blueprints 283
can also be stored in the repository 32, becoming part of the managed supply chain.
The system 10 may run cloud engines 36 to execute jobs. When the application
sends a command to the cloud agent 34 to run a job, the cloud agent 34 determines
which components are required to start running the solution and es those
components from the repository 32. For example, the cloud agent 34 creates the cloud
engine 36a, the cloud engine 36a loads the graph 28 or blueprint 28a, acquires the
required licenses from the license server 42, and runs the job. The cloud engine 363
may dynamically acquire new ent licenses on the fly as required by the currently
running workflow. When the job is complete the licenses are returned to the pool of
licenses managed by the license server 42.
Development Framework
The development framework 12 may include components 24, compound
components 26 nents ed within other components), data containers 56,
graphs 28, and blueprints 283. The development framework 12 may be accessible
through a software development kit (SDK) 20, web services or by using the visual
design subsystem 30. Both the SDK 20 and the visual design tem 30 may be
used to develop components 24, compound components 26, and graphs 28, and to
define new data types to define new data containers 56. System 10 may provide a
graph-based processing engine, where graphs 28 are made up of reusable
components 24 with discrete processing functions. The system 10 may also provide the
framework for development and deployment of graphs 28 and components 24. As
noted above, the development framework 12 may be accessible through web services,
where a user may create graphs and components using the web es application
programming interface.
-20_
As noted, the SDK 20 may be used to define components 24, graphs 28, data
containers 56, and other features of the system 10. The SDK 20 may be language
independent. An example of the SDK 20 in java is:
public class LoggingComponent extends JavaComponent{
public void process(DataContalner data, String inputPinName ){
super.process(data, inputPinName);
log( LevellNFO, ss: "+ data );
getOutputPin().process(data);
Other languages may also be used and this is an example only.
The SDK 20 may include a framework APl and a component API.
The framework APl may be used to create the graphs 28 used by an application,
to control g and executing graphs 28 and to e status to the application.
The component API may be used to create individual components 24.
Separating the component APl from the framework APl may allow component
pers to focus on the development of a component 24 without requiring
knowledge about the logistics of the whole environment.
[67] The SDK 20 may include application programming ace in multiple
languages (such as java, C++ for example) to create components. Components may be
created using one or more languages.
Components 24 are building blocks of the system 10. A component 24 is an
object, plug in or set of software code that defines a processing mechanism and uses
the SDK 20 to interact with the development framework 12. At application e, a
component 24 is configured to process data flowing through the component 24 as a
data container 56. Each component 24 may be a single component 24 or a compound
component 26 built up of multiple embedded components 24. A component 24 may
contain plug in files, and other files, such as jar files, dlls, and so on.
[69] Referring now to there is shown a block diagram of example ents
24 in accordance with an example embodiment. Examples of components 24 for a
.. 21 ..
media application context include video input 24a, video process 24b, file sink 240, logic
branch 25 (decisioning and routing based on criteria, such as for e video
format), strip letterbox 24d, and aspect ratio 24e. Other examples of components 24
are shown in FIGS. 5 and 6, such as file source 24f, interlace or 249, film removal
24h, deinterlacer 24i, noise reduction 24j, buffer 24k, file sink 24x, YUV to RGB 24m,
image converter 24h, flow control 240, file input 24u, color space converter 24v, scaler
24w, and AVC encoder 24y.
Components 24 may have properties and values for those properties. A
component’s 24 properties configure its behavior. The properties provide runtime
information, transient information, s, and so on. Referring now to there is
shown a table representing example properties 25 and values 27 of an example
component 24 (file source 245) in accordance with an example embodiment. Examples
of properties 25 include description, configuration warning, default input pin, default
output pin, description, file, name, last error, log, progress, read buffer size, and so on.
Each property may have an associated value 27. A components 24 properties 25 may
be set and modified h the visual design subsystem 24 or other interface.
Properties modify the way a component 24 s, such as how it processes
data or the type of output it produces. For instance, properties can be used to change
the format of a component’s 24 output or provide scaling information. Properties can be
thought of as instance variables for components 24. Properties can be exposed to other
components 24 through pins.
Property attributes may change the way a ty is used, exposed and saved.
They may be set in the property ation in the pluginxml file. For example,
properties may have one or more of the following attributes: transient (when a graph is
saved to file, property values may be saved with it by default, r, if a ty is
transient, its value may not be saved and the default value may be used when the
graph is loaded from file), required (the property must be set for the component to run),
hidden (the ty is used internally by a component and may not be e to a
user), advanced (the property generally does not need to be modified by the user, but
may be of interest to experienced users), interprocess (the property may be accessible
to processes that are spawned by its graph), and so on.
_ 22 _
Properties can be d on pins. A property exposed on a pin can be n
to or read by another component. The name of a property pin is the same as the name
of the property defined in the property deciaration. The property pin’s display name in
the visual designer 30 may be the same as the property name unless a pin display
name is provided. Properties may be declared within component definitions in the
plugin file, such as a plugin .xml file for example.
Example attributes of properties may include property name, y name,
description, required, advanced, hidden, transient, interprocess, value type, and initial
value. A property may be d as advanced if it generally does not need to be
modified, but may be of st to experienced users. For example, setting the attribute
ed=“true” may hide the property in the visual designer 30. The property may
become visible when an “advanced” box is selected in the visual designer 30. Setting
hidden=“true” may hide the property in the visual designer 30. The property may
become visible when a “hidden” box is selected in the visual designer 30. When a
graph is saved to file, the property values of its components may also be saved. Setting
transient=“true” may result in the ty value not being saved. The t property
value may be used when the graph is loaded from file. Setting interprocess=“true” may
make a property accessible to processes spawned by its graph. A property l value
may be the default value of the property when the component is initially instantiated.
[75] Property values may be restricted. For example, ty values may be
restricted to strings, numbers, integers, range of numbers defined by a minimum and a
maximum, and so on. ty value restriction attributes may include a minimum
value, a m value, a list of enumerated values, data type for values, and so on.
An attribute property initial value may set the initial value of a property that has
been declared elsewhere (for example, if a property has been declared in an inherited
component, in the component’s source code or in the framework itself). An example
use of ty initial value may be for instructing a C++ or dual Java/C++ component
how to construct its native peer.
Components 24 may be language independent and may be developed by
different deveiopers in different languages, and used together to create a graph 28,
blueprint 29 or compound component 26.
_ 23 -
Each component 24 may have one or more versions. A n is a specific form
or state of the ent 24, and may reflect new developments or implementations of
the component 24. Each version of a component 24 may be referenced using a version
name or . For example, each version may be assigned a number in increasing
order. As will be explained herein in relation to , the system 10 maintains
versioning to keep track of and use different versions of components 24 in graphs 28,
blueprints 28a, and compound ents 26. This may result in more flexible system
as ent versions of the same component 24 may be usable by graphs, media
applications and users, and each are not required to use the same component and
version thereof.
Components 24 may be written for different architectures or contexts, such as 32
bit and 64 bit architectures. As will be explained herein, system 10 is operable to
develop and deploy an application instance which combines components written for
both 32 bit and 64 bit architectures. For example, system 10 is le to detect
whether a particular media application has been developed using both components 24
for 32 bit architectures and components 24 for 64 bit architectures. if so, system 10 is
operable to create a separate process space or instance for each context and handle
inter process communications using g and a shared memory. For example, the
system 10 is operable to create a 32 bit architecture process instance and a 64 bit
architecture process instance and manage communications between the s
instances.
r, components 24 may be self-contained and isolated from a dependency
point of view. The entire dependency set of a ent 24 may be self-contained,
being ied and packaged in the component distribution unit (e.g. plugin). The
component 24 dependencies may also be isolated, referring exclusively to the specific
component 24 and version(s) they depend on. This may enable the system 10 to
realize complex workflows while resolving components 24 dependencies without user
intervention. Further, the dependency isolation may allow the system 10 to provide
distinct behavior while executing blueprints built with the same components 24 by
isolating the different versions of these components 24 and their dependencies.
-24..
ents 24 may have pins. Pins t to pins on other components 24.
Pins may be referenced by name. Pins may t to multiple components 24, which
may be referred to as branching. In ance with some embodiments described
herein, components 24 do not have to be of the same type to connect to the same pin.
There may be different types of pins. There may be input pins, such as an input
push and input pull, which can be used to decouple components 24 with different
fundamental architectures. A push pin pushes its output to the next pin and a pull pin
calls for data on its input pin. The pin model ls the flow of data between
components. There are output pins. There are control pins, including event (out),
property (in/out), and command (in) pins.
Pins may be used to pass data between components 24. Pins may expose
properties and events, and may be used to trigger commands. Static pins may be
defined within the component 24 definition in the plugin.xm| file and may be created
every time a component 24 is instantiated. Dynamic pins may be defined within the
component 24 source code and may be added after the component 24 has been
instantiated.
Input and output pins may be d as default pins. A default pin may not need
to be referred to by name in the component source code. There may be only one
default input pin and only one default output pin per component.
[85] As noted herein, there may be different types of pins. For example, an
OUTPUT__PUSH pin is a type of output pin. Data may be sent through an output pin to
the next component's 24 input pin. lNPUT__PUSH and lNPUT_PULL are two different
types of input pins. When using a push input pin, a component may process data as it
arrives on the pin. When using a pull input pin, a ent 24 may request data from
the pin, blocking until data arrives. Pull input pins may be used in situations where there
is more than one input pin on a component 24 and the data coming in through each pin
needs to be controlled and coordinated. OUTPUT_lO and lO pins are further
examples. l/O pins act as both input and output pins and are typically used to pass data
between graphs 28 or as output and input pins in compound components 24. A
PROPERTY pin may expose a component’s property so that it can be read or modified
by other components 24. There may also be EVENT pins. When an event occurs, the
_. 25 _.
data object generated by the event may be encapsulated in a Data Container 56 that
may be pushed onto the event pin. if the event generates a null object, an empty Data
Container 56 may be placed on the pin. The ation of a Data Container 56 on an
event pin signals that the event has occurred. COMMAND pins act as triggers to call
ds on components. When a data container is placed on a command pin, the
command is executed. The data on the data container may be used by the command,
butfidoesnotneedtobe.
Pins may set data types. For example, a data type feature may provide
information to the end user about a component’s expected input and output types. This
may be useful at graph 28 design time to ensure the compatibility of connected
components. Once the graph 28 starts, data types describing the actual data passing
between components 24 may be used. A warning may be generated when incompatible
pins are connected. The data type of a static pin may be set in the pin declaration using
a data type ion name. The data type definition name may take the name of a data
type definition, which is a set of key/value pairs that describe features such as image
dimensions, audio format, and encoding format. For example, a pins data type for an
input push pin may be set to integer.
The data type tion may be the default data type of an unconnected input
pin. When components are connected, the input pin may acquire the data type of the
output pin it is ted to. A ent’s 24 default output pin may acquire the data
type of the component’s default input pin unless the pins have been decorrelated. All
other output pins may use their own data type tion names to set their data types.
A data type transform may be used to change the default output pin’s data type.
A data type transform may add or remove data type definitions from the output pin’s
acquired data type. Consider the foiiowing example where the default input pin is
defined with a data type of integer. When the component is tiated, the default
input pin and the default output pin may both have the same data type, namely the
“integer” data type. To change the default output pin’s data type to string, a data type
transform may be used to remove the “number” data type definition (of which integer is
a e) and add the string data type definition.
Data type restrictions may be used to provide more detail about what types of
input a pin can accept. While setting a data type on a pin may act a simple data type
restriction, setting a formal data type ction on the pin can narrow down the type of
data that is acceptable. Explicit data type restrictions may override the restriction
implied by the pin’s data type. Data type restrictions may be nested using logic
operators (AND, OR, NOT). The syntax for data type restrictions may follow prefix
notation. For example, say you want your pin to accept all numbers that are not
integers: numbers AND (NOT r), and in prefix notation this may be: AND number
(NOT integer).
[90] There may be a pin definition schema. A component’s static pins may be
declared in its tion. Static input, output, event, command and property pins may be
declared in the component tion. In the case of event, d and property pins,
the name of the pin may need to match the name of the event, command or property,
respectively. A pin’s data type may be defined in the plugin.xml file using a data type
definition name, data type restrictions and data type transforms can also be set within
the plugin.xml file.
A pin definition may include a name, type, default, data type, and y name.
For a pin name, upon compiling the plugin.xml, a constant of the form P|N_<name>
may be generated. The pin may be referenced in source code using this nt. For
example, the constant PlN_file may be generated for a pin named “file”. input, output,
event and command pins may be displayed on the component with this name in the
visual designer 20. Property pins may have a different display name. The alternate
display name may be set in the property declaration. Pins can be of type,
INPUT_PUSH, lNPUT_PULL, OUTPUT_PUSH, COMMAND, PROPERTY,
OUTPUT_lO, lNPUT__lO or EVENT. A t may be used with input or output pins.
There may be only one default output pin and only one t input pin per component.
Setting default to "true” may te that this pin is the default pin to use when referring
to this type of pin. The data type definition may define the expected input or output data
type. The pin data type may act as a data type restriction on an input pin. The display
3O name may be the pin name displayed in the visual designer 30. if the display name is
set on a property pin for which the defined property also has a dispiay name, the pin
_ 27 _
display name may appear on the component and the ty display name may
appear as the ty name.
A data type restriction may be used to restrict the type of input a pin can accept.
When an output pin with a data type that does not meet the restriction conditions is
connected to the pin, a warning may be generated. Data type restrictions may override
restrictions based on the pin’s defined data type. Data type restrictions may be
combined using logic operators to create complex restrictions. The syntax of the data
type restriction may foliow prefix notation. Example restrictions e string, number,
integer and so on. Logic operators AND, OR and NOT may be used to create complex
data type restrictions.
A data type transform of a default output pin may be set by the default input pin’s
data type. if the default output pin will be producing output which is different than the
input pin’s data type, a data type transform may be used to change its data type. The
data type transform may remove parts of a data type definition or entire definitions. it
can also add data type ions to a pin data type
The set of components that represent a workflow may be saved as a graph 28.
Data is ulated in data containers 56 that may be passed between connected
components in a graph 28. Data ners 56 may t of raw data as well as meta-
information about the data.
[95] Graphs 28 can be created, saved, and loaded programmatically or with the
visual designer 30. The visual designer 30 is a visual design tool for creating, g
and saving workflows.
Workflows can be saved as graphs 28 or as blueprints 28a. Blueprints 28a are
graphs 28 with attached meta‘data. Graphs 28, ints 28a and components 24 may
be registered into a repository 32 that is used to store, manage, and access
components 24 and blueprints 28a in a controlled environment that may be centralized
or distributed. Repositories 32 may be chained together into managed supply chains
where downstream repositories 32 can be synced with upstream repositories 32.
Access to components 24 may be controlled through a floating license server 42.
A component’s 24 license may be checked out when it is being used.
-23...
Applications that use graphs 28 may submit theirjobs to an agent 34, which may
be a web e interface. The agent 34 may acquire the graph’s 28 components 24
from the repository 32 and launch engines 36 to run the jobs. The engines 36 may load
the graphs 28 or blueprints 28a, and may acquire the es needed to run the job.
Once the job is complete, the licenses may be returned to the license server 42.
A component 24 development steps may include one or more of the ing:
designing the component 24 and its interface, including a determination of which
features and elements the component 24 may need; saving the design that defines the
component 24 in a file for use in graph 28, such as in a pluginxml fiie, where the design
1O of the component 24 may also include the features and elements of the component 24;
writing and storing the ent’s 24 source code. Component definitions may be
used to instantiate components 24 as a component 24 definition may contain all the
ation ed to instantiate a component 24, including ations for
properties, commands, events and static pins.
[100] Examples of properties may e a name, class name, unique identifier (such
as a GUID for example), a description and a category. A component may be declared
with a name, and an example of which is may be a Java class name and a unique
GUID. Upon compiling the pluginxml, a constant of the form NAME_<name> may be
generated. The component’s name may be referenced in source code using this
constant. For example, the constant NAME_AudioMixer may be generated for a
component named “AudioMixer”.
The class name property may reference the component constructor class. For
example, when writing a Java or Dual component, the component’s Java class may be
used, or when writing a C++ component, may use uniform NativeComponent class.
[102] Each ent may have a unique identifier which may be referred to as a
GUlD. The unique identifier may be for component licensing. For example, upon
compiling the pluginxml, a constant of the form GU|D_<guid> may be ted. The
component’s GUID may be referenced in source code using this constant.
The description property may be a description of your component.
[104] The category property may reference the categories to which the component
belongs. Categories may be used for grouping components in a display such as in the
__ 29 _.
visual designer 30. Each category may be defined within its own element. You may
create subcategories by separating the category names with rd s. For
example, if you defined the category as “Company o”, the component would
appear in the Audio subcategory of the Company X category.
A component’s 24 definition may declare pins (dynamic and static), properties,
events, commands, and capabilities. These elements can be used, modified and
managed in the component 24 source code. When a component’s 24 plugin.xml file is
compiled, header and jar files may be generated. These files declare string constants
that correspond to the component elements.
[106] A data container 56 holds the media data that flows between components 24.
The data container 56 may define a data type and data . The data type may be
metadata describing the data container 56 and may include key-value pairs of
information (eg. width, height). The data types may be configured to implement
inherency and hierarchies. Examples of data types include image dimension (width,
height) and pixel format (color space, bits per sample). The data object may be the raw
data, such as in the form of a buffer, string, and so on. Examples of data objects
include file name (string), audio sample (buffer), video frame (buffer), asset XML, and
so on.
A data container 46 may include a amp in on to the media data stored
therein as the data . Media data packets typically need to be associated with a
timeline as they are received and processed to in sequencing and timing.
Including a timestamp for the media data stored in the data container 56 enables non-
linearity of processing and decouples the processing of the media data from the
timeline typically associated with media data. A data container 56 may define a normal
form for the input data to be processed by graphs 28 and ints 28a. A data
container 56 may associate raw data with a data type so that both the raw data and
data type flow as a unit to provide concurrency, multiprocessing, which may enable the
context to switch at the data container 56 boundaries, and so on. Data containers 56
may include an individual timestamp with reference to the raw data to le the raw
3O media data from its state dependent on a timeline. Data container 56 properties may
include read only, write only, and read/write. This may be useful if, for example, a data
_ 30 _
container 56 reaches a branch and needs to be duplicated. One data container 56 may
be marked read only so that the contents cannot be modified while a separate
operation is processing the ts of the duplicate data container 56, for example.
Referring now to there is shown a block diagram of example data
container (metadata, raw buffer) 56a that flows between two components 24, video
process 24p and strip letterbox 24q.
Data may be passed between components 24 as data container 56 objects. A
data container may include a raw data object and a data type object describing the
contents of the raw data. A data type object may be a collection of key/value pairs. lts
keys may be defined in one or more data type definitions defined in plugin.xml files, for
example. Data type definitions may be features such as image dimensions, audio
format, and encoding . Data type definitions may be inherited and extended.
Data types may consist of a set of keys that accept values of a certain type. The
values of the keys can be of either simple or complex types. A simple type may be a
primitive type such as INTEGER, BOOLEAN or STRING, while a complex type may be
a type where the key’s value is itself a data type. Data type tions may be defined
in pluginxml files, for example. A data type tion may include keys and their value
types, and inherited data type definitions. Data type definitions may be defined using
the data type definition schema. A data type definition may have attributes or properties
such a name (used to reference the data type definition), comments (which may include
information about what the data type definition refers to, and how and when it should be
use, which may appear in the Data Type frame of the visual designer help interface),
inherits (a data type definition can inherit and extend other data type definitions), and so
[111] Data type definitions should be decoupled from component definitions. As such,
te .xml files may be used for data type definitions and for your ent
tions. If you have defined your own data type definition, you may need to compile
its plugin.xml file before using it. ing its plugin.xml may generate a header and
class file for each defined data type. These may automatically be exported to an SDK
installation.
_31_
Data type definitions declare a set of keys used to describe raw data. Key value
types are ed in the key definitions. Acceptable values for the keys can be
specified explicitly in the declaration. Examples definitions include channel
configurations, language, and so on.
Key definitions may have attributes or properties. Examples include: simple
type/complex type which indicates whether the key’s type is a simple type (the value is
a primitive type) or a complex type (the value is a data type), key name which may be
used to reference the key definition, key comments which may include a description of
what ty of the raw data the key refers to, the key’s value type which may be
1O simple (primitive type), for example R, STRlNG, BOOLEAN, or complex (a
DataTypeDefinition), where this type should agree with the simpleType/complexType
tag, multivalued which indicates that the key may accept a list of O or more values, such
as for e, the audio_channel_details key may have as many values as there are
audio channels, and enumeration value which ates all possible values for the
key. If a key can only have certain acceptable values, these can be listed as
enumerationValues. EnumerationValues may be referenced in the source code using
string constants of the form ey_name>_<value>. For example, the l80639_‘1
value of the language standard key may be referred to by the constant,
VAL__language_standard_l80639_1.
[114] A plugin package may be a grouping of data type definitions and component
ions defined in a pluginxml file, for example. A plugin package can consist of
more than one component or data type definition. The plugin may also contain libraries,
header files and java files that may be used to load and support its components or data
types.
[115] Data type definitions and components may be distributed in plugin packages.
The framework 12 may be shipped with plugin packages. Additional functionality can be
added to the framework 12 by sing or developing additional plugin es.
Licenses for the framework 12 and its ents may be included in a license
package. A properties file may be included in the SDK 20 package and may be edited
to point to the license server 42.
Components 24, data type tions and plugin packages can be created using
the SDK 20. Each component created may have a unique GUID and each plugin
created may need to be signed. A license package may include GUle and a signing
key.
Plugin package attributes may be defined within .xml files. ent 24
definitions and data type definitions may be defined within the plugin package definition
in the plugin.xml file. The plugin package definition may require a name, a unique
pluginlD, a plugin n, and, optionally, a provider and description. The name is the
name of the plugin, the plugin lD is a name that is unique to each plugin (to guarantee
uniqueness, the pluginlD may be structured using a reverse domain name, for
example), a pluginVersion refers to the plugin version, a provider refers to the
organization providing the plugin, and ption provides a description of the plugin.
This should include the type of components or data type definitions that are distributed
in the plugin package.
[118] All plugin packages may be . Signing guarantees authorship and prevents
unauthorized modification of the plugin package. Signing may happen automatically
when the plugin.xml file is compiled. At compile-time a private key is requested from the
license . A signature is then generated using this private key. A public key, the
plugin certificate, is also generated. When the plugin package is loaded, the certificate
is used to verify that the plugin package has not been modified from build time. If the
plugin package has been modified, or if it has not been signed, it will not load.
Plugin packages may be compiled using a gradle tool. Plugin packages, even
those which do not contain source code, may be compiied to te header files and
files that are used to instantiate their components and data type definitions. Compiling
the plugin package automatically signs your plugin package and installs it in the SDK 20
installation. The framework 12 may use gradie to build plugin packages. A SDK 20
installation may come with template gradle build files (buildgradle) for different types of
projects.
A graph 28 may be a template describing a set of components 24 (including
compound components 26, other graphs 28), the ter values for the components
24, and the connections between the pins of the components 24. A graph 28 may
._ 33 _
define a set of components 24 having specific connections and have specific properties
(with values). A graph 28 may define a set of components having ic connections
and having specific properties. A graph 28 may be referenced within r graph by a
label to rence from the underlying graph 28. This may be useful for versioning, as
described herein.
A blueprint 28a may be a final embodiment of a graph 28 and may reference a
on set of components using a label. A blueprint 28a may be used to instantiate a
graph 28 at application runtime, and may also meta-data such as include business logic
about the graph 28. A blueprint 28a may connect the functionality of a graph to a
1O running environment. A solution set of components 24 may be a set of specific versions
of components. A blueprint 28a may form part of the repository 32. Blueprints may be
viewed as a business level container of graphs 28. A int 28a may include one or
more graphs as part of a single life cycle of the ation, which may be executed
nested or in parallel, or multiple graph 28 stages may be executed in serial form, one
after the other. A int 28a may be a container of one or more graphs 28. A graph
28 can contain other graph328 but all run in one lifecycle, whereas the graphs 28
contained at the blueprint 283 level may run simultaneously, or sequentially.
A graph 28 can be represented as a file (eg. XML file) in its blueprint 28a form
or as dynamically instantiated object code at runtime. Graphs 28 may be viewed as
having two lives, as a running live instance and as a description of how that ce is
saved for communication, transportation, distribution or storage needs. In the live
context, it will be referred to herein as a graph 28. In the description context for
reference in communication, transportation, distribution or storage needs, it may be
referred to herein as a blueprint 28a.
[123] A simple e file is follows:
<pinConnections>
<connecdon>
<sourcePath>File Souroelout<lsourcePath>
<deslinationPath> Bufferfin<ldestinationPath>
<Iconnection>
<connection>
<sourcePath>Bufierlout<lsourcePath>
<destinalionPath>YUV to RGBfin<IdesiinafionPalh>
<Iconnection>
</pinConnections>
A graph 28 and blueprint 28a may contain components 24, compound
components 26, and may contain other graphs 28 (compound graphs 28). A compound
graph 28 can be exported and referenced as a single component 24. The system 10 is
operable to reference components 24, nd components 26, graphs 28,
blueprints 28a and compound graphs 28 in the same manner.
Referring now to there is shown a block diagram of an example graph 28
surrounded by a blueprint 28a (e.g. final ment of the graph 28) in accordance
with an e embodiment. The graph 28 is a compound graph and includes an
outer graph 28 and an inner sub-graph 28, both of which n ents 24 (file
source 24f, interiace detector 249, film removal 24h, deinterlacer 24i, noise reduction
24j, file sink 24x). The components 24 and graphs 28 may be connected by pins.
A graph 28 and blueprint 28a may be used by system 10 to develop and deploy
a media application, and may be loaded from the repository 32 at application runtime. A
graph 28 and blueprint 283 can simultaneously handle source data in one or more of its
possible forms in a ent 24.
As shown in , components 24, compound components 26, graphs 28,
compound graphs 28, and data containers 56 are maintained in one or more linked
repositories 32. A graph 28 may implement a variety of processing roles such as an
installer, manager and executor.
A component’s 24 lifecycle is ted to that of its parent graph 28. Different
component methods may be called during different graph 28 lifecycie states. Before a
graph 28 starts, its components 24 may be instantiated and connections may be made
between them. ents 24 can complete their configuration after they have been
3O instantiated and before the graph 28 starts. When a graph 28 is loaded from file, its
components 24 are instantiated as soon as the graph 28 is fully loaded. Additional
graph 28 and ent 24 configurations may take place after the graph 28 is loaded,
_ 35 _
but before it starts. Once a graph 28 starts, its lifecycle may go through a realize, pre-
process 1, pre-process 2, sources start and sources stop states, for example.
If the graph or one of its components encounters an error, the graph may abort.
When a component 24 has completed its sing, it may move from the
active state to the inactive state. A graph’s 28 lifecycle is done when none of its
components 24 remain in the active state. The graph 28 may be able to keep track of
the state of its components 24 unless these components 28 start their own worker
threads. If a component 24 starts its own worker thread, it is responsible for setting its
own active flag. A set active method may be used for this purpose, for example. Once
all components 24 have become inactive, the graph 28 may enter the Finish state.
Component 24 cle actions include, for example, realize (components load
native libraries and perform self-setup, such as allocating memory and reading
properties), pre-process 1 and pre-process 2 (components send their output data type
information through the graph, and any components that need data type information
block until they e it), sources start (source components start transmitting data,
ents process data coming through their input pins), sources stop (source
components stop transmitting data and processing continues until all data has passed
through the graph), abort (a signal is sent to all components to cease activity and pass
the abort signal to their threads, and threads may exit their run loop as soon as
possible), and finish (all ents are inactive and all data transmission and
processing has d).
If a component 24 needs to perform lifecycle-related actions, they may need to
implement the appropriate lifecycle method or function. Example component life cycle
methods include post initialize, post load from document, life cycle realize, life cycle
pre—process1, life cycle pre—process 2, life cycle sources start, life cycle s stop,
process, life cycle abort, and life cycle finish.
Post initialize may be called after the component 24 has been instantiated, while
the graph is still in the l state. it may be called to complete the configuration of the
component by adding ts that can only be added once the ent is
initialized. Post lize may be implemented to create a complex data type restriction,
~36—
add a property change listener, set the data type on an output pin, dynamically add pins
to the component, or perform ty validation, for example.
Post load from document may be called after a saved graph has finished g
and while the graph is still in the initial state. Post load from document may be
implemented to configure a component based on its connections to other components
in the graph, for e.
Life cycle realize may be the first method called when the graph is started. There
may be no data passing through the graph when iife cycle realize is called, so the
component may only have access to its own properties and to data types. Life cycle
realize may be implemented to create a worker thread (worker thread is started in
sources start), or get and/or verify properties that are set when the graph starts, for
example. If a property is only read during the realize state, any s made to the
property value while the graph is running may not be picked up or used by the
component. The changes may be picked up and used in subsequent executions of the
graph.
Life cycle ocess 1 may be called once the graph is started and all
components are instantiated and configured. Empty data containers, consisting of only
their data types, may be sent through the graph to prime components with run-time
data type information, such as image sizes and video frame rates. Source ents
implement life cycle pre-process 1 to send their empty data containers h the
graph. Life cycle pre—process 1 may be implemented to provide run-time data type
information to other components. With regards to life cycle pre-process 2, as data type
information is passed through the graph to prime components, the components that use
the data type information can block in life cycle pre-process 2until they receive the
ation they need to m their configurations. Life cycle pre-process 2 may be
implemented to block until your component receives data type information needed to
complete its configuration or perform its processing, or block sending data through the
graph until the graph is completely configured, for example.
For life cycle sources start, once the graph has been primed and all the
components are configured, data can begin g through the graph. Components
can start pushing data through the graph in life cycle sources start. Life cycle sources
_ 37 _
start may be implemented to transmit data through the graph, or start a worker thread,
for example.
If source data running through the graph is stopped through external methods
(timed broadcast, user-controlled streaming), life cycle sources stops may be called
when a signal to stop the source is detected. Any source clean—up (stopping threads,
closing files, etc.) should be implemented in this method. Life cycle sources stops may
be implemented to stop the source stream based on an external event, for example.
The process method may be called any time data arrives on a component’s input
pin. The process method is where all data processing occurs. The process method is
where data is retrieved from input pins and placed onto output pins. The process
method may be implemented to transform data, retrieve data from input pins, push data
onto output pins, change or set properties, and so on.
If an error occurs in the graph, life cycle abort may be called. Life cycle abort is
used to stop s and close files. Life cycle abort may be implemented to stop
worker threads, or close files, for example.
Life cycle finish may be the final method called in the graph iifecycle. It may be
called when no components remain in the active state. Any final clean-up needed to be
done should be implemented in this method. Life cycle finish may be implemented to
close files, release allocations, close socket tions, or wait for internal threads to
finish, for example.
The repository 32 is operable to manage versioning of components 24, graphs
28, and ints 28a in order to keep track of updates, variations, and modifications
made to components 24, graphs 28, and blueprints 28a. The repository 32 is operable
to handle runtime libraries and engines used by graphs 28, blueprints 28a, and
components 24, such that the repository is self-managed with t to versioning.
The repository 32 is further operable to receive the development framework 12 to
manage versioning of and updates to the pment framework 12. That is, the
tory 32 can load up-to-date versions of the development framework 12, including
e libraries and engines. The pment framework 12 may be loaded upon
t so that appropriate and updated versions of the development framework 12 are
used. The graphs 28 and ints 28a are loaded at run time so that the appropriate
._ 38 _
version of the graph 28 and each component 24 in the graph 28 is used. A blueprint
28a may reference a solution set of components. A solution set of components is a set
of components 24 and specific versions of each component 24 in the set. The blueprint
28a may nce a solution set of components using a label. A blueprint 28a may
reference a solution set using a label in order to dereference from the specific
components 24 and versions of the solution set. That way, if the solution set changes,
such as if a component 24 is added or removed from the solution set, or a version of a
component 24 changes in the solution set, then the same label will reference the
updated solution set without requiring cation to the blueprint 28a containing the
label. This may result in more nt processing as a reduced number of
modifications and updates are required. r, components 24 may be self-contained
and isolated from a dependency point of view. The entire ency set of a
component 24 may be self-contained, being specified and packaged in the component
distribution unit (e.g. plugin). The component 24 dependencies may also be isolated,
referring exclusively to the specific ent 24 and version(s) they depend on. This
may enable the system 10 to realize complex workflows while ing components 24
dependencies without user ention. Further, the dependency isolation may allow
the system 10 to provide distinct behavior while ing blueprints built with the same
components 24 by ing the different versions of these components 24 and their
dependencies. Processing errors may also be reduced as the system 10 and user may
not have to manually track and ly update components defined by blueprints 28a
or graphs when a label is used.
Referring now to there is shown a block diagram of an example interface
100 for defining a solution set of components in accordance with example
embodiments. in this example, the interface 100 displays different types of components
102 along one axis and different versions 104 of each type of component along another
axis. ln this e there are 6 different types of components 102 and each type may
be associated with a component identifier such as for example, c1, c2, c3, c4, 05, c6.
System 10 may use the component identifier to reference a particular type of
component.
-39..
There may be multiple versions 104 of each type of component, or some types
of components may only have one version. For example, a type of ent 102 ci
may have 8 different versions. Each version may be associated with a n identifier
such as for example: c1v1, c1v2, c1v3, c1v4, c1v5, clv6, civ7, c1v8. System 10 may
use the version identifier to reference a particular version of a specific type of
component.
A solution set 106 references a set of components, and more particularly may
reference a ic version of each type of component for use by a computing
ation. In this example, the solution set 106 is the specific version of each type of
component that is intersected by a line (c1v4, 02v3, c3v4, c4v1, c5v2, c6v1). The
solution set 106 may be modified by changing a version of a specific component, by
removing a ular type of component, by adding a new type of ent, and so
on. The interface 10 may provide a mechanism for a user to efficiently modify, test, and
deploy different versions of components by changing the solution set. For example, the
interface 100 may change a solution set 106 by sliding a new version of a component to
intersect with the line, g all versions of a type of component line so that no
versions of a ular type of component ect with the line (Le. which may indicate
that a particular component is no longer part of the solution set), and so on. System 10
is operable to test and deploy a modified solution set 106 for a particular computing
application, and can also test all updates made to a solution set 106.
A blueprint 28a is operable to reference a particular solution set 106 using a
label to dereference from the specific components and versions of the solution set. If
the ts of a solution set 106 changes then the int 28a label will reference
the changed solution set 106 without requiring modification to the blueprint 28a.
Multiple blueprints 28a may reference the same solution set 106. A blueprint 283 may
also reference multiple solution sets 108. if a change is made to the solution set 106
and a label is not used to dereference from the specific ents and versions of the
solution set, then multiple blueprints 283 may require updating and tracking as
referencing the solution set 106 in order to ensure that the blueprints 28a nce the
appropriate components 102 and versions 104 thereof.
Also, in accordance with some embodiments, a solution set may itself have a
version number. The version number can be nced by the label of the int
28a to identify which version of the solution set the blueprint is working with. In other
embodiments, a blueprint may ignore the version number and tically update to
refer to the latest version of the solution set. The solution set version number may
provide a mechanism to maintain and create a history of solution sets as changes are
made thereto.
The development framework 12 enables complexity abstraction by defining an
entire graph 28 or blueprints 28a as a component 24. A component 24 which defines an
1O entire graph 28 may in turn be used in another graph 28 or blueprint 28a, such that a
graph 28 may be embedded within another graph 28 or blueprint 28a. The graph 28 or
blueprint 28a may nce the ent 24 using a label to dereference from the
specific instance of the ent 24. That way, if the component 24 is modified then
the label of the graph 28 or blueprint 28a will reference the modified component 24
without requiring additional modification to the graph 28 or blueprint 28a. That is, the
graph 28 or blueprint 28 will tically update to reference the modified component
24 by virtue of the label reference. The development framework 12 also provides
properties redirection through the ability to expose a component 24 property as a
property of an enclosing graph 28. The development framework 12 further enables
modular and recursive construction of graphs 28 through the use of pre-packaged
graphs 28 and components 24.
Commands, along with events, provide a means for components, graphs and
calling ations to communicate outside of the regular data flow. Commands are
functions that may be called on components. A command may be executed by passing
the command name and an argument (in the form of a data container 56) to a process
command . A data container 56 may be passed to process command ,
even if the command will not be using the data. lf the command will not be using any
data, an empty data container 56 may be passed to process d on.
Command pins may act as triggers to call commands on components. When a
data container 56 is placed on a command pin, the process command function may be
called with the command name (the name of the pin) and the data container 56 as
__ 41 ._
arguments. The data in the data container 56 may be used by the command, but it
does not need to be. Commands may be defined within the component definition file,
such as a plugin.xm| file for example. Commands may have attributes or properties,
such as a name (which may be used to access the command) and description (which
may include information such as what the command does and any data the d
expects including data type. An example command may be “read” (name) which reads
a certain number of bytes starting from a particular location and takes a data container
with the number of bytes and a seek position (description).
Events, along with commands, may provide a means for components, graphs
and their calling applications to communicate outside of the regular data flow. Events
may follow an event listener pattern where ents fire events to registered event
ers implementing a node event er interface. A node event may include the
event name (String) and raw data (Object). The identity of the component that fired the
event may also be contained within a node event.
[152] Events may be exposed on pins. When an event , the data object
generated by the event may be encapsulated in a data container that is pushed onto
the event pin. if the event generates a null object, an empty data container may be
placed on the pin. The propagation of a data container on an event pin may signal that
the event has occurred. Events may be defined within a ent’s definition file,
such as for example a plugin.xml file. Events may have attributes or properties, such as
a name (which may be used to access the event) and description (which may include
ation such as under what circumstances the event is fired and whether the event
contains any data
Capabilities may be externally defined cts that define how a component
with a particular lity should behave and appear. A capability definition may
include information such as the ties and pins that are expected in a component
with this capability. Capabilities are intended for components, graphs and applications
to request components dynamically based on their functionality. A component may
declare that it implements more than one capability. Capabilities are declared in the
component definition file, such as for example in a pluginxml file. Capability names
may be unique.
- 42 _
Data processing in a component can follow either a push or pull paradigm. in the
push paradigm, data is pushed onto a component‘s input pin, calling that component’s
process method. In the pull paradigm, a component will request data from its input pin
on a separate internal thread. The internal pulling thread will block until data is received.
When a graph is started, source components may send a priming data container
consisting only of data type information through their output pins. The data type
information in these "empty” containers may be used by components downstream to
complete their configuration. A source component will initially produce an empty data
container. Once the empty data container has been sent h the graph, real data
1O will be output. All components should check for empty data containers arriving on their
input pins. When an empty data container is ed on an input pin, the component
should validate the data type information and send out an empty data container of its
own consisting of its output data type. An application programming interface class may
provide convenience s for pushing containers, ing empty ners onto
output pins.
g mutable data containers may allow components to perform data
processing without making copies of the data. However, ce modifications should
only be done if the data is not being used elsewhere. if the data will be used in two
locations (threads, components, methods, etc.) at once, it may be made immutable. if
an output pin feeds into multiple input pins, the same data container will be passed to
each input pin; the data container will tically become immutable to prevent the
receiving components from modifying the same object.
A component may call clone if immutable method on the input data container if it
will be modifying the data container’s data object as well as its data type. The method
clone if immutable returns a clone of the data container if the data container is
immutable; ise it returns the data container itself. The method clone if immutable
will make a full copy of the entire data object in the data container, potentially using a
substantial amount of memory. if the component only needs to modify the data type
within the data container, then clone if immutable should only be called on the data type
before it is modified.
-43..
All stream data type definitions may inherit from a base stream data type
definition (data type stream). The stream data type tion includes an end of stream
key that indicates whether or not this data container is the last. Marking end of stream is
important to signal that no other data will be ng for sing. All stream source
components may set end of stream to true on the last data container they send. The
end of stream data container can be empty (no data object). Stream processing
components may check for end of stream in the data Type of each data ner they
receive. When they receive the end of stream data container, they must in turn send
out an end of stream data ner.
[159] In the push data processing model, data gets pushed onto the component’s input
pin, calling that component’s s method. The process method is effectively called
by the component pushing the data onto the input pin. The push model is a e
model. Data containers are “pushed” onto the input pin by the previous component in
the w. A process method may be called whenever a data container arrives on
the input pin. The component may not be active unless it is processing data.
The push model may be used in cases where there is only one pin, or in cases
where the input of multiple pins do not need to be coordinated. If the data containers
arriving on multiple input pins need to be coordinated, then the pull model may be used.
in the pull model, the component will block until either a data container arrives on
the input pull pin or a timeout expires. A worker thread may be used to drive pulling
data from the input pins, processing the data, and pushing output to the next
component. Methods may be called on puli input pins to pull the data, blocking until
either a data container arrives on the pin, or a timeout expires. if the timeout expires
before a data container s on the pin, a null data ner may be returned.
[162] To prevent a pull component from entering a busy loop, it is preferable to block
until data arrives rather than until a timeout expires. However there are cases where a
timeout is necessary, for example if one pin only receives sporadic input. if one pin will
not always receive data, the component can use a timeout on this pin to allow it to
continue its processing without this pin’s input. Unlike a push processing component, a
pull processing component needs to be aware of when to stop processing. An end of
stream data container can be used for this purpose. if the parent graph is aborted, a
.. 44 ..
null data container will be returned by the pull method. Pull components may need to
verify whether their parent graph has aborted whenever they receive a null data
container.
Visual design subsystem
The visual design subsystem 30 is operable to output graphs 28 and blueprints
28a for developing media applications using components 24, nd components
26, blueprints 283 and other graphs 28. The visual design subsystem 30 defines
onships between components 24, compound components 26, and graphs 28 using
pins to define the connections.
[164] In one example embodiment, the visual design subsystem 30 may be accessible
via a cloud computing . The visual design subsystem 30 may allow a user to
create components 24 and graphs 28, and define an order of operations or workflow for
the graph 28. The visual design subsystem 30 may allow the user to group components
24 (including compound ents 26 and graphs 28) into onal blocks and
arrange those functional blocks into specific orders of operation. The visual design
subsystem 30 further allows the construction of logic branches which allow for flexibility
in execution of the components 24 or onal blocks. The visual design subsystem
may also allow for the uction of functional blocks which may e linearly in
time, non-linearly, or as discrete operations with te lifecycie management.
[165] The visual design subsystem 30 defines a graph by connecting components 24,
compound components 26, and graphs 28 using connection mechanisms such as pins.
The visual design subsystem 30 allows parameters for the components 24 to be
set and monitored. The visual design subsystem 30 may also allow graphs 28 to be
instantly reviewed to test functionality and performance. The visual design subsystem
30 may simplify component 24 and graph 28 testing, development and deployment.
The visual design subsystem 30 may provide an ace, such as interface 10
of , in order to define solution sets of components 24 and versions thereof for
use in graphs 28, blueprints 28a, and other components 24. The visual design
subsystem 30 is operable to test and deploy a solution set for use in graphs 28 and
-45..
blueprints 28a. The visual design subsystem 30 is operable to test and deploy a n
of a component 24 for use in a solution set for graphs 28 and blueprints 28a.
Referring now to there is shown a block diagram of an example interface
for a visual design subsystem 30 in accordance with an example embodiment. The
example interface for a visual design subsystem 30 includes a graph 28 and
components 24 (file input 24u, color space converter 24v, logic branch 25 with routing
based on image width and height, scaler 24w and AVC encoder 24y). Other example
components 24 include YUV to RGB, java image ller, scripted component, flow
l component, and so on. The example interface for a visual design subsystem 30
rates an interface 80 for setting the properties 25 and values 27 for components
The visual design tem 30 outputs a graph 28 or blueprint 28a, which may
be stored in the repository 32 for subsequent use and reference. For example, the
visual design subsystem 30 may output a file (eg. XML file) which describes a graph
28, components 24, compound components 26, blueprints 28a, and compound graphs
28. The file describes the components 24 that are used, the parameter , and the
connections between the pins of the components 24. A graph 28 may be used as part
of media applications and may be loaded by the system 10 at run time to ensure the
appropriate components 24 of the graph 28 are used. For example, a graph 28 may
reference a solution set of ents 24 and versions thereof. The solution set may
change or update, and e the graph 28 references the solution set by label the
appropriate solution will be loaded by the system 10 at run time. The repository 32
maintains a collection of graphs 28, blueprints 28a, and components 24. The tory
32 manages versioning of components 24 and graphs 28 to keep track of updates
made to components 24 and graphs 28, and new versions thereof. The graphs 28 are
loaded at run time so that the appropriate version of the graph 28 and each component
24 in the graph 28, as defined by a solution set for example, is used. Further, the
graphs 28 may reference the solution set by label so that if the solution set is d
the graph 28 will automatically reference the changed solution set without requiring a
manual update to the graph 28. That is, the blueprint with the label may automatically
reference the d solution set without requiring a manual update. A solution set
_ 46 _
may be referenced by different blueprints 28a using the same or different labels. For
example, a user may configure a blueprint 28a with a label for a solution set, such as
“ready for testing” or “passed testing” and r user may configure the same or
different blueprint 28a with a different label for the same solution set, such as “MY
SET”, for example. The label provides a ptive mechanism for a user and also
provides efficient processing and propagation of updates. The label may continue to
reference a solution set even if a modification is made thereto. Labels may also be
used to reference components 24, blueprints 28a, graphs 28, and so on. Different
labels may be used to reference the same components 24, blueprints 28a, graphs 28,
and so on.
The visual design subsystem 30 may export a blueprint 28a or a graph 28. For
e, the blueprint 28a may be instantiated on a desktop platform as a local engine
or subset of an application, or in the cloud by a cloud engine 36. A blueprint 28a may
be considered to be a final embodiment of a graph 28. A blueprint 28a and a graph 28
reference a solution set of components and ns thereof using a label.
The visual design subsystem 30 may be an engine or object code that can be
run through an application interface or through the set of SDKs. The visual design
subsystem 30 is le to construct graphs 28, test graphs 28, m run time
validation, and simulate graphs 28.
[172] The visual design subsystem 30 may perform design time media inspection and
ate media type, data container ation and component configuration
changes across graphs 28 and blueprints 28a thus validating proper realization of the
graph 28 and blueprint 28a into a media application that can process the desired type
of media. For example, labels may be used to reference solution sets so that if the
solution set changes then label used in the blueprints 28a will also reference the
updated solution set without requiring the blueprint 28a to be updated. The visual
design subsystem 30 enables complexity abstraction by defining an entire graph 28 or
int 28a as a component 24. Accordingly, data containers 56, components 24,
compound ents 26, graphs 28, and blueprints 28a may be generally referred to
herein as components 24, and may be used like components 24 as building blocks for
computing applications.
_ 47 _.
The visual design subsystem 30 may e ties redirection through the
ability to expose a component 24 property as a property of an enclosing graph 28. The
visual design subsystem 30 enables modular and recursive construction of graphs 28
through the use of pre-packaged or pre—constructed graphs 28 and components 24.
The visual design subsystem 30 uses the repository 32 to provide graph 28 and
blueprint 28a persistence storage and versioning strategy enabling backward
ible changes. The visual design tem 30 provides dynamic, override-able
and decoupled user interface support.
Referring now to there is shown a block diagram of the data flow of a
system 12 for dynamic development and deployment of computing applications, in
accordance with an example embodiment.
The system 12 may include a user system 14 delivering a plug in package that
may contain one or more components 24, graphs 28, and blueprints 28a. The system
12 is also shown to include a tory 32, agent 34, engine 36 and an application
system 15.
ents 24 may be stored in a repository server 32. The repository server
32 manages the components availability, versioning and OS/platform capability. When
a new job is running the agent 34 will contact the repository server 32 to acquire the
components 24 required by the engine 36 which will be running the graph 28/blueprint
288.
Components 24 may be delivered to the repository server 32 as Plugin
Packages. The Plugin Packages contain one or more components 24 of related
functionality. The Plugin Packages may also include graphs 28 or blueprints 28a for
example. Note that each Plugin e may also be signed and have a
cturer’s digital certificate. Third party Plugin Packages may require a certificate
with their company’s identifier before the package may be recognized by the repository
32. This certificate may be provided by a certification agent, as will be described in
relation to .
The visual designer 30 may provide a graphical interface that can be used to
create new graphs 28, or to create new compound components 26 based on existing
ents 24. Compound components 26 include ents 24 embedded within
.. 48 _.
other components 26, and may be referred to herein simply as components 24.
Components 24 are the basic data processing elements. Components 24 may have
input pins (which allow data to enter the component), output pins (which allow data to
leave the ent) and settings (which allow the user to set some of the
parameters/properties which define what s to the data when it is processed by
the component). Compound components 26 can be created using existing ents
24 and these compound components 26 can be saved as new components 24.
A graph 28 is a set of connected components 24. Components 24 are connected
via their pins. Data is encapsulated and passed between components in data
containers 56. A data container 56 may be comprised of a data object (the raw data
that is being processed) and a data type (meta-information about the raw data).
A graph 28 can be a specific workflow solution or a graph 28 can be embedded
within r graph 28 as part of a more complex workflow. Complete ow
solutions can be saved to the repository 32 as blueprints 28a.
Deployment Subsystem
The deployment subsystem 14 may include one or more linked repositories 32, a
license server 42, cloud agents 34 on user computing systems, cloud engines 36 run by
the cloud agents 34, a job manager 50, and a security module 46. The deployment
subsystem 14 provides al interfaces 38 to tories 32 to manage components
24, blueprints 28a and graphs 28, to the job manager 50 to manage application jobs,
and to cloud s 36 to manage the execution of graphs 28 and blueprints 28a.
The deployment subsystem 14 may include a computing application used to
manage the workflow and to define the graphs 28. This application may optionally
provide access to a graph creation tool such as the visual designer 30. The deployment
subsystem 14 may include an agent 34 which may exchange commands and status
between the application and engines 36. One agent 34 can communicate with more
than one engine 36. The ment subsystem 14 may include an engine 36 which is
operable for running components 24 in a graph 28.
The deployment subsystem 14 may include a license server 42 used by engines
36 to check in and out licenses for the purchased ents 24. The license sewer
_4g_
42 may also be used to enable the application. The deployment subsystem 14 may
include a tory server 32 used to store the components 24 that are to be deployed
on engines 36.
There may be two types of deployments: alone/desktop deployment; and
network deployment.
Referring now to there is shown a block diagram of stand-alone
deployment. In this type of deployment the application 47 accesses the development
framework 12 APl directly. All of the components of the deployment can be installed on
a single host system 49. That is, the local disk is used to store components 24, graphs
1O 28 and blueprints 28a. Alternatively, the repository 32 may be used instead of the local
disk to provide a se of plugin packages. The repository 32 can be used by more
than one host system 49. The license server 42 can be installed on the host system 49
for a true “stand alone” deployment, or it can be led on the network so that it can
be accessed by more than one host system 49, to allow for network licensing.
[186] Referring now to there is shown a block diagram of network deployment.
in this type of ment an agent 34 is required to icate with the higher level
management application 55 and to communicate with the engines 34. The agent 34
may reside on one to n different host systems 51, 53. Access to a repository 32 may
be required for network deployment.
[187] An agent 34 may be the ch/coordinating service installed on all host
systems which will run engines 36. An agent 34 may coordinate management and
monitoring of systems on the network and dispatches and monitors jobs running on
engines 36. An agent 34 communicates with higher level applications (for example, job
manager 50) through a web es interface, for example. Agents 34 may include a
communication service and a server service. Agents 34 can coordinate management
and monitoring of more than one engine 36 or a mix of engines 34 on the same system,
the only restriction may be the practical limits of the host ’s resources (cpu,
memory, bandwidth, etc).
An engine 36 is a g version of a graph 28 or blueprint 283. An engine 36
may access source files, write output files and return status to the agent 34 or Kayak-
based application. An engine 36 communicates with the agent 34 to acquire the
.. 50 ..
required components 24 and with the license server 42 to authorize the components 24
required to run the graph 28.
The repository 32 stores components 24, compound ents 26, blueprints
28a and graphs 28. As one example, the repository 32 may be a web services based
repository accessible h a cloud computing system via external interfaces 38. As
another example, the repository 32 may be stored on a local system. The deployment
subsystem 14 may use one or more linked repositories 32 for version management,
maintenance and deployment. The repository 32 is a hosted collection of components
24 and graphs 28 which are accessed by a protocol, identified as required, and
transferred to the target host environment. As an illustrative analogy, a graph may be
viewed as a recipe (i.e. template) listing different ingredients (i.e. ents) and the
repository 32 contains the int 28a for the graph 28 and components thereof to
provide the user with both the “recipe” and the “ingredients” listed in the “recipe”.
The repository 32 organizes each component 24 dless of r it is a
standalone component 24 or is a compound component 26, graph 28, blueprint 28a, or
solution set with reference to other components 24) with t to revision (i.e.
versions of the component), ownership structure, licensing requirements, and
dependencies on other components 24 or logies. These dependencies or
requirements may further e specific revisions of technologies or ents 24
for proper function.
The repository 32 manages versioning such that it can ine the most
appropriate version of a component 24, graph 28, and blueprint 28a. The appropriate
version of a component 24 may be defined by a solution set. The repository 32 allows
access to any of the available versions of components 24 and graphs 28, which
include the most recent version but necessarily. The repository 32
may interact with
interface 10 in order to provide available versions for each component and define
solution sets. For example, a customer may want a version of a component 24 that they
have tested instead of the latest version, and may include the tested version in the
solution set. This is may be ant for downstream management. When a graph 28
and blueprint 28 thereof is used by an application the components 24 defined by the
solution set referenced by the label in the blueprint 28a or graph 28
are loaded from the
_ 51 _
repository 32 at media application runtime so that the proper n of the components
and graphs are used. The repository 32 is red to provide versioned components
24 with multistage lity, automatic component update propagation and gated
component update release.
Referring now to there is shown a block diagram of an example user
interface 90 for a repository 32 in accordance with an example embodiment. The
example interface 90 for the repository 32 displays an address 91 for the tory,
such as a uniform resource locator. The example interface 90 for the repository 32
displays a listing 92 of names 93 of components 24, compound components 26, and
graphs 28, along with an associated description 96, provider 95, and version 94. The
listing 92 may also include an associated status, such as complete, tested, and so on.
The interface 90 may also include the interface 10 of .
There may be multiple linked repositories 32a, 32b and a media application
access the multiple repositories when a graph 28 or int 28a is used by the media
application at e. Examples of repositories 32 include staging, preproduction, and
production.
A cloud agent 34 may be provided to a user computing system to manage the
local resources of the host computing system. The term ’cloud’ as used herein
describe a heterogenous environment where agents can live in the cloud
or on
desktops, laptops, mobile s, and so on, and is not limited to ‘cloud computing
systems’ accessible through the lnternet. That is, a cloud agent 34 may also refer to a
desktop agent, local agent, and so on. The cloud agents 34 may ct with cloud
engines 36 to execute graphs 28 and blueprints 28a thereof in order to run media
applications, or other computing applications. At application runtime, a pool of one or
more cloud agents 34 can access a shared tory 32 of components 24 and graphs
28 to construct the application. A cloud agent 34 is operable to instantiate blueprints
28a ofa graph 28 and run them in a cloud engine 36.
A cloud engine 36 provides a running environment for blueprints 28a of
graphs
28 and creates media applications on the blueprints 28a of the
graph 28. The term
3O ' as used herein may describe a heterogenous nment
where engines can
live in the cloud or on desktops, laptops, mobile devices, and
so on, and is not limited to
_ 52 _
‘cloud computing systems' accessible through the lnternet. That is, a cloud engine 36
may also refer to a desktop engine, local engine, and so on. The cloud engine 36 is a
runtime construct which receives blueprints 28a of graphs 28, analyzes and organizes
component 24 dependencies, executes protocols for retrieval of the required
components 24, constructs those components 24 into new run—time executables and
dispatches those executables against a c job or process. The dispatch of new
me executables can be persistent or dynamic in nature. ln persistent mode the
cloud agent 34 ers the availability of cloud engines 36 with the calling server or
application and no further deployment (installation) is required. ln dynamic mode each
executable can be ‘renewed’ at each job instantiation creating a new ‘product’ with each
deployment.
The cloud agent 34 can be implemented as a desktop application or a cloud
based ation (using an external interface 38). For a cloud based ation, the
cloud agent 34 may be required to manage the cloud engine 36 and provisioning for
specific components 24, graphs 28, blueprints 28a and other resources. For the
p application, a dynamic link library may be used and the system SDK 20 may
allow for dynamic updates of components 24 and graphs 28.
The cloud engine 36 is operable to coordinate the e server 42 and the
repository 32. The cloud agent 34 is operable to dispatch, manage, and run
independent, ted onality on a single host system.
The cloud agent 34 is operable to provide interfaces and control over lifecycle of
onal blocks of components.
The cloud agent 34 is operable to monitor, aggregate, and report information
about the environment in which it is running to allow for maximum zation and
balance of work.
A cloud engine 36 is operable to execute a graph 28 or blueprint 28a thereof at
application runtime in order to construct and deploy a media application, or other
computing application. At application runtime a cloud engine 36 is operable to use a
blueprint 28a of a graph 28 and the solution set referenced in the blueprint 28a in order
to identify components 24 and other graphs 28. Further as a facilitator for version
resolution, components 24 may be ontained and isolated from a dependency point
.. 53 _
of view. The entire ency set of a component 24 may be self-contained, being
specified and packaged in the component distribution unit (e.g. plugin). The component
24 dependencies may also be isolated, referring exclusively to the specific component
24 and version(s) they depend on. This may enable the system 10 to realize complex
ows while resolving components 24 dependencies without user ention.
Further, the dependency isolation may allow the system 10 to provide distinct behavior
while executing blueprints built with the same components 24 by isolating the different
versions of these components 24 and their dependencies.
The cloud engine 36 is operable to send a request to the repository 32 for the
identified components 24 and graphs 28, receive a copy of the components 24 and
graphs 28 from the tory 32, and dynamically build a media application using the
components 24 and graphs 28. Cloud agents 34 run the cloud engines 36. A cloud
agent 34 is operable to instantiate blueprints 28a of graphs 28 and run them in a cloud
engine 36.
[202] A cloud engine 36 is registered with a shared repository 32 and dispatched by
job manager 48. The shared repository 32 works similar to a local repository but its
contents are shared by a pool of cloud agents 34. The job manager 50 dispatches
blueprints 28a of graphs 28 cloud agents 34 referencing available es in the
license pool 44 as ined by the license server 42.
[203] The cloud agent 34 may provide life-cycle management services for the cloud
engine 36 which in turn manages the ents 24, blueprints 28a and graphs 28.
The cloud engine 36 is operable to control all components in a multi-threaded and
multi—process execution environment and to manage initialization. The cloud engine 36
may enable early ation of data type information. The cloud engine 36 may
provide graceful and non-graceful termination.
The cloud engine 36 is operable to provide component configuration services for
graph 28 execution. The cloud engine 36 is le to provide the ability to auto-
configure ent 24 settings based on the input data type avoiding unnecessary
user input.
-54..
The cloud engine 36 is operable to provide the ability to ure individually
each input pin to function according to a push or pull model allowing heterogeneous
components 24 to connect to e the graphs (blueprints).
The cloud engine 36 is operable to provide memory management services
through memory pools, garbage collection and lifecycle management for large data
objects.
The cloud engine 36 is operable to manage data communication pathways in
between components 24 allowing them to connect and pass data to realize the
blueprints 28a.
[208] The cloud engine 36 is le to define generic media data type and metadata
model (video, audio, time code, subtitles, closed captions), a specific application
domain data dictionary, a mechanism to encapsulate data and data-type information
with data packets for richer information and optimizes data container management The
cloud engine 36 is operable to e hierarchical data-type representation of the
information occurring in the graph. The cloud engine 36 is operable to provide ype
transformation gies to ease component manipulation of data-types.
The cloud engine 36 is operable to provide hreaded data integrity through
immutable only) packets and data access performance optimization, components
altering ble’ packets in—place, copying only read-only data.
[210] The cloud engine 36 is operable to provide out of process execution support,
thus enabling blueprints execution in separate processes, while managing large data
ures transfer, inter process communication and transparent shared memory when
possible.
The cloud engine 36 is operable to provide support for multi-language
component development with communication and interoperability between them.
The cloud engine 36 is operable to provide cross platform application execution
support, allowing graphs to be executed on multiple types of platforms, ing
Windows, Mac, Linux platforms, for example.
The license server 42 is le to dynamically manage a license pool 44 of
licenses and associate licenses with components 24 and graphs 28. The license server
42 is operable to determine whether a requesting user has the appropriate license for
_ 55 _
the components 24 identified in a graph 28 that forms part of the media application. A
user may only be permitted to use components 24 and graphs 28 if they have the
required and appropriate license. This allows a user to use the technology across
departments, groups and companies depending on the conditions of the license
associated with the various components 24 of the graphs 28. Further, this enables a
provider to control and track use of its components 24 and graphs 28. The license
server 42 provides tracking of all ‘in use’ technology and provides for a central
accounting mechanism. The licenses can be controlled by concurrency, physical
system, floating, and leased.
1O [214] That is, the license server 44 provides runtime authorization to ents 24
through a pool of available licenses. Referring now to , there is shown an
e browser based console 41 which can be used to access the license server 44
to show which features are available, how many are currently in use, and which
systems are using the features. The license server 44 can access and/or import plug-in
package 43 of licenses. Engines 36 can access the license server 44 to check out
licenses for the components 24 ed to run a graph 28 and to check in those
licenses once the graph 28 has finished running. An application system 45 can access
the license server 44 to check out licenses for the components 24 required to run an
application and to check in those es once the ation has finished running.
[215] The job manager 50 is configured to e job/engine dispatch, failover,
tracking and reporting. The job r 50 dispatches cloud engines 36 based on
available resources, system bility, processing capability, available es in the
license pool 44 maintained by the license server 42. In particular, the job manager 48
dispatches cloud engines 36 based on the latest or riate graph blueprints 28
registered with production repository 32 and available licenses in the license pool 44.
The job manager 50 may also be configured for mapping graphs 28 to cloud engines
36. The job manager 50 may also be configured to provide the highest level of
access
to the running cloud engines, and provide centralized access to the cloud engines 36
less of state (running or not). The job r 50 may further self-extend
interfaces (e.g. web services) based on the graph 28/blueprint 28a that is loaded on the
cloud engine 36 to e a namespace (for example, similar to the web) which
_ 55 ..
allow the developer to discover which graphs 28 and components 24 are used in that
particular computing application, query parameters, set parameters, and so on.
Referring now to there is shown a block m of an example interface
60 for a job manager 50 in accordance with an example embodiment. The interface 60
provides a g 62 of resources managed by the job manager 50 including a start
time, end time, resource name, status (eg. running, completed, failed, cancelled),
progress, average encoding rate, estimated time remaining, failures, source file, project,
and notes. The interface 60 may also include a search box 64 for searching forjobs
managed by job manager 50. The search box 64 may provide a variety of search
1O parameters such as date range, project name, resource, group, current state, and
events (eg. dropped, failover), for exampte. The interface 60 is le to provide a
variety of pages or windows, such as a y, network monitor, groups, resources,
schedule, jobs, and alerts, for example.
The ty module 46 provides for secure connections and communications
within system 10.
A code signing module 40 is operable to digitally sign each component 24 to
ate a developer, license, or both with each component 24.
The translation module 58 is operable to translate multiple languages into a
common language for system 10.
[220] An interface application may provide users with a way to create graphs 28 and to
run those graphs 28. The graph 28 creation may be programmatic, where a graph 28 is
generated based on a few user selected ters, and the actual graph 28 itself is
hidden from the user. At the other end of the spectrum the interface application may
provide full access to the visual designer 30, with the user choosing and connecting the
components in the graph 28 manually. The interface application may also provide a way
to select the data inputs for the graph 28 (e.g., source files), to set the outputs for the
graph 28 (e.g., archive files), and to monitor and control the ion of the graph 28.
An example of an interface application is job manager 50 with job engines 34.
The job manager 50 may be a media r server which s file transcode
jobs. User access to the Media Manager Server may be via an interface application.
Jobs are submitted to the server by adding source files to watch s. Watch folders
_ 57 ..
are associated with job projects (graphs 28 for transcoding jobs). Graphs 28 may be
created using a customized version of the visual designer 30. The Media Manager
Server may have access to a pool of transcode host systems, and each transcode host
system may communicate with the Media Manager Sewer using an agent 34 installed
on the host. When a job is submitted the source file and project (graph 28) are sent to a
host system with an agent 34 which will then manage the engine 36 which processes
the job. Status is returned to the Manager Sever while the job is being processed and
when the job tes.
Referring now, to there is shown a block diagram of the data flow of a
1O system for dynamic development and deployment of media applications, in accordance
with an example embodiment. A int 28a is a container of one or more graphs 28.
A graph 28 can n other graphs28 but all run in one lifecycle, whereas the graphs
28 contained at the blueprint 28a level may run simultaneously, or sequentially. Cloud
agents 34 and cloud engines 36 may be operable to e a blueprint 283 and use it
to instantiate a graph 28 of components 24, compound ents 26, and data
containers 56.
The normalization module 52 is operable to receive input media files 54 (which
may be files as in the original document, iive media and so on), and convert and parse
the input media files 54 into data containers 56 to be processed by the graph 28/
blueprint 28a. The normalization module 52 extracts as much data as possible from the
input media file 54 to populate the data containers 56 and the data type and data
s of the data containers 56. The normalization module 52 can match the input
data to a dictionary of languages linked to data types in order to populate the data type
ent of the data containers 56. Normalization module 52 capability may be
distributed across various components (being actually provided by c components
or for example a media file input component).
System 10 may be implemented as a cloud computing system and the user may
access system 10 through al interfaces 28 (such as web services for example).
Referring now to FIGS. 9 and 10 there is shown block diagrams 70, 80 of
example web services implementations in accordance with example embodiments.
—58—
As shown in web services 72 may connect and ct with a broadcast
system 74, post broadcast system 76, and cable/IPTV system 78. Web services 72
may e a virtual layer between external systems (e.g. service system 74, post
processing system 76, cable/lPTV system 78) and the components of system 10. The
web services 72 interact with job manager 50, which in turn dispatches and
manages
one or more cloud engines 36. The job manager 50 may also interact with e
server 42 and license pool 44 to comply with license ctions. The web services 72
may be provided as a cloud computing system. One feature of embodiments bed
herein is automatic generation web services 72 based on the components that exist in a
1O running engine. The web es 72 can be further filtered through access control by
the author/designer of the application.
As shown in , web user interfaces 81 may connect and interact with
or more service system(s) 86, post processing system(s) 85, and other external
systems 87 (e.g. cable/lPTV system). Web user interfaces 81 may provide a virtual
layer between external s (e.g. e system(s) 86, post processing system(s)
85, other system(s) 87) and the components of system 10 (referred to as virtual
appliances 84). in some embodiments, some web user interfaces 81 may interact with
an application 82a to access the components of system 10. In some embodiments,
ss logic residing on web servers 83 is operable to control interactions between
web user aces 81 and the components of system 10.
An example embodiment may implement an asset management and publishing
system that is responsible for implementing actions including storing, conforming,
searching, and hing large amounts of media data to individual publishing profiles.
A hing profile can be a VOD target, a device, a service, and
a network, for
example. The asset management and publishing may be implemented as a web
application.
Referring now to FIGS. 11 and 12 there is shown diagrams of example data
flows for implementing an asset management and publishing system in accordance with
example embodiments. At 200, system 10 is operable to ingest digital media assets
(such as input media files). At 202, system 10 is operable to determine whether the
digital media assets meet codified requirements and use job r 50 to dispatch
.. 5g _
cloud s 36 for processing and modifying the l media assets to meet such
codified requirements. At 204, system 10 is operable to store digital media . in
some embodiments, the job manager 50 may be used to manage the storing of the
digital media assets. At 206, system 10 is operable to search through the digital media
assets for refinement based on search parameters. At 208, system 10 is operable to
publish the processed and d digital media assets to a customer by using the job
manager 50 to dispatch corresponding cloud engines 36 for preparing (e.g. advanced
video coding) and publishing the digital media assets to specific customers at 210.
Referring now to there is shown a block diagram of an example
certification system 160 in accordance with example embodiments. The certification
system 160 may certify or sign a solution set, graph 28, component 24, blueprint 28a,
and so on (referred to generally herein as components) with digital certificates 142, 132
to indicate acceptance that the particular component will m or
carry out a function
as expected, by one or more user computing systems 140 associated with the particular
component and one or more component provider systems 130. illustrates
multiple user ing system 140, each associated with different media applications
developed and deployed using system 10. further illustrates multiple
component provider systems 130, each having provided resources such as one or more
components for use by system 10 or one or more hardware resource used to execute
computing applications, for example.
A user ing system 140 may be any ked computing device operated
by a user of system 10 including a sor and memory, such as an electronic tablet
device, a personal computer, workstation, server, portable computer, mobile device,
al digital assistant, laptop, smart phone, WAP phone, an ctive teievision,
video display terminals, gaming consoles, and le electronic devices
or a
combination of these. A networked device is a device capable of communicating with
other devices and components of system 10 and certification system 160 through
communication network such as network 152. A network device
may couple to the
communication network through a wired or wireless connection. rly,
component
provider system 130 maybe any networked computing device ed by a resource
provider including a processor and memory
__ 50 _.
Network 152 may be any network(s) e of carrying data including the
Internet, Ethernet, plain old telephone service (POTS) line, public switch telephone
network (PSTN), integrated services digital network (ISDN), digital subscriber line
(DSL), l cable, fiber optics, satellite, mobile, wireless (e.g. Wi-Fi, WlMAX), SS7
signaling network, fixed line, local area network, wide area network, and ,
including any combination of these.
The user computing systems 140 and the component er systems 130 use
digital certificates 132, 142 to indicate that they agree that a particular component
operates properly. That is, the digital certificates 132, 142 signify acceptance by both
the user computing system 140 and the component provider 130 that a particular
component satisfies a performance standard. For example, the digital cate 142
may sign a particular ent when a user computing system 140 tes a digital
button “I agree” linked to a digital representation of a e agreement. it may be
important to track and record acceptance by users and providers to efficiently resolve
disputes and to ensure only accepted components are used in the computing
applications so that the computing application functions as agreed. It may be ant
that the functionality performed by the application or the deliverable (what the
application creates or transforms) can be tracked, agreed upon, etc. A digital certificate
may be an onic file that identifies an individual or organization and electronically
indicates to system 10 that the individual or organization accepts a component. A digital
certificate may be issued by system 10 or by a third party cate issuer. A digital
certificate may have security es to authenticate the individual or organization
associated therewith so that the digital certificate is not used lently.
The certification system 160 may extend to multiple user computing systems
140, each with a digital certificate 142 for signing components, blueprints, and so on.
The certification system 160 uses digital certificates 132, 142 to create a 'chain of trust’
between all aspects of the system 10. Trusted components may create trusted graphs
(which may include trusted or untrusted components). A graph may become a trusted
graph when signed by multiple user computing systems 140 and provider systems 130
to create trusted exchange of trusted graphs (blueprints) for use within a media
application. That is, components may be signed using l certificates 132, 142, and
_ 51 _
the signed components may be used to create graphs and blueprints. The graphs and
blueprints may also be signed using l certificates 132, 142. Those signed graphs
and blueprints may form part of a computing application, and system 160
may check to
ensure all of the computing application components are signed (i.e. accepted by a user
and er) prior to executing the computing application.
As an example, a user computing system 140 may use system 10 to develop a
media application involving a particular component and may test and deploy the
particular component to ensure it functions ly. Once the user computing system
140 agrees that the particular component satisfies a performance standard (i.e.
functions properly) then the user computing system 140 can indicate acceptance using
a digital certificate 142 associated with the user computing system 140. The use of
digital certificate to indicate acceptance enables the system 10 to automatically and
efficiently track and check the acceptability of media ations and components
f. The user computing system 140 signifies acceptance by signing the
component with the digital certificate 142 associated with the user computing system
140. rly, the component provider 130 signifies acceptance by signing the
component with a digital certificate 132 ated with the component provider 130,
The license sewer 42 includes a certificate ent matrix 146 which
manages records relating to digital certificates 132, 142. in ular, a record links the
digital icates 132, 142 and the accepted resources, such as components, graphs,
computing applications, hardware resources used to execute computing applications,
and so on. A component may be used in multiple computing applications associated
with different user computer systems 140, where each user computer system 140 is
ated with a different digital certificate 142. Accordingly, the certificate
component
matrix 146 may include multiple records associated with the same
component, where
each record links the component to a ent digital certificate 142 associated
with a
different user er system 140.
in accordance with some embodiments, the computing application is deployed
internally by system 10 or externally by a remote computing . For example, the
remote computing system may be cloud based infrastructure. The
remote computing
system may be operated by system 10 or may be operated by a third party, for
_ 52 _
example. A cloud engine 36 or a cloud agent 34 may query the license server 42 to
ensure that a component has been signed by both digital certificates 132, 142 before
executing a component at runtime as part of a media application associated with the
user computing system 140. If the component has not been signed by both digital
certificates 132, 142 then the cloud engine 36 or the cloud agent 34 may not execute
the component and may provide an error message requesting that the component be
signed by the digital certificates 132, 142. The cloud engine 36 or the cloud agent 34
may submit a query that es both the user computer system 140 associated with
the media application and the component. The cloud engine 36 or the cloud agent 34 is
1O operable to verify that the relevant user ing system 140 has accepted the
component, as a component may be associated with a different user computing system
140. Further, the cloud engine 36 or the cloud agent 34 is operable to verify that the
component er 130 has accepted the component before deployment.
Further, the license server 42 may include a record database 150 which stores a
record or report each time the resource operates successfully, such as when a
component successfully executes within the computing application, when a signed
hardware resource successfully executes the computing application, and so on. The
record establishes that the resource operated successfully (eg. a component or
blueprint executed successfully) in the event of a e between the user computing
system 140 and the component provider 130. The license server 42 may generate
summary of all records associated with a resource for provision to the ent
provider 130, system 10 or user computing system 140. The summary provides a
mechanism to notify the component provider 130 or user computing system 140 that
the resource is or is not successfully operating.
[239] Using certification system 160, system 10 is operable to supply some signed
components (signed with digital icates) to guarantee a certain level of functionality.
A signed component may establish trust regarding performance and functionality.
For example, a gas company may e user computing system 140 and
create a ing ation (including a int 28a, graph 28, ents 24,
and so on) which contains some signed components ed to perform
a particular
job, such as a component configured to perform job data mining on a large database of
_ 53 ._
information about potential drill sites, geophysical information for each of these sites,
potential risks associated with each site, costs, government limitations, environmental
liabilities, and so on. The company may not have the processing resources to perform
the computations of the computing application in a required timeframe and may use a
remote computing system, such as a cloud based infrastructure for example, to execute
the computing application in order to perform the computations. The company may
want a guarantee of a performance level in relation to the execution of the computing
application by the remote computing system.
The system 10 may engage third parties, such as a component er system
130 (eg. vendor of components 24), to provide the remote computing system, such as
the cloud based infrastructure. The system 10 and component provider system 130
may be ated with a e level agreement that guarantees performance of the
remote computing system provided by the ent provider system 130. In order for
the gas company operating the user computing system 140 to trust system 10 to run
their computing applications there may be a chain of trust established between the
component er system 130, the system 10, and the user computing system 140.
Accordingly, there may be more than two digital certificates signing the computing
application (or components 24, blueprints 28a, graphs 28 thereof). System 10 may use
its own digitai certificate to sign to the computing application to guarantee that it
functions exactly as the gas y associated with the user computing system 140
requires. In addition, the user computing system 140 may use their digital certificate
142 to sign the computing application and the component provider 130 (which may also
be referred to as a service provider as it may provide the cloud computing resources in
this example) may use their digital certificate 132 to sign the media application. In
effect, instead of offering remote ing system resources (such as raw cloud
resources for example), the system 10 may be viewed as offering a “Workflow As A
e“ in some examples. The system 10 may not know exactly what job the media
application is performing only that it ons in the remote computing system
tructure properly. The l certificates 142, 132 provide a chain of trust and
acceptance between the parties, a security mechanism and a validating reference point
that the s can feel confident about. Accordingly, in this example, the gas company
_ 64 _
operating the user computing system 140 signs the application (or blueprint, graph,
component thereof) with their digital certificate, the system 10 countersigns the media
application with their digital certificate, and the component provider system 130 signs
the media application with their digital certificate. The system 10 is operable to only
execute the blueprint only if all l signatures (made by the digital certificates) are
valid. Security may be derived from the digital certificates and chain of trust established
thereby, as only signed components, blueprints and graphs may be executed by system
in some embodiments.
As system 10 generates reports each time the computing application (and
blueprints, graphs, and components thereof) successfuily execute and those reports
are stored by license server 42 in the record database 150. A results summary may be
generated and transmitted to the user computing system 140. The chain of trust is
ined so that the gas company operating the user computing system 140 can trust
the results of the media application, third party service, etc. and the fact that the data
and blueprints have been protected throughout the entire process.
Other ial types of computing applications and ts include voter
registration, integrity management, and vote tally applications.
For e, a computing application may include a int defining a
workflow at place at each voting station. The computing application
may be a trusted
application using trusted (is. ) components, running on a trusted system 10.The
user (voter) ers their data and signs the data using their digital certificate and the
system 10 adds a user specific certificate to the data. At voting time, the trusted user
data is interjected into the trusted ow and the vote is recorded by system 10. All of
the voter ns involved send trusted data into the remote computing system
hardware infrastructure ing the computing application where trusted
processes
are running inside of a trusted environment. The remote computing system solution
provider (e.g. component provider system 130) countersigns the computing application
implementing the workflow with their l ure, the data coming back is secure
(is. ), the public and government can trust the system 10 because it is secure
(i.e. signed) , and the results are trusted because of the nature of the system 10. The
signing of a component may involve encrypting the component to enhance security.
_ 55 _
The use of digital cates in the described example embodiments differs from
a traditional https style of key exchange. For an https system, when a user accesses a
website, say a bank, the user can trust the bank because the certificate at the bank
website is ied against that particular system. in a remote environment, such as a
cloud environment, where the system providing the remote hardware infrastructure may
be unknown, and using the described embodiments, the digital certificates may be used
to secure the s implemented by the computing applications. The described
embodiments may use l certificates to sign the entire media ation, ing
the individual components and entire workflows implemented by blueprints 28a, and
even multiple stages of workflow.
As r example, a media company may sign a media application provided
by system 10 as a Workflow As A Service. The media company may operate the user
computing system 140 to access system 10 and p blueprints (workflow) for the
media application. System 10 or a third party, such as a media company association,
may certify that the workflow (blueprint or media application) is secure when all involved
companies sign the blueprint using their digital certificates. The signatures makes all
parties feel protected, and ensures with some level of ility that the media will be
secure, the processes will be secure and operate as expected, and the chain of trust is
secure.
[247] A process flow associated with the certification system 160 may include the
following steps in accordance with an example embodiment.
A user computing system 140 may be provided, via network 140, with access to
the system 10, which includes a development framework 12 with a software
development kit 20, components 24, data containers 56, pins, graphs 28, blue prints
28a and so on, in order to construct a ing application. The software
development kit 20 may be used to define the components 24, data containers 56, pins,
graphs 28, blue prints 28a, and solution sets (each lly referred to as
components). Each component defines a computing processing mechanism for
processing data containers 56 of data at application runtime. Each graph 28 or blueprint
28a comprises a reference to a solution set of components, where the solution set of
components is a set of ular versions of components.
_ 66 _
The user computing system 140 may be provided, via network 140, with access
to the visual design subsystem 30 configured to define and output graphs 28 and
blueprints 28a in order to develop ing ations. The visual design subsystem
is operable to arrange components into functional blocks and define specific orders
of operation for the functional blocks. The user computing system 140 may use the
visual design subsystem 30 in order to define solution sets for blueprints 28a and
graphs 28 using interface 10.
The user computing system 140 may be provided with a digital icate 142
associated therewith. The certificate 142 may be provided by system 10, and in
particular by license server 42. The certificate 142 may also be ed by a third party
certificate provider.
The component provider 130 provides one or more resources, such as
components to system 10 for use by the user computing system 140 to develop
components and computing applications. Other example ces include signed
computing appiications and hardware resources to execute computing applications. The
component provider 130 is provided with a cate 132 associated with the ed
components. The certificate 132 may be provided by system 10, and in particular
license server 32. The certificate 132 may also be provided by a third party certificate
provider.
[252] The user computing system 140 may use the component provided by the
component provider 130 for one or more of their media applications. As described
herein, the user computing system 140 may require a e to use the component, as
managed by the license server 42 and the e pool 44.
The user ing system 140 may be provided, via network 140, with access
to the deployment subsystem 14 for deploying the computing applications including the
component. As described herein, the deployment subsystem 14 includes a repository
32, cloud agent 36, cloud engine 34, and other modules. The computing application
may identify graphs 28, blue prints 28a, compound components 26, and components
24, including the component ed by the component provider 130. The repository is
configured to store graphs 28, blueprints 28a, and components 24 for ioading at
application runtime.
_ 57 ._
The user computing system 140 may use the deployment tem 14 to test
and deploy the component provided by the ent provider 130. if the user
computing system 140 is satisfied that the component provided by the component
provider 130 functions as expected then the user computing system 140 may accept
the performance by signing the component using the digital certificate 142 associated
therewith. The license server 42 receives the digital certificate 142 from the user
computer system 140 via network 152 and creates a record in the certificate component
matrix that links the signed component with the digital certificate 142. Similarly, the
component provider 130 may accept the mance by signing the component using
the digital certificate 132 associated therewith. The license server 42 receives the digital
cate 132 from the ent provider 130 via network 152 and updates the
record in the certificate component matrix to also link the signed component with the
digital certificate 132.
A cloud engine 36 provides a running environment for the media applications
(and the graphs 28 and blueprints 28a f) and executes the graphs 28 and
blueprints 28a at runtime to instantiate the media application. The cloud agent 34
controls the cloud engine(s) 36. At runtime, the deployment subsystem 14 dynamically
constructs and deploys a computing application by sending a t at runtime to the
tory 32 for the graphs 28, ints 28a, compound ents 26, and
ents 24 identified in the media applications. The ment subsystem 14,
and in particular the cloud engine 36 or the cloud agent 34, is operable to query the
license server 42 at runtime to ensure that the component of the computing application
has been accepted by both the user computing system 140 and the component
provider 130 prior to executing the component and running the computing application.
The license server 42 is operable to respond to the query by checking the certificate
component matrix 146 for a record that links the component to the user computing
system 140 and the component er 130. If the component has not been accepted
an error message may be provided requesting acceptance.
Each time the component of the computing application successfully executes the
cloud agent 36 or the cloud engine 36 may provide a record or report of the successful
execution to the license server 42.
_ 68 _
The job manager 50 is operable to store the record of successful execution in
the record database 150 in association with the component, the user computing system
140 or the component er 130. That way, if a dispute arises in on to the
operation of the component the job manager 50 can provide a copy of the record to
establish that the component did or did not execute successfully to resolve the dispute.
The license server 42 may know whether or not ic technology resources are in
use, may not know r or how the technology resources used was actually
successful.
in accordance with some ments, system 10 is operable to provide
support for mixed architectures. This may provide increased flexibility as typically a
process needs to be compiled for the same architecture binary. For example, a 32 bit
CODEC library would typically have to run on a 32 bit context, and typically could not
run in a 64 bit context or with a 64 bit y. in accordance with some embodiments,
system 10 is operable to develop and deploy an application ce which combines
components 24 written for both 32 bit and 64 bit architectures. System 10, and in
particular cloud engine 36a, is operable to detect whether a particular media application
has been developed using both components 24 for different architectures, such as
components 24 for 32 bit architectures and components 24 for 64 bit architectures, for
e. If so, system 10 is operable to create a separate process space or instance
for each context and handle inter process communications using mapping and a shared
memory. For example, the system 10 is operable to create a 32 bit architecture process
instance and a 64 bit ecture s ce and manage communications
between the process instances.
Referring now to there is shown a block diagram of a mixed architecture
in accordance with example embodiments. A graph 28 may contain components 24
developed for different architectures, such as components 24 for 32 bit architectures
and components 24 for 64 bit architectures, for e. System 10, and in particular
cloud engine 36a, is operabie to detect whether a particular graph 28 includes
components 24 for different architectures. ln this example, graph 28 contains
components 24a, 24b, 24c, 24d developed for 32 bit architectures and r contains
a subgraph 28’ with components developed for 64 bit architectures. System 10 is
.. 69 _.
operable to create a separate s space or ce of graph 28’ for the 64 bit
context and handle inter process communications using mapping and
a shared memory
to receive and provide input and output. The subgraph 28’
may run out of process for
example.
In accordance with some embodiments, system 10 is operable to provide
ive scalability, through dynamic ioning, and deployment based
on the
individual workload of a component 24, group of components 24,
or entire graph
28/blueprint 28a. System 10 is operable to analyze the runtime processing of a
blueprint 28a and break down the blueprint 28a into te running graphs 28 based
on derived knowledge of the sing overhead (e.g. bottlenecks) of particular
components which exist in a particular workflow. System 10 is further operable to
isolate those component(s) 24 and partition the blueprint 28a into le graphs
(e.g.
create new graphs 28) on the fly which can exist on a separate host system (or the
same host system) while maintaining the communication and ity of the al
workflow/blueprint 28a. For example, system 10 is operable to use a separate host
system with more resources to process to overhead/bottlenecks and manage data flows
between. The modular nature of components 24 and graphs 28 enable
system 10 to
partition a graph 28 into multiple graphs and run them separately on different computing
resources.
[261] Referring now to there is shown a biock m of a graph 28
partitioned into two graphs 28, where each is run on a separate host system 300, 302.
System 10 is operable to identify a processing bottleneck (i.e. graph 28') in a
graph 28
running on a host system 300, isolate the components 24 associated with the
processing bottleneck, and create a new graph 28’ on the fly which can exist on
separate host system 302. The separate host system 302 provides additional
resources
to process graph 28. System 10 is operable to
manage communications and data flow
(eg. via a shared memory) between the host systems 300, 302 and graphs 28, 28’.
. In accordance with some embodiments, system 10 is operable to provide
security through dynamic re-location of graphs 28 and components 24 that make
up a
particular computing application. The process is r to as described above in
relation to selective scalability except that a graph is ioned for the purpose of
_. 7o _
sed security, as opposed to the purpose of isolating a processing bottleneck. For
example, security module 46 may interact with license server 42 to determine whether a
ular graph 28 for a media application refers to components 24 (or embedded
graphs 28) that are signed. it so, system 10 may partition the graph 28 for the media
application by ing the signed components 24 for increased security by obfuscating
the original footprint of the media application. The partitioned components
may then run
on a separate host system (similar to that shown and described in ). System 10
is operable to further limit access to the running footprint of a ular blueprint 28a
and relocate sections of the blueprint 28a onto different hosts in a dynamic fashion.
This may be viewed as 'scrambling' the application footprint at the functional level
making it harder to find and compromise. For example, an attacker may focus on one
host to monitor and listen, so splitting a process onto multiple hosts
may make it more
difficult for an attacker to monitor and tamper with the program. The al
functionality is maintained as system 10 manages communications and data between
the multiple host systems. Security is maintained as the
process of creating the new
sub-graphs 28 may involve automatic injection of secure messaging ents (e.g.
tion). As an ion of this, system 10 is further operable to create multiple
ces of each new sub-graph 28 and to randomize the data paths h the
running ents 24 of the sub-graphs 28. Further, system 10 may be operable to
maintain a set of potential host systems and randomize selection of
a subset of those
host systems on which to run the program, so that the host systems
are also randomly
selected.
In accordance with some embodiments, system 10 is operable to
control access
to resources using virtual priority management. That is, system 10 is operabie
to tune a
media application to control ty of processing, parallelization and threading.
runtime, system 10 is operable to manage the execution of a component 24 in such a
way as to make it process faster or siower than normal. There may be an nce of
resource utilization between components 24 in a media application, and system 10 is
operable to manage the processing prioritization of a particular component while other
components 24 are prioritized independently. For example, if a more important
component 24 is running then system 10 is operable to manage, control, manipulate
_ 71 ..
throttle (i.e. slow down) another component that may be consuming a larger amount of
resource until the more important component 24 has completed its processing.
As an illustrative example, the virtual priority management may be implemented
using a virtual clock as a mechanism to control priority. A virtual clock is one example
and the implementation could be done a number of different ways.
As noted above system 10 is le to limit resources allocated to a particular
component 24. For example this may be limiting component access to a thread pool,
memory, or other ism. System 10 may throttle the data while not touching
anything. An example may be a source component such as a complex signal
generator. The signal generator may be the first component in the pipeline and may
generate frames faster than they can be consumed but while doing so can also use
some amount of CPU. lf system 10 decides at e to limit the CPU activity the
system 10 may simply trigger the signal generator less often. This may not require any
manipulation of threads, memory, or source data s. Another example may be
something at the end of the pipeline that is designed to send out notifications or
s but is also blocking while doing so. The component may send email
ations and the act of sending those notifications takes longer than the rest of the
pipeline does in actual processing. System 10 may limit the number of ations by
throttling the packets that normally r the component to start execution.
[265] In accordance with some embodiment, components 24
may be ontained
and isolated from a dependency point of View. The entire dependency set of
component may be self-contained, being specified and packaged in the component 24
distribution unit (plugin). The component 24 dependencies
may also be isolated,
referring exclusively to the specific component 24 and version(s) thereof they depend
on. Thus the system 10 may be able to realize complex ows while resolving
components dependencies without user intervention. Further the dependency isolation
may allow the system 10 to provide distinct behavior while executing different solution
sets (blueprints 28a) built with the same components 24, by isolating the different
versions of these components 24 and their encies.
[266] As described herein, graphs 28 and blueprints 28a are le and
may be
packaged to run anywhere there is a cloud agent 34.
_ 72 ..
In accordance with some embodiments, components 24 and graphs 28 may
support promotion of ties and values. For example, if one component 24 is
embedded within another component 24, the inner component 24 may promote one or
more of its properties to the outer-component 24. A user may pass expressions to
ents 24 to change/promote ties. That is, properties may be reauthored as
they are promoted. Properties may be selectively promoted in that not all properties
need to be promoted together. Properties may be promoted without exposing the
values, and without exposing the values of the properties that are not promoted.
Embodiments have been described herein in relation to media applications as an
1O illustrative example. The system 10 and s described herein may be used to
develop and deploy other type of re applications and are not limited to media
applications, such as natural resource applications, voting applications, and so on.
Embodiments have been described here by way of example only. Various
modification and ions may be made to these exemplary embodiments.
We
Claims (17)
1. A system for dynamic deployment of computing applications comprising: one or more linked repositories storing blueprints, graphs, and components; one or more processors to receive a d to deploy at least one ing application to s at least one input data stream, and in response, trigger deployment of the at least one computing application over a ity of disparate host systems; the at least one ing application realized by a blueprint of the blueprints stored in the one or more linked repositories; at least one host system of the plurality of disparate host systems comprising a cloud agent and one or more cloud engines instantiated by the cloud agent; wherein the cloud agent is configured to receive the command to deploy the at least one computing application to process the at least one input data stream and, in response, instantiate the one or more cloud engines on the respective host system and provide a g nment for the one or more cloud engines; wherein the one or more cloud engines is configured to dynamically construct the at least one computing application on the respective host system by realizing requirements of the blueprints of the at least one computing application, the requirements identifying at least one graph from the graphs stored in the linked one or more tories, a plurality of components from the ents stored in the linked one or more repositories, and by sending a request at runtime to the one or more linked repositories to load the blueprint, the at least one graph and the plurality of components on the respective host system; wherein the one or more cloud engines deploys the dynamically constructed at least one computing application on the respective host system by: instantiating the at least one graph using the blueprint, the at least one graph enting a workflow of the plurality of components, the workflow defining an arrangement of the plurality of components; detecting that the plurality of components comprise a first set of components written for a first architecture and a second set of components written for a second ecture; representing at least a portion of the at least one graph as a first subgraph and a second subgraph; using the first subgraph to define connections between the first set of components using pins and the workflow of the at least one graph, and using the second subgraph to define connections between the second set of components using additional pins and the workflow of the at least one graph, each ent of the first and second sets of components having at least one input pin for ing at least one input data container and at least one output pin for providing at least one output data container, the respective component transforming the input data ner into the output data container using a computing sing mechanism, each component being a bution plug—in unit to provide a portable and isolated dependency set for the computing processing mechanism of the respective component; connecting the second subgraph to the first subgraph using at least one additional input pin and at least one additional output pin to pass at least a portion of the data containers between the first subgraph and the second ph, the connected first and second subgraphs maintaining the workflow of the graph; instantiating a running process instance of the first architecture on the respective host system for the first set of components of the first ph; instantiating a running process instance of the second architecture on another host system of the disparate host systems for the second set of components of the second subgraph, the running process instance of the first architecture being linked to the running process instance of the second architecture by the one or more cloud engines at deployment time to link the first subgraph and the second subgraph; and processing the input data stream as a plurality of data containers using the ity of components to generate an output data stream, the plurality of data containers flowing between the plurality of components using the pins; using the at least one additional input pin and the at least one onal output pin to pass at least a portion of the data containers between the running process ce of first architecture on the respective host system and the g process instance of the second architecture of the other host system to generate the output data stream.
. The system of claim 1, wherein each of the plurality of components is associated with one or more versions, wherein the at least one graph of the at least one computing application comprises a label as a reference to a solution set of components, wherein the solution set identifies a set of components and a version for each component in the set.
. The system of claim 1 or claim 2, further comprising a license server, wherein the license server cally manages licenses and ates licenses with the plurality of components and the at least one graph, wherein use of the plurality of components and the at least one graph at application runtime requires the associated license.
. The system of any one of claims 1 to 3, further comprising a job manager, wherein the job manager dispatches the blueprint and the graph to cloud agents based on available licenses d by the license server.
. The system of any one of claims 1 to 4, further comprising a security manager, n the security manager provides for secure connections and communications between the disparate host system components.
The system of any one of claims 1 to 5, further comprising a job manager configured to provide job and cloud engine dispatch, failover, tracking and reporting.
The system of any one of claims 1 to 6, wherein each graph in the one or more linked repository identifies components, connections between the ents, and properties for the components, wherein components are connected by different types of pins.
The system of any one of claim 1 to 7, wherein the data container defines a data type and a data object, wherein the data type is metadata bing the data container and the data object maintains raw data.
The system of any one of claims 1 to 8, wherein the one or more linked repository manage versioning of the components and the graphs to keep track of updates made thereto, wherein the one or more repositories serve the components and graphs at ation e using appropriate versions of the graphs and the components.
10.The system of any one of claims 1 to 9, wherein the cloud agent is provided to each of the ate host systems to manage the local resources of the respective host system, wherein the cloud agent interacts with the one or more cloud engines to instantiate the graph using the blueprint.
11 .The system of any one of claims 1 to 10, further comprising a normalization module operable to receive input data files and convert and parse the input data files into a plurality of data ners for processing by the at least one computing application.
12.The system of any one of claims 1 to 11, further sing a code signing module operable to digitally sign each of the plurality of components to associate at least one of a developer and license with at least one component.
13. The system of any one of claims 1 to 12 further comprising: - a l certificate associated with a component provider subsystem, wherein the component provider subsystem es the plurality of components; ~ a digital certificate associated with a user computing subsystem, wherein the user computing subsystem transmits commands to define the at least one computing application, wherein the at least one computing application involves a component of the plurality of components provided by the component provider computing ; — a e server configured to digitally sign the component of the plurality of components by linking the component to the digital icate associated with the user computing subsystem and the digital certificate associated with the component provider subsystem to indicate that the user computing system and the component er subsystem accept performance of the digitally signed component; wherein at runtime prior to deploying each component the deployment subsystem queries the license server to ine whether the component is linked to the l icate associated with the user computing subsystem and the digital certificate associated with the component provider subsystem.
14.The system of any one of claims 1 to 13, wherein the deployment subsystem is further ured to partition the graph into a plurality of sub-graphs and handle interprocess communications n the plurality of sub-graphs.
15. A method for dynamic deployment of computing applications: — providing one or more linked repositories storing blueprints, graphs and components; ~78- configuring one or more processors to e a command to deploy at least one ing application to process at least one input data stream, and in response, trigger ment ofthe at least one computing application over a plurality of disparate host systems, the at least one computing application realized by a blueprint of the blueprints in the one or more linked repositories; uring at least one host system of the ity of disparate host systems with a cloud agent and one or more cloud s instantiated by the cloud agent; receiving, at the cloud agent, the command to deploy the at least one computing application to process the at least one input data stream and, in response, instantiating, using the cloud agent, the one or more cloud engines on the respective host system and providing a running environment for the one or more cloud engines; using the one or more cloud engines to dynamically construct the computing application on the respective host system by realizing requirements of the blueprint of the at least one computing application, the requirements identifying at least one graph from the graphs stored in the linked one or more repositories, a plurality of components from the components stored in the linked one or more tories, and by sending a request to the one or more linked repositories to load the blueprint, the at least one graph and the plurality of components on the respective host system; deploying using the one or more cloud engines the dynamically constructed at least one computing application on the respective host system by: instantiating the at least one graph using the int, the at least one graph representing a workflow of the plurality of components, the workflow defining an arrangement of the plurality of components; detecting that the plurality of components comprise a first set of ents written for a first architecture and a second set of components written for a second architecture; representing at least a portion of the at least one graph as a first subgraph and a second subgraph; using the first subgraph to define connections between the first set of components using pins and the workflow of the at least one graph, and using the second subgraph to define connections between the second set of components using additional pins and the workflow of the graph, each component of the first and second sets of components having at least one input pin for receiving at least one input data container and at least one output pin for providing at least one output data container, the respective component transforming the input data ner into the output data container using a computing processing mechanism, each component being a distribution plug-in unit to provide a portable and isolated dependency set for the computing processing mechanism of the respective component; connecting the second subgraph to the first ph using at least one additional input pin and at least one additional output pin to pass at least a portion of the data containers between the first subgraph and the second subgraph, the connected first and second subgraphs ining the workflow of the graph; instantiating a running process instance of the first architecture on the respective host system for the first set of components of the first instantiating a running s instance of the second architecture on another host system of the disparate host s for the second set of components of the second subgraph, the g process instance of the first architecture being linked to the running process instance of the second architecture by the one or more cloud engines at deployment time to link the first subgraph and the second subgraph; and processing the input data stream as a plurality of data containers using the plurality of components to generate an output data stream, the plurality of data containers g n the plurality of components using the pins; using the at least one additional input pin and the at least one additional output pin to pass at least a portion of the data containers between the running process instance of first architecture on the respective host system and the g s instance of the second architecture of the other host system to generate the output data stream.
16. The method of claim 15, further comprising: providing a digital certificate associated with a component provider subsystem, wherein the component provider subsystem provides at least a n of the plurality of components; providing a digital certificate ated with a user computing subsystem, wherein the user computing subsystem sends control commands to define the at least one computing application, wherein the at least one computing application involves a component of the portion of the plurality of components ed by the component provider computing system; providing a license server configured to digitally sign the component by linking the component to the digital certificate associated with the user computing tem and the digital certificate ated with the component provider subsystem to indicate that the user computing system and the component provider subsystem accept performance of the digitally signed component; receiving, at a license server, acceptance of the component provided by the component provider subsystem in the at least one computing application associated with user computing system by receiving the digital icate from the user computing tem and the digital certificate from the ent provider computing system; —81— — linking, at the license , the component provided by the component provider subsystem in the computing application associated with user computing system to the l certificate from the user computing subsystem and the l certificate from the component provider computing system; and — at application runtime prior to deploying each component, querying the license server to determine whether the component is linked to the l certificate associated with the user computing tem and the digital certificate associated with the component provider subsystem.
17.Computer—readable storage storing executable program code which, when executed by one or more computing processors, implements the system of any one of claims 1 to 14 or the method of claim 15 or 16. —82—
Applications Claiming Priority (5)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201161531953P | 2011-09-07 | 2011-09-07 | |
US61/531,953 | 2011-09-07 | ||
US201261598670P | 2012-02-14 | 2012-02-14 | |
US61/598,670 | 2012-02-14 | ||
PCT/CA2012/000820 WO2013033824A2 (en) | 2011-09-07 | 2012-09-06 | Systems and methods for computing applications |
Publications (2)
Publication Number | Publication Date |
---|---|
NZ623020A NZ623020A (en) | 2015-11-27 |
NZ623020B2 true NZ623020B2 (en) | 2016-03-01 |
Family
ID=
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10515205B2 (en) | Systems and methods for determining trust levels for computing components | |
US10216490B2 (en) | Systems and methods for computing applications | |
US10310824B2 (en) | Distributed ledger platform for computing applications | |
CA2939379C (en) | Systems and methods for partitioning computing applications to optimize deployment resources | |
US10628578B2 (en) | Systems and methods for determining trust levels for computing components using blockchain | |
CA2939400C (en) | Systems and methods for controlling branch latency within computing applications | |
EP3502871A1 (en) | System and methods for determining trust levels for computing components using blockchain | |
Gedik et al. | A model‐based framework for building extensible, high performance stream processing middleware and programming language for IBM InfoSphere Streams | |
Hnetynka et al. | Comparing the service component architecture and fractal component model | |
CA3099814C (en) | Distributed ledger platform for computing applications | |
NZ623020B2 (en) | Systems and methods for computing applications | |
Palmer et al. | Eksten et al.(43) Pub. Date: Aug. 28, 2014 | |
Schmeling et al. | Application Design Decisions | |
Chatley | Predictable Dynamic Plugin Architectures | |
Ukis | Deployment Contracts for Software Components | |
Hoare | 6. System Design |