CN110740274B - System and method for unified access to framework based on image video algorithm - Google Patents
System and method for unified access to framework based on image video algorithm Download PDFInfo
- Publication number
- CN110740274B CN110740274B CN201911019359.2A CN201911019359A CN110740274B CN 110740274 B CN110740274 B CN 110740274B CN 201911019359 A CN201911019359 A CN 201911019359A CN 110740274 B CN110740274 B CN 110740274B
- Authority
- CN
- China
- Prior art keywords
- algorithm
- image
- interface
- framework
- video
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The invention discloses a system and a method for uniformly accessing a framework based on an image video algorithm, wherein the system comprises: the algorithm application layer is used for calling image and video processing algorithms; the unified access framework is used for defining an interface to be accessed with an image and video processing algorithm; and the algorithm layer is used for realizing the interface defined by the unified access framework. The invention defines the unified algorithm interface and parameter, the image video algorithm can realize the algorithm service only by realizing the interface and parameter of the access framework, the unified image video algorithm access framework realizes the internal decoupling with each image video algorithm, and the light weight and strong expansibility of the unified image video algorithm access framework is ensured.
Description
Technical Field
The invention relates to the field of image processing, in particular to a system and a method for uniformly accessing a framework based on an image video algorithm.
Background
With the development of mobile internet, the shot amount of images and videos is greatly increased. The development of image and video processing techniques has been rapidly advanced. At present, image and video processing technologies mainly include video synthesis, image content analysis, video transcoding, image segmentation and the like, and specifically include: the method comprises the steps of material special effect video synthesis, picture and video content scene analysis, video transcoding, video subtitle synthesis, picture segmentation and picture fusion.
The invention patent with publication number CN 107870816 a discloses a method and apparatus for image processing and storage, and specifically discloses: a method for image processing and storage, comprising: monitoring a request through a main process, and creating a request processing sub-thread after a received request message; executing the request processing sub-thread to analyze the request message to obtain a request parameter, and transmitting original image data and the request parameter into a called image processing interface; processing the original image data according to the transmitted request parameters through the created image processing sub-thread to obtain a processing result; and generating a response message according to the processing result through the request processing sub-thread, and feeding back the response message aiming at the request message.
The current image video algorithm service mode is that an integrated algorithm framework is connected with an image video algorithm to define a set of interfaces, when the image video algorithm relates to the change of request parameters and response parameters, the integrated algorithm framework needs to be changed, and the integrated algorithm framework is not decoupled from a specific image video algorithm, namely corresponding algorithm application service. With the increasing of the accessed image video algorithms and services, the number of the interfaces involved in the integrated algorithm framework is increased, and the integrated algorithm framework and the service coupling are also increased in a straight line.
Therefore, how to implement a system for uniformly accessing an image video algorithm to a framework to decouple application services corresponding to a specific image video algorithm is a problem to be solved in the field.
Disclosure of Invention
The invention aims to provide a system and a method for uniformly accessing a framework based on an image video algorithm aiming at the defects of the prior art. By defining a uniform algorithm interface and parameters, the image video algorithm can realize the service of the algorithm only by realizing the interface and the parameters of the access framework, the uniform image video algorithm access framework realizes the internal decoupling with each image video algorithm, and the light weight and strong expansibility of the uniform image video algorithm access framework is ensured.
In order to achieve the purpose, the invention adopts the following technical scheme:
a system for unifying access frameworks based on image-video algorithms, comprising:
the algorithm application layer is used for calling image and video processing algorithms;
the unified access framework is used for defining an interface to be accessed with an image and video processing algorithm;
and the algorithm layer is used for realizing the interface defined by the unified access framework.
Further, the interface includes:
an algorithm initialization interface, an algorithm processing interface and an algorithm release interface; the algorithm initialization interface is used for loading image and video processing algorithms; the algorithm processing interface is used for realizing the processing of images and videos; the algorithm release interface is used for releasing the algorithms of image and video processing.
Further, the algorithm application layer and the algorithm layer define request parameters and response parameters through a ProtoBuf protocol.
Further, the algorithm application layer and the unified access framework communicate by using a GRPC protocol.
Further, the system further comprises:
the system comprises a resource file downloading module, a resource file uploading module and a resource file caching module.
The invention also provides a method for uniformly accessing the framework based on the image video algorithm, which comprises the following steps:
s1, the algorithm application layer sends request parameters to the unified access framework;
s2, calling an algo _ process interface and transmitting request parameters to an algorithm layer by the unified access framework;
s3, the algorithm layer processes the algorithm according to the parameters and returns response parameters to the unified access frame through the algo _ process interface;
and S4, the unified access framework returns response parameters to the algorithm application layer.
Further, the step S1 is specifically:
and the algorithm application layer encapsulates the request defined by the algorithm layer, and sends a GRPC request to the unified access framework by using the ProtoBuf serialized request.
Further, the step S3 is specifically:
s3.1, deserializing the ProtoBuf serialization request to obtain a request parameter corresponding to an algorithm application layer;
s3.2, processing the request parameters according to a processing algorithm to obtain response parameters;
and S3.3, serializing the response parameters into a ProtoBuf byte array and returning the ProtoBuf byte array to the unified access framework.
Further, the method further comprises:
and S5, the algorithm application layer deserializes the response parameters of the response ProtoBuf byte array by using ProtoBuf.
Further, the step S4 is specifically:
and the unified access framework calls a complete algo _ process interface, acquires data returned by the algorithm layer and returns the data to the algorithm application layer.
The invention provides a system and a method for uniformly accessing a framework based on an image video algorithm. Each accessed image video processing algorithm only needs to rewrite and realize a corresponding interface to complete the access of the algorithm, and a set of interfaces are not required to be defined when the algorithm is accessed into a frame every time. Meanwhile, when the algorithm application layer and the algorithm layer are updated or the algorithm request parameter and the algorithm response parameter are changed, the unified image video algorithm access framework does not need to be changed. The algorithm dynamic library realizes three interfaces defined by the unified image video algorithm access framework, and the unified image video algorithm access framework can integrate the corresponding image video algorithm. The unified image video algorithm access framework uniformly defines the interface and parameters of the algorithm. The image video algorithm can realize the service of the algorithm only by realizing the interface and the parameter of the access framework. The downloaded resources and the uploaded resources required in the algorithm are exchanged through appointed parameters. Therefore, the unified image video algorithm access framework realizes the internal decoupling with each image video algorithm, and ensures the light weight and strong expansibility of the unified image video algorithm access framework.
Drawings
Fig. 1 is a system structure diagram of a unified access framework based on an image video algorithm according to an embodiment;
fig. 2 is a flowchart of a method for unifying access to a framework based on an image video algorithm according to the second embodiment.
Detailed Description
The embodiments of the present invention are described below with reference to specific embodiments, and other advantages and effects of the present invention will be easily understood by those skilled in the art from the disclosure of the present specification. The invention is capable of other and different embodiments and of being practiced or of being carried out in various ways, and its several details are capable of modification in various respects, all without departing from the spirit and scope of the present invention. It is to be noted that the features in the following embodiments and examples may be combined with each other without conflict.
It should be noted that the drawings provided in the following embodiments are only for illustrating the basic idea of the present invention, and the components related to the present invention are only shown in the drawings rather than drawn according to the number, shape and size of the components in actual implementation, and the type, quantity and proportion of the components in actual implementation may be changed freely, and the layout of the components may be more complicated.
The invention is further described with reference to the following drawings and specific examples, which are not intended to be limiting.
Example one
As shown in fig. 1, the present embodiment provides a system for unified access to a framework based on an image-video algorithm, including:
the algorithm application layer is used for calling image and video processing algorithms;
in the application of processing images and videos, corresponding image and video processing algorithms need to be called for realization. For example, a face synthesis related application needs to invoke a picture fusion algorithm. The algorithm application layer calls the image and video processing algorithm to generate corresponding processing results by requesting corresponding parameters, namely corresponding response parameters are returned. For example, in the face fusion application, the algorithm application layer sends a source face image and a fusion target image before fusion, and the image and video processing algorithm returns a face image after fusion.
In the invention, an algorithm application layer and an image and video processing algorithm define request parameters and response parameters through a ProtoBuf protocol. ProtoBuf is a lightweight and efficient structured data storage format that can be used for structured data serialization, or serialization. It is well suited for data storage or RPC data exchange formats. Therefore, the invention realizes the request and response of the parameters through the ProtoBuf, can realize the high-speed exchange of data and save the storage space of the data.
The unified access framework is used for defining an interface to be accessed with an image and video processing algorithm;
in order to avoid defining a set of interfaces every time an image and video processing algorithm is accessed, the invention discloses a uniform access framework for all image and video processing algorithms to access. The interface for defining the image and video processing algorithm to be accessed comprises an algorithm initialization interface, an algorithm processing interface and an algorithm release interface.
The algorithm initialization interface is used for loading the image and video processing algorithm. Defining an algorithm initialization interface algo _ init, and starting an image and video processing algorithm by calling a start function. Meanwhile, corresponding operating environments are provided for image and video processing algorithms. For example, assigning threads to images, video processing algorithms, providing run memory, etc.
The algorithm processing interface is used for realizing the processing of images and videos. An algorithm processing interface algo _ process is defined to access various processing of images and videos. The unified access framework is suitable for accessing all image and video processing algorithms. Therefore, the image and video processing algorithms complete the access of each processing algorithm by rewriting the algorithm processing interface.
Image video processing algorithms include various processing of images, videos, e.g., material special effects video composition. Therefore, the system of the invention further comprises a resource file downloading module, a resource file uploading module and a resource file caching module. The resource file downloading module is used for downloading a corresponding resource file, such as a video file to be processed from a network. The resource file uploading module is used for uploading resource files, such as video files needing to be processed locally. The resource file caching module is used for caching the resource files. For the acquired resource file, the file may be cached to wait for processing of the resource file. And the resource file downloading module, the resource file uploading module and the resource file caching module are interacted with the algorithm layer to realize the processing of each resource. The resources required by the image video processing algorithm are exchanged by the agreed parameters.
The algorithm release interface is used for releasing the algorithms of image and video processing. The definition algorithm releases the interface algo _ hierarchy. After the image and video processing algorithms are used, the accessed processing algorithms can be deleted, and meanwhile, the cleaning work related to the algorithms is completed. For example, the space for the application of the algorithm is released.
Therefore, the interface defined by the unified access framework of the present invention is:
and the algorithm layer is used for realizing the interface defined by the unified access framework. The image and video processing algorithm is to be accessed into the unified access framework, three interfaces which are determined by the unified image and video algorithm access framework are rewritten and realized, and the unified image and video algorithm access framework is compiled into a dynamic library, so that the corresponding algorithm can be integrated into the unified image and video algorithm access framework in a plug-in mode. Therefore, the decoupling of the unified image video algorithm access framework and the specific image video algorithm can be achieved without defining a set of interface for each image video algorithm.
Specifically, the algorithm application layer and the unified image video algorithm access framework communicate by using a GRPC protocol. GRPC is a high-performance, universal, open-source RPC framework, designed based on the HTTP2 protocol standard, developed based on the ProtoBuf serialization protocol, and supports numerous development languages.
And the algorithm application layer converts the request parameters into byte arrays according to a ProtoBuf protocol sequence and transmits the byte arrays to the unified image video algorithm access framework. The unified image video algorithm access framework receives the request parameters of the algorithm application layer, calls an algo _ process interface of the algorithm dynamic library, and transmits the well serialized ProtoBuf byte parameters of the algorithm application layer. And the algorithm dynamic library acquires the serialized byte parameters of the application layer in the algo-process interface, acquires the algorithm application layer request parameters through ProtoBuf deserialization, and then performs corresponding image video processing according to the request parameters. The algorithm dynamically processes the request, converts the response parameters into byte parameters by using ProtoBuf serialization and returns the byte parameters to the unified image video algorithm access framework, the unified image video algorithm access framework returns the response parameters of the algo _ process to the algorithm application layer, and the algorithm application layer obtains the response parameters from the unified image video algorithm access framework and then obtains the algorithm response parameters by using ProtoBuf deserialization on the byte array returned by the image video algorithm.
Therefore, the unified image video algorithm access framework is used, and the unified image video algorithm access framework does not need to be changed when the algorithm application layer and the algorithm layer are updated or when the algorithm request parameters and the algorithm response parameters are changed. The algorithm dynamic library realizes three interfaces defined by the unified image video algorithm access framework, and the unified image video algorithm access framework can integrate the corresponding image video algorithm.
Example two
As shown in fig. 2, based on the system for uniformly accessing a framework based on an image video algorithm provided in the first embodiment, the present embodiment provides a method for uniformly accessing a framework based on an image video algorithm, which includes:
s1, the algorithm application layer sends request parameters to the unified access framework;
as described above, the unified access framework defines three interfaces, the algorithm layer implements the functions of the corresponding interfaces by rewriting the three interfaces, and the algorithms are compiled into a dynamic library, so that the corresponding algorithms can be integrated into the unified image video algorithm access framework in a plug-in manner. Therefore, when the algorithm application layer responds to realize the processing of the image and the video, corresponding request information needs to be sent to the unified access framework, and the corresponding request is sent to the corresponding algorithm layer through the unified access framework, so that the processing of the algorithm application layer request is realized.
Specifically, the algorithm application layer encapsulates the algorithm-defined request and uses the ProtoBuf serialization request to send a GRPC request to the unified image video algorithm access framework. The algorithm application layer converts the request parameters into byte arrays according to a ProtoBuf protocol sequence and transmits the byte arrays to the unified image video algorithm access framework, and the algorithm application layer and the unified image video algorithm access framework are communicated by using a GRPC protocol.
S2, calling an algo _ process interface and transmitting request parameters to an algorithm layer by the unified access framework;
the unified image video algorithm access framework receives the request parameters of the algorithm application layer, calls an algo _ process interface of the algorithm layer and transmits the well serialized ProtoBuf byte parameters of the algorithm application layer.
S3, the algorithm layer processes the algorithm according to the parameters and returns response parameters to the unified access frame through the algo _ process interface;
and the algorithm layer acquires bytes brought by the algo-process interface, and deserializes the byte array parameters of the algorithm application layer to obtain the corresponding request parameters of the algorithm application layer. And the algorithm layer performs corresponding algorithm processing according to the request parameters of the algorithm application layer and returns response parameters. And the algorithm dynamic library serializes the response parameters of the algorithm application layer into a ProtoBuf byte array and returns the ProtoBuf byte array to the unified image video algorithm access framework.
And S4, the unified access framework returns response parameters to the algorithm application layer.
The unified image video algorithm access framework calls a finished algo _ process interface, obtains data returned by the algorithm and returns the data to the algorithm application layer. And the algorithm application layer acquires response parameters from the unified image video algorithm access framework. And the algorithm application layer carries out the deserialization of the response parameters by using ProtoBuf to obtain the response parameters processed by the algorithm.
Therefore, the system and the method for uniformly accessing the framework based on the image video algorithm, which are provided by the invention, use the uniform image video algorithm to access the framework, and when the algorithm application layer and the algorithm layer are updated or the algorithm request parameter and the response parameter are changed, the uniform image video algorithm access framework does not need to be changed. The algorithm dynamic library realizes three interfaces defined by the unified image video algorithm access framework, and the unified image video algorithm access framework can integrate the corresponding image video algorithm.
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims (2)
1. A system for unifying access to a framework based on image-video algorithms, comprising:
the algorithm application layer is used for calling image and video processing algorithms;
the unified access framework is used for defining an interface to be accessed with an image and video processing algorithm;
the algorithm layer is used for realizing the interface defined by the unified access framework;
the interface includes:
an algorithm initialization interface, an algorithm processing interface and an algorithm release interface; the algorithm initialization interface is used for loading image and video processing algorithms; the algorithm processing interface is used for realizing the processing of images and videos; the algorithm releasing interface is used for releasing the algorithms of image and video processing;
the algorithm application layer and the algorithm layer define request parameters and response parameters through a ProtoBuf protocol;
the algorithm application layer and the unified access framework are communicated by using a GRPC protocol;
the system further comprises:
the system comprises a resource file downloading module, a resource file uploading module and a resource file caching module.
2. A method for unifying access frameworks based on image-video algorithms, based on the system of claim 1, characterized by comprising the steps of:
s1, the algorithm application layer sends request parameters to the unified access framework;
s2, calling an algo _ process interface and transmitting request parameters to an algorithm layer by the unified access framework;
s3, the algorithm layer processes the algorithm according to the parameters and returns response parameters to the unified access frame through the algo _ process interface;
s4, the unified access framework returns response parameters to the algorithm application layer;
the step S1 specifically includes:
the algorithm application layer encapsulates the request well defined by the algorithm layer, and sends a GRPC request to the unified access framework by using a ProtoBuf serialized request;
the step S3 specifically includes:
s3.1, deserializing the ProtoBuf serialization request to obtain a request parameter corresponding to an algorithm application layer;
s3.2, processing the request parameters according to a processing algorithm to obtain response parameters;
s3.3, serializing the response parameters into a ProtoBuf byte array and returning the ProtoBuf byte array to the unified access framework;
the method further comprises the following steps:
s5, the algorithm application layer uses the ProtoBuf byte array of the response to deserialize the response parameters;
the step S4 specifically includes:
and the unified access framework calls a complete algo _ process interface, acquires data returned by the algorithm layer and returns the data to the algorithm application layer.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911019359.2A CN110740274B (en) | 2019-10-24 | 2019-10-24 | System and method for unified access to framework based on image video algorithm |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911019359.2A CN110740274B (en) | 2019-10-24 | 2019-10-24 | System and method for unified access to framework based on image video algorithm |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110740274A CN110740274A (en) | 2020-01-31 |
| CN110740274B true CN110740274B (en) | 2022-02-25 |
Family
ID=69271250
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911019359.2A Active CN110740274B (en) | 2019-10-24 | 2019-10-24 | System and method for unified access to framework based on image video algorithm |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110740274B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111385604A (en) * | 2020-03-27 | 2020-07-07 | 杭州趣维科技有限公司 | Unified algorithm combination service method |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108337236A (en) * | 2017-12-28 | 2018-07-27 | 杭州马猴烧韭科技有限公司 | A kind of gRPC call methods and device based on Protobuf and HTTP/1.1 |
| WO2019184606A1 (en) * | 2018-03-27 | 2019-10-03 | 华为技术有限公司 | Smart video processing apparatus |
-
2019
- 2019-10-24 CN CN201911019359.2A patent/CN110740274B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108337236A (en) * | 2017-12-28 | 2018-07-27 | 杭州马猴烧韭科技有限公司 | A kind of gRPC call methods and device based on Protobuf and HTTP/1.1 |
| WO2019184606A1 (en) * | 2018-03-27 | 2019-10-03 | 华为技术有限公司 | Smart video processing apparatus |
Non-Patent Citations (1)
| Title |
|---|
| 基于深度学习的视频检索系统设计与实现;姚锦江等;《计算机测量与控制》;20190630;第231-235页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110740274A (en) | 2020-01-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101258478B (en) | High level graphics stream | |
| US11516152B2 (en) | First-in first-out function for segmented data stream processing | |
| JP7392136B2 (en) | Methods, computer systems, and computer programs for displaying video content | |
| US11516628B2 (en) | Media streaming with edge computing | |
| JP6398114B2 (en) | Communication protocol between platform and image device | |
| WO2021062115A1 (en) | Compute resource estimation for function implementation on computing platform | |
| CN108073350A (en) | A kind of object storage system rendered for cloud and method | |
| WO2023093776A1 (en) | Interface generation method and electronic device | |
| CN112307403A (en) | Page rendering method, device, storage medium and terminal | |
| US11861411B2 (en) | Variable and event reporting in a cloud service system | |
| CN110740274B (en) | System and method for unified access to framework based on image video algorithm | |
| WO2020215833A1 (en) | Offline cache method and apparatus, and terminal and readable storage medium | |
| JP7399548B2 (en) | Method and apparatus for media scene description | |
| US20170186129A1 (en) | Method and system for reviewing medical study data | |
| WO2023093779A1 (en) | Interface generation method and electronic device | |
| US20220107832A1 (en) | Method and apparatus for cloud service | |
| US11750676B2 (en) | Network-based media processing (NBMP) deployment with framework for live uplink streaming (FLUS) and 5G application function (AF) | |
| CN111651146B (en) | Function generation method, device, equipment and medium | |
| CN113342837B (en) | Data transmission method, device, electronic equipment and computer readable medium | |
| CN116958300A (en) | Image processing apparatus, image processing method, image capturing apparatus, and electronic apparatus | |
| CN110855539B (en) | Device discovery method, device and storage medium | |
| WO2019079940A1 (en) | Graphics processing method, and related apparatus and device | |
| US20230338834A1 (en) | Smart client for streaming of scene-based immersive media to game engine | |
| CN111338739B (en) | Page content processing method, page content display method and page content display system | |
| US20230007067A1 (en) | Bidirectional presentation datastream |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 22nd floor, block a, Huaxing Times Square, 478 Wensan Road, Xihu District, Hangzhou, Zhejiang 310000 Applicant after: Hangzhou Xiaoying Innovation Technology Co.,Ltd. Address before: 16 / F, HANGGANG Metallurgical Science and technology building, 294 Tianmushan Road, Xihu District, Hangzhou City, Zhejiang Province, 310012 Applicant before: HANGZHOU QUWEI SCIENCE & TECHNOLOGY Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |