CN114222164A - Video plug-flow method, device, intelligent equipment and storage medium - Google Patents

Video plug-flow method, device, intelligent equipment and storage medium Download PDF

Info

Publication number
CN114222164A
CN114222164A CN202111552137.4A CN202111552137A CN114222164A CN 114222164 A CN114222164 A CN 114222164A CN 202111552137 A CN202111552137 A CN 202111552137A CN 114222164 A CN114222164 A CN 114222164A
Authority
CN
China
Prior art keywords
preset
video
data
alive
plug flow
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111552137.4A
Other languages
Chinese (zh)
Inventor
李增
黎兢
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen TCL New Technology Co Ltd
Original Assignee
Shenzhen TCL New Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen TCL New Technology Co Ltd filed Critical Shenzhen TCL New Technology Co Ltd
Priority to CN202111552137.4A priority Critical patent/CN114222164A/en
Publication of CN114222164A publication Critical patent/CN114222164A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
    • H04N21/23412Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs for generating or manipulating the scene composition of objects, e.g. MPEG-4 objects
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/80Responding to QoS
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
    • H04N21/2343Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements

Abstract

The embodiment of the application discloses a video plug-flow method, a video plug-flow device, intelligent equipment and a storage medium, and belongs to the technical field of communication; when the intelligent device is detected to be in the preset plug flow state, starting a preset plug flow process to keep alive; when the preset plug flow process is successfully kept alive, acquiring image data sent by image acquisition equipment; converting the image data into video data by a rendering component; the video data is pushed to the streaming server in real time, and the non-inductive streaming of the intelligent equipment can be realized.

Description

Video plug-flow method, device, intelligent equipment and storage medium
Technical Field
The present application relates to the field of communications technologies, and in particular, to a video stream pushing method, an apparatus, an intelligent device, and a storage medium.
Background
The plug flow refers to a process of transmitting the acquired audio and video data to a server, namely a process of transmitting the audio and video data acquired by the acquisition device to a network. Plug flow techniques find application in many application scenarios. For example, in a live application scenario, push streaming is one of the basic technologies to implement live broadcasting. For another example, in an application scenario of real-time security monitoring, plug flow is also one of the basic technologies. In the prior art, it is often necessary for a device to be able to preview the content of a push stream in order to push the stream. For example, for an application scenario of real-time security monitoring, it is often necessary that a device is in a working state, and the content of the stream pushing can be previewed by the device to achieve the stream pushing.
Disclosure of Invention
The embodiment of the application provides a video plug flow method, a video plug flow device, intelligent equipment and a storage medium, and can realize non-inductive plug flow.
The embodiment of the application provides a video plug flow method, which comprises the following steps:
when the intelligent equipment is detected to be in a preset plug flow state, starting a preset plug flow process to keep alive;
when the preset plug flow process is successfully kept alive, acquiring image data sent by image acquisition equipment;
converting the image data into video data by a rendering component;
and pushing the video data to a streaming server in real time.
Correspondingly, the embodiment of the present application further provides a video stream pushing apparatus, including:
the keep-alive unit is used for starting a preset plug flow process to keep alive when the intelligent equipment is detected to be in a preset plug flow state;
the acquisition unit is used for acquiring image data sent by the image acquisition equipment when the preset plug flow process is successfully kept alive;
a conversion unit for converting the image data into video data by a rendering component;
and the pushing unit is used for pushing the video data to the streaming server in real time.
In an embodiment, the conversion unit may include:
the storage subunit is configured to store the image data into a preset canvas in the rendering component, so as to obtain image stream data corresponding to the image data;
the texture attachment processing subunit is configured to perform texture attachment processing on the image stream data by using the rendering component to obtain texture data corresponding to the image data in the preset canvas;
an encoding subunit, configured to encode the texture data into the video data.
In an embodiment, the encoding subunit may include:
the callback module is used for calling back the texture data from the rendering component by using a preset callback function;
the acquisition module is used for acquiring audio data corresponding to the image data;
and the coding module is used for coding the texture data and the audio data to obtain the video data.
In an embodiment, the keep-alive unit may include:
the information generation subunit is configured to generate process keep-alive starting information when it is detected that the intelligent device is in a preset plug flow state, where the process keep-alive starting information includes the preset plug flow process;
the information adding subunit is used for adding the process keep-alive starting information into a process white list in a preset application interface layer to obtain an added white list;
and the execution subunit is configured to execute the added white list through the preset application interface layer to start the preset push flow process keep-alive.
In an embodiment, the keep-alive unit may include:
the calling subunit is used for calling a preset list file when the intelligent device is detected to be in a preset plug flow state, wherein the preset list file comprises a priority parameter of the preset plug flow process;
the adjusting subunit is used for modifying the priority parameter of the preset plug flow process to obtain a modified list file;
and the broadcasting subunit is used for broadcasting the modified manifest file so as to start a preset stream pushing process to keep alive.
In an embodiment, the video plug flow device further includes:
the device comprises a request receiving unit, a video push streaming starting unit and a video pushing unit, wherein the request receiving unit is used for receiving a video push streaming starting request which carries authority verification information;
the verification unit is used for verifying the authority verification information;
and the first detection unit is used for detecting the state of the intelligent equipment when the verification is passed.
In an embodiment, the video plug flow device further includes:
the device comprises an instruction receiving unit, a time configuration unit and a time configuration unit, wherein the instruction receiving unit is used for receiving a time configuration instruction, and the time configuration instruction carries time configuration information;
a cycle generation unit configured to generate an apparatus state detection cycle based on the time configuration information;
and the second detection unit is used for detecting the state of the intelligent equipment in the equipment state detection period.
Correspondingly, the embodiment of the application also provides the intelligent device, which comprises a memory and a processor; the memory stores a computer program, and the processor is configured to execute the computer program in the memory to perform the steps in the video plug flow method provided in any of the embodiments of the present application.
Correspondingly, the embodiment of the present application further provides a storage medium, where the storage medium stores a computer program, and the computer program, when executed by a processor, implements the steps in the video stream pushing method provided in any of the embodiments of the present application.
When the intelligent device is detected to be in the preset plug flow state, starting a preset plug flow process to keep alive; when the preset plug flow process is successfully kept alive, acquiring image data sent by image acquisition equipment; converting the image data into video data by a rendering component; the video data is pushed to the streaming server in real time, and non-inductive streaming pushing can be achieved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a schematic scene diagram of a video plug-streaming method provided in an embodiment of the present application;
fig. 2 is a schematic flowchart of a video plug-streaming method according to an embodiment of the present application;
fig. 3 is a schematic flowchart of a video plug-streaming method according to an embodiment of the present application;
fig. 4 is a schematic diagram of another scene of a video plug-flow method provided in an embodiment of the present application;
fig. 5 is a schematic structural diagram of a video plug-flow device provided in an embodiment of the present application;
fig. 6 is a schematic structural diagram of an intelligent device provided in an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, however, the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments of the present application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The embodiment of the application provides a video plug flow method, which can be executed by a video plug flow device, and the video plug flow device can be integrated in an intelligent device. Wherein, the intelligent device can comprise an intelligent television
The terminal can comprise a personal computer, a tablet personal computer, an intelligent television, a smart phone, a smart home, wearable intelligent equipment, VR/AR equipment, a vehicle-mounted computer and the like.
The server may be an interworking server or a background server among a plurality of heterogeneous systems, an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, a cloud server providing basic cloud computing services such as cloud service, a cloud database, cloud computing, a cloud function, cloud storage, network service, cloud communication, middleware service, domain name service, security service, big data, an artificial intelligence platform and the like, and the like.
In an embodiment, as shown in fig. 1, the video stream pushing apparatus may be integrated on an intelligent device such as a terminal or a server, so as to implement the video stream pushing method provided in the embodiment of the present application. Specifically, the smart device may include smart homes such as a smart television, a smart air conditioner, and a smart door lock. In addition, the intelligent device can also comprise a mobile phone, a tablet, a computer and other terminals.
The smart home (home automation) integrates facilities related to home life by using a home as a platform and utilizing a comprehensive wiring technology, a network communication technology, a security technology, an automatic control technology and an audio and video technology, so that a high-efficiency management system of home facilities and home schedule affairs is constructed, home safety, convenience, comfort and artistry are improved, and an environment-friendly and energy-saving living environment is realized.
In an embodiment, as shown in fig. 1, the video plug-streaming apparatus provided in the embodiment of the present application may be integrated on an intelligent device such as an intelligent home, so as to implement the monitoring method for the intelligent device provided in the embodiment of the present application. Specifically, when the intelligent device is detected to be in a preset plug flow state, starting a preset plug flow process to keep alive; when the preset plug flow process is successfully kept alive, acquiring image data sent by image acquisition equipment; converting the image data into video data by a rendering component; and pushing the video data to a streaming server in real time.
The following are detailed below, and it should be noted that the order of description of the following examples is not intended to limit the preferred order of the examples.
The video stream pushing method provided by the embodiment of the present application will be described from the perspective of integrating a video stream pushing device in an intelligent device.
As shown in fig. 2, a video plug-streaming method is provided, and the specific process includes:
101. and when the intelligent equipment is detected to be in a preset plug flow state, starting a preset plug flow process to keep alive.
The intelligent equipment can comprise an intelligent television, an intelligent air conditioner, an intelligent door lock and other intelligent homes, and can also comprise a mobile phone, a tablet, a computer and other terminals.
In an embodiment, when the intelligent device is in a working state, a conventional stream pushing method can be adopted to realize stream pushing of the video. When the intelligent device is not in a working state, the conventional plug-flow method cannot be adopted because the plug-flow of the video generally realized cannot be generally previewed. For example, when the smart device is in a sleep state, since the collected video data cannot be previewed by the smart device, the video cannot be pushed by a sub-ordinary push streaming method. Therefore, the embodiment of the application provides a video stream pushing method capable of realizing non-inductive stream pushing. By the video stream pushing method provided by the embodiment of the application, the video stream pushing can be still realized in the state that the intelligent equipment is not working.
The preset plug flow state may include a state of the smart device other than the operating state. For example, the preset push flow state may be a state in which the smart device is in sleep. For another example, the preset push flow state may be that the smart device is in a standby state, and so on.
In an embodiment, a video stream pushing apparatus may be integrated in the intelligent device, wherein when the video stream pushing apparatus detects that the intelligent device is in a preset stream pushing state, a preset stream pushing process may be started to keep alive.
For example, when the video stream pushing device detects that the intelligent device is in a standby state, a preset stream pushing process can be started to keep alive.
In an embodiment, an Android system may be used as an operating system for the smart device provided in the embodiment of the present application. When the android system detects that the intelligent device is in a dormant state or a standby state and the like, the android system kills part of processes in the intelligent device, so that the number of processes of the intelligent device running in the dormant state or the standby state and the like is reduced, the loss of the intelligent device in methods such as a battery and a storage is reduced, and the service life of the intelligent device is prolonged.
In an embodiment, in order to avoid that the operating system of the smart device kills the preset stream pushing process when the smart device is in the preset stream pushing state, the stream pushing process needs to be started to keep alive.
The preset stream pushing process may refer to a corresponding process when the video stream pushing method provided in the embodiment of the present application is executed.
Wherein, the process keep-alive instruction process is not killed, and can be taken all the time.
In an embodiment, by keeping the preset stream pushing process alive, when the intelligent device is in a preset stream pushing state, the preset stream pushing process is not cancelled by the system, so that the video stream pushing method provided by the embodiment of the present application can be executed. For example, by keeping the preset stream pushing process alive, when the intelligent device is in a sleep or standby state, the preset stream pushing process is not cancelled by the system, so that the video stream pushing method provided by the embodiment of the present application can be executed.
102. And when the preset plug flow process is successfully kept alive, acquiring image data sent by the image acquisition equipment.
In an embodiment, when the preset stream pushing process is successfully kept alive, it indicates that the method for executing video stream pushing proposed in the embodiment of the present application is not killed. At this time, the video plug-flow device can realize the non-inductive plug-flow. Specifically, the video stream pushing device may send an image acquisition instruction to the image acquisition device. When the image acquisition equipment receives an image acquisition instruction, the image acquisition equipment can start to acquire images and send acquired image data to the video plug flow device.
Wherein the image capturing device may comprise a device capable of capturing image or video data. For example, the image capture device may include a camera, or the like. As another example, the image capture device may be a mounted Camera (Camera), or the like.
In one embodiment, the image acquisition device and the intelligent device have an association relationship. For example, the image capture device may be connected to the smart device via a data line. For another example, the image capture device may be embedded in the smart device as part of the smart device.
For example, when the intelligent device is an intelligent television and the image acquisition device is a camera, the camera can be externally connected to the intelligent television through a data line, and can also be integrated in the intelligent device as a part of the intelligent device.
In one embodiment, for example, when the smart device is in the preset plug flow state, the image capturing device is often in the corresponding state. For example, when the smart device is in a sleep state, the image capture device is often also in the sleep state. In order to realize the non-inductive stream pushing, the video stream pushing device can send an image acquisition instruction to the image acquisition device so as to activate the image acquisition device to work.
After the image acquisition equipment receives the image acquisition instruction, the image acquisition equipment acquires a video clip of the environment where the image acquisition equipment is located, converts the video clip into image data and sends the image data to the video plug flow device. Therefore, the video plug-flow device can acquire the image data sent by the image acquisition device.
It should be noted that, the image capturing device captures an actual video clip, and in order to improve the transmission efficiency, the image capturing device often converts the video clip into image data and then transmits the image data. Therefore, the image data here often includes data of a plurality of video frames.
103. The image data is converted to video data by a rendering component.
In an embodiment, after the video stream pushing apparatus obtains the image data, since the intelligent device is in a preset stream pushing state such as a sleep state, the image data that cannot be acquired by the intelligent device is converted into video data, and previewed on the intelligent device. Therefore, video plug-streaming devices generally need to convert image data into video data through a rendering component.
Wherein the rendering component may comprise a component capable of converting image data into video data. For example, the rendering component may include a SurfaceTexture, SurfaceView, or TextureView component.
For example, image data may be converted into video data using surface texture, and so on. For example, the image data may be treated as an OpenGL ES texture using the SurfaceTexture, thereby obtaining preview data stream data corresponding to the image data.
OpenGL is an Application Programming Interface (API) that includes a series of functions that can manipulate graphics and images. OpenGL ES (OpenGL for Embedded Systems) is a subset of OpenGL, and is designed for Embedded devices such as mobile phones, PDAs, and game consoles.
OpenGL is a specification that defines a cross-programming language, cross-platform programming interface for three-dimensional or two-dimensional images. OpenGL is a professional graphical program interface, and is a bottom-layer graphics library which is powerful in function and convenient to call. OpenGL ES is an authorization fee free, cross-platform, fully functional, 2D and 3D graphics application program interface API, designed primarily specifically for a variety of embedded systems, including consoles, mobile phones, handheld devices, home appliances, and automobiles. The system consists of well-defined desktop OpenGL subsets, and creates a flexible and powerful bottom layer interactive interface between software and graphics acceleration.
104. And pushing the video data to a streaming server in real time.
In one embodiment, after the video data is obtained, the video stream pushing device can push the video data to the stream server in real time,
wherein, the streaming server may be a streaming media server.
Streaming media refers to a form of media that streams audio, video, and multimedia files over a network. Compared with the network playing mode of watching after downloading, the streaming media is typically characterized in that continuous audio and video information is compressed and then put on a network server, and a user can watch while downloading without waiting for the whole file to be downloaded.
The streaming media server is a core system of streaming media application, and is a key platform for an operator to provide video services to users. The main functions of the streaming media server are to collect, cache, schedule, transmit and play streaming media contents. The main performance aspects of a streaming media application system depend on the performance and quality of service of the media server. Therefore, the streaming server is the foundation of the streaming application system and is the most important component.
The embodiment of the application provides a video plug flow method, which comprises the following steps: when the intelligent equipment is detected to be in a preset plug flow state, starting a preset plug flow process to keep alive; when the preset plug flow process is successfully kept alive, acquiring image data sent by image acquisition equipment; converting the image data into video data by a rendering component; and pushing the video data to a streaming server in real time. By the video plug-flow method provided by the embodiment of the application, the image acquisition equipment can still acquire image data when the intelligent equipment is in a dormant state or a standby state and the like, and the acquired image data is sent to the video plug-flow equipment. Next, the video stream pushing device converts the image data into the video data by using the rendering component, so that the image data can be rendered in the states of dormancy or standby of the intelligent device, and the video data corresponding to the image data is obtained. Then, the video stream pushing device can push the video data to the stream server, so that the non-inductive stream pushing of the intelligent device is realized.
Wherein, the non-sensory plug flow may refer to realizing the plug flow without being perceived by the user.
For example, when the intelligent device is a smart television and the image acquisition device is a Camera, the Camera still can acquire image data of the surrounding environment in a dormant state of the smart television, and then the image data is converted into video data and then pushed to the streaming server, so that a security monitoring effect is achieved. Under the condition of dormancy, the camera that the intelligent television linked can play the effect of a security protection camera, thereby enriching the use scene of the intelligent equipment.
The method described in the above examples is further illustrated in detail below by way of example.
The method of the embodiment of the present application will be described by taking an example that a video plug-streaming method is integrated on an application program. Specifically, as shown in fig. 3, a flow of the video plug-streaming method provided in the embodiment of the present application may include:
201. and when the intelligent equipment is detected to be in a preset plug flow state, the application program starts a preset plug flow process to keep alive.
In an embodiment, the video stream pushing method provided by the embodiment of the present application may also be integrated in an application program, so that the application program has a function of sensorless stream pushing.
The application program comprises a computer program for completing one or more specific functions, which is operated in an interaction mode, can interact with a use object of the application program and is provided with a visual display interface.
For example, the application programs may include application software on various terminals. For example, the Application program may include various mobile phone software (APP), tablet Application software, Application software on a notebook Computer, Application software on a Personal Computer (PC), and the like.
For example, the application programs may include browsers, video playback software, game software, and so forth.
As another example, an application may also include an applet. Among them, the applet (Mini Program) is an application that can be used without downloading and installing. In order to provide more diversified business services to users, developers can develop corresponding applets for applications (such as instant messaging applications, shopping applications, mail applications, and the like) of the terminal, the applets can be embedded into the applications of the terminal as sub-applications, and the corresponding business services can be provided for the users by running the sub-applications (i.e., the corresponding applets) in the applications.
In one embodiment, the application program can be downloaded to the intelligent device, so that the intelligent device can realize the non-inductive plug flow.
In an embodiment, when the intelligent device is detected to be in the preset push flow state, the application program may start the preset push flow process to keep alive. For example, as shown in fig. 4, when the application detects that the smart device is in a preset push flow state, the application may initiate a preset push flow process keep-alive.
There are various ways to start the preset plug flow process keep-alive. For example, process keep-alive can be initiated by increasing process priority, or by using a white list, and so on.
In an embodiment, the step "starting a preset stream pushing process keep-alive when the intelligent device is detected to be in a preset stream pushing state" may include:
when the intelligent device is detected to be in a preset plug flow state, generating process keep-alive starting information, wherein the process keep-alive starting information comprises a preset plug flow process;
adding the process keep-alive starting information into a process white list in a preset application interface layer to obtain an added white list;
and executing the added white list through a preset application interface layer to start a preset plug flow process to keep alive.
The process keep-alive starting information may be information for explaining which processes are kept alive. For example, the process keep-alive starting information includes a preset stream pushing process, which indicates that the preset stream pushing process is kept alive.
In an embodiment, the process keep-alive starting information may be added to a process white list in a preset application interface layer, so as to realize preset push flow process keep-alive.
Wherein the default application interface layer may include a library that provides a number of APIs. For example, a developer may write functions as APIs and store the APIs in a preset application interface layer to provide upper layer calls. For example, the predetermined application interface layer may be a Framework layer.
Wherein, the process white list may include processes that are not killed.
For example, after the process keep-alive starting information is added to the process white list in the Framework layer, the preset interface layer knows that the preset plug flow process cannot be cancelled when the intelligent device is in the preset plug flow state through the process keep-alive starting information in the process white list, and also needs to provide a corresponding API for the preset plug flow process, so that the keep-alive of the preset plug flow process is realized.
In an embodiment, after adding the process keep-alive starting information to the process white list, the preset interface layer is also triggered to execute the added white list. And triggering the preset interface layer to execute the added white list, namely enabling the preset interface layer to traverse the added white list, so that the preset interface layer knows which to keep alive.
In an embodiment, the process can also be kept alive by a method of increasing the priority of the process. Specifically, the step "when detecting that the intelligent device is in the preset stream pushing state, starting the preset stream pushing process to keep alive" may include:
when the intelligent equipment is detected to be in a preset plug flow state, calling a preset list file, wherein the preset list file comprises a priority parameter of a preset plug flow process;
modifying the priority parameter of a preset plug flow process to obtain a modified list file;
and broadcasting the modified manifest file to start a preset stream pushing process to keep alive.
The preset manifest file may be a file having various information of the process. For example, the preset manifest file may have a priority parameter of the process, and so on. For example, a preset manifest file may include a priority parameter of a preset plug-flow process, and the like.
In an embodiment, the priority of the predetermined plug-flow process may be modified by modifying a priority parameter of the predetermined plug-flow process.
In an embodiment, after the priority parameter of the preset push flow process is modified to obtain the modified manifest file, the modified manifest file may be broadcasted to start the preset push flow process to keep alive.
By broadcasting the modified process of the manifest file, other devices can know the updated process priority, so that the process with high priority cannot be killed.
In one embodiment, the video streaming function may be initiated by a user. The application will only detect the state of the smart device when the user has initiated the non-sensory push flow function in the smart device. Therefore, before the step "when the intelligent device is detected to be in the preset push flow state, the application program starts the preset push flow process to keep alive", the method may include:
receiving a video plug flow starting request, wherein the video plug flow starting request carries authority verification information;
verifying the authority verification information;
and when the verification is passed, detecting the state of the intelligent equipment.
In one embodiment, when a user initiates a video push streaming initiation function through an application, the application receives a video initiation request. In order to avoid malicious starting of the video plug-flow function by malicious molecules, the video plug-flow starting request carries authority verification information, so that whether the video plug-flow starting request is normal or not can be judged through the authority verification information.
The permission verification information may include information having a verification effect on the video push streaming start request. For example, the authorization verification information may include account information of the user, a device ID, a location where the device was when the request was sent, and so on.
In one embodiment, after obtaining the authorization verification information, the authorization verification information may be subjected to a verification process. After the verification is passed, the state of the intelligent device can be detected.
Among them, there are various ways to verify the authority verification information.
For example, the authority verification information and the preset verification reference information may be matched. And when the authority verification information is matched with the preset verification reference information, the authority verification information is verified to be passed.
The preset verification reference information may include preset account information, a device ID, a location where the device is located when the request is sent, and the like.
For example, it may be stated that the authentication is passed when the authority authentication information and the preset authentication reference information are the same.
In one embodiment, the user can also set the execution time of the non-inductive plug-flow function by configuring the time. For example, the user may set the smart device to operate at 6: this function of noninductive plug flow is performed during this time period of 00-18: 00. Then, at 6: in the time period of 00-18:00, when the intelligent device is in functions of dormancy or standby, the function of noninductive plug flow can be started. Specifically, the step "when detecting that the intelligent device is in the preset stream pushing state, before starting the preset stream pushing process keep-alive", may further include:
receiving a time configuration instruction, wherein the time configuration instruction carries time configuration information;
generating a device state detection period based on the time configuration information;
and in the equipment state detection period, detecting the state of the intelligent equipment.
The time configuration information may include a user configured time for starting the non-inductive plug flow function. For example, the time configuration information may be a time interval.
In an embodiment, when the user configures the time for starting the non-sensory stream pushing function, the application program may receive a video configuration instruction, where the video configuration instruction carries time configuration information.
The user may then generate a device state detection period based on the time configuration information.
For example, when the time configuration information is 6: 00-18:00, the device status detection period may also be 6: 00-18:00. Thus, when the application knows through the timer that the current time is 6: 00-18:00, the application will be in the range of 6: and detecting the state of the intelligent equipment in the time period of 00-18: 00.
202. And when the preset plug flow process is successfully kept alive, the application program acquires the image data sent by the image acquisition equipment.
In an embodiment, when the preset stream pushing process is successfully kept alive, the application program may send an image acquisition instruction to the image acquisition device. When the image acquisition device receives the image acquisition instruction, the image acquisition device can start to acquire images and send acquired image data to the application program.
The image data may include, among other things, video frames of a video captured by an image capture device.
203. The application converts the image data to video data through the rendering component.
In an embodiment, after the image data is obtained, since the smart device is in a preset push-streaming state such as a sleep state, the image data that cannot be acquired by the smart device is converted into video data and previewed on the smart device. Thus, the application typically needs to convert the image data to video data through a rendering component.
For example, after acquiring the image data, the application may acquire preview data stream data corresponding to the acquired image data by using the SurfaceTexture to treat the image data as an OpenGL ES texture.
In one embodiment, the step of converting the image data into video data by a rendering component may comprise:
storing the image data into a preset canvas of a rendering component to obtain image stream data corresponding to the image data;
performing texture attachment processing on the image stream data by using the rendering component to obtain texture data corresponding to the image data in a preset canvas;
the texture data is encoded into video data.
In an embodiment, since the intelligent device is in a sleep state and cannot render the image data, the image data can be rendered into a preset canvas of the rendering component to obtain image stream data corresponding to the image data.
The preset canvas may be a frame buffer (FrameBuffer) of OpenGL. Wherein the frame buffer may correspond to a canvas in OpenGL. However, the frame buffer is not a buffer in the conventional sense, and it is not an object that actually stores data. Framebuffer is a video output device that drives a video display device from a memory buffer containing complete frame data. The color value composition of each pixel on the screen is normally contained in the frame buffer. Color values are typically stored in 1-bit (black and white), 4-bit toned, 8-bit toned, 16-bit high, 24-bit true color formats. An additional alpha channel is sometimes used to preserve pixel transparency information.
In an embodiment, the image data may be first stored in a preset canvas of the rendering component, and image stream data corresponding to the image data is obtained.
For example, the image data may be first stored in the surfacene to obtain image stream data corresponding to the image data.
In an embodiment, the rendering component may be used to perform texture attachment processing on the image stream data to obtain texture data corresponding to the image data in a preset canvas.
The texture attachment process may refer to a process of coloring content in image stream data and texturing a surface of the content.
For example, the texture attachment process may include a color attachment process, a depth attachment process, and a template attachment process, etc. The color adhesion outputs color data of a drawing image, that is, RGBA data of an image which is common at ordinary times. If Multiple Render target (Multiple Render Targets) technology is used, the number of color attachments may be greater than one. The depth attachment outputs depth data of a drawn image, and the depth data is mainly used in 3D rendering and is generally used for judging the distance of an object to realize the shielding effect. Template data is output by attaching to a template, and the template data is a higher-level usage in rendering, is generally used for removing and shielding effects at a pixel level during rendering, and is commonly applied to scenes such as the delineation of three-dimensional objects.
In an embodiment, after the texture data is obtained, the texture data may be encoded into video data. The video data may include data having a dynamic picture and audio.
In one embodiment, the step of "encoding texture data into video data" may comprise:
calling back the texture data from the rendering component by using a preset call-back function;
acquiring audio data corresponding to the image data;
and coding the texture data and the audio data to obtain video data.
The preset callback function may be an onPreviewFrame function.
In one embodiment, the image capture device also captures corresponding audio data when capturing image data. Therefore, audio data corresponding to the image data can also be acquired.
In one implementation, after the audio data is obtained, the texture data and the audio data may be encoded to obtain the video data.
There are various ways to encode the texture data and the audio data to obtain the video data.
For example, the texture data and the audio data may be encoded by using a Real Time Messaging Protocol (RTMP) to obtain video data.
204. The application program pushes the video data to the streaming server in real time.
The embodiment of the application provides a video plug flow method, which comprises the following steps: when the intelligent equipment is detected to be in a preset plug flow state, an application program starts a preset plug flow process to keep alive; when the preset plug flow process is successfully kept alive, the application program acquires image data sent by the image acquisition equipment; the application program converts the image data into video data through the rendering component; the application program pushes the video data to the streaming server in real time. By the video plug-flow method provided by the embodiment of the application, the image acquisition equipment can still acquire image data when the intelligent equipment is in a dormant state or a standby state and the like, and the acquired image data is sent to the video plug-flow equipment. Next, the video stream pushing device converts the image data into the video data by using the rendering component, so that the image data can be rendered in the states of dormancy or standby of the intelligent device, and the video data corresponding to the image data is obtained. Then, the video stream pushing device can push the video data to the stream server, so that the non-inductive stream pushing of the intelligent device is realized.
In order to better implement the video stream pushing method provided by the embodiment of the present application, in an embodiment, a video stream pushing apparatus is further provided, and the video stream pushing apparatus may be integrated in an intelligent device. The terms are the same as those in the video plug-streaming method, and the details of the implementation can be referred to the description in the method embodiment.
In an embodiment, a video plug-streaming apparatus is provided, where the video plug-streaming apparatus may be specifically integrated in an intelligent device, as shown in fig. 5, and the video plug-streaming apparatus includes: the keep-alive unit 301, the obtaining unit 302, the converting unit 303 and the pushing unit 304 are specifically as follows:
the keep-alive unit 301 is configured to start a preset flow pushing process to keep alive when detecting that the intelligent device is in a preset flow pushing state;
an obtaining unit 302, configured to obtain image data sent by an image acquisition device when the preset plug flow process is successfully kept alive;
a conversion unit 303 for converting the image data into video data by a rendering component;
a pushing unit 304, configured to push the video data to a streaming server in real time.
In an embodiment, the converting unit 303 may include:
the storage subunit is configured to store the image data into a preset canvas in the rendering component, so as to obtain image stream data corresponding to the image data;
the texture attachment processing subunit is configured to perform texture attachment processing on the image stream data by using the rendering component to obtain texture data corresponding to the image data in the preset canvas;
an encoding subunit, configured to encode the texture data into the video data.
In an embodiment, the encoding subunit may include:
the callback module is used for calling back the texture data from the rendering component by using a preset callback function;
the acquisition module is used for acquiring audio data corresponding to the image data;
and the coding module is used for coding the texture data and the audio data to obtain the video data.
In an embodiment, the keep-alive unit 301 may include:
the information generation subunit is configured to generate process keep-alive starting information when it is detected that the intelligent device is in a preset plug flow state, where the process keep-alive starting information includes the preset plug flow process;
the information adding subunit is used for adding the process keep-alive starting information into a process white list in a preset application interface layer to obtain an added white list;
and the execution subunit is configured to execute the added white list through the preset application interface layer to start the preset push flow process keep-alive.
In an embodiment, the keep-alive unit 301 may include:
the calling subunit is used for calling a preset list file when the intelligent device is detected to be in a preset plug flow state, wherein the preset list file comprises a priority parameter of the preset plug flow process;
the adjusting subunit is used for modifying the priority parameter of the preset plug flow process to obtain a modified list file;
and the broadcasting subunit is used for broadcasting the modified manifest file so as to start a preset stream pushing process to keep alive.
In an embodiment, the video plug flow device further includes:
the device comprises a request receiving unit, a video push streaming starting unit and a video pushing unit, wherein the request receiving unit is used for receiving a video push streaming starting request which carries authority verification information;
the verification unit is used for verifying the authority verification information;
and the first detection unit is used for detecting the state of the intelligent equipment when the verification is passed.
In an embodiment, the video plug flow device further includes:
the device comprises an instruction receiving unit, a time configuration unit and a time configuration unit, wherein the instruction receiving unit is used for receiving a time configuration instruction, and the time configuration instruction carries time configuration information;
a cycle generation unit configured to generate an apparatus state detection cycle based on the time configuration information;
and the second detection unit is used for detecting the state of the intelligent equipment in the equipment state detection period.
In a specific implementation, the above units may be implemented as independent entities, or may be combined arbitrarily to be implemented as the same or several entities, and the specific implementation of the above units may refer to the foregoing method embodiments, which are not described herein again.
The video stream pushing device can realize the non-inductive stream pushing of the intelligent equipment.
The embodiment of the application also provides the intelligent equipment, which can comprise a terminal or a server; for example, the intelligent device may be a server, such as a video streaming server. As shown in fig. 6, it shows a schematic structural diagram of a terminal according to an embodiment of the present application, specifically:
the smart device may include components such as a processor 401 of one or more processing cores, memory 402 of one or more computer-readable storage media, a power supply 403, and an input unit 404. Those skilled in the art will appreciate that the smart device architecture shown in FIG. 6 does not constitute a limitation of smart devices and may include more or fewer components than shown, or some components in combination, or a different arrangement of components. Wherein:
the processor 401 is a control center of the smart device, connects various parts of the entire smart device using various interfaces and lines, and performs various functions of the smart device and processes data by running or executing software programs and/or modules stored in the memory 402 and calling data stored in the memory 402, thereby performing overall monitoring of the smart device. Optionally, processor 401 may include one or more processing cores; preferably, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly handles operating systems, user pages, application programs, and the like, and the modem processor mainly handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 401.
The memory 402 may be used to store software programs and modules, and the processor 401 executes various functional applications and data processing by operating the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data created according to use of the computer device, and the like. Further, the memory 402 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 access to the memory 402.
The smart device further includes a power supply 403 for supplying power to the various components, and preferably, the power supply 403 may be logically connected to the processor 401 through a power management system, so as to implement functions of managing charging, discharging, and power consumption through the power management system. The power supply 403 may also include any component of one or more dc or ac power sources, recharging systems, power failure detection circuitry, power converters or inverters, power status indicators, and the like.
The smart device may also include an input unit 404, the input unit 404 being operable to receive input numeric or character information and to generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.
Although not shown, the smart device may further include a display unit and the like, which will not be described herein. Specifically, in this embodiment, the processor 401 in the smart device loads the executable file corresponding to the process of one or more application programs into the memory 402 according to the following instructions, and the processor 401 runs the application program stored in the memory 402, thereby implementing various functions as follows:
when the intelligent equipment is detected to be in a preset plug flow state, starting a preset plug flow process to keep alive;
when the preset plug flow process is successfully kept alive, acquiring image data sent by image acquisition equipment;
converting the image data into video data by a rendering component;
and pushing the video data to a streaming server in real time.
The above operations can be implemented in the foregoing embodiments, and are not described in detail herein.
According to an aspect of the application, there is provided a computer program application or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the method provided in the various alternative implementations of the above embodiments.
It will be understood by those skilled in the art that all or part of the steps of the methods of the above embodiments may be performed by a computer program, which may be stored in a computer-readable storage medium and loaded and executed by a processor, or by related hardware controlled by the computer program.
To this end, embodiments of the present application further provide a storage medium, in which a computer program is stored, where the computer program can be loaded by a processor to execute the steps in any one of the video stream pushing methods provided in the embodiments of the present application. For example, the computer program may perform the steps of:
when the intelligent equipment is detected to be in a preset plug flow state, starting a preset plug flow process to keep alive;
when the preset plug flow process is successfully kept alive, acquiring image data sent by image acquisition equipment;
converting the image data into video data by a rendering component;
and pushing the video data to a streaming server in real time.
The above operations can be implemented in the foregoing embodiments, and are not described in detail herein.
Since the computer program stored in the storage medium can execute the steps in any video stream pushing method provided in the embodiments of the present application, beneficial effects that can be achieved by any video stream pushing method provided in the embodiments of the present application can be achieved, and detailed descriptions are omitted here for the foregoing embodiments.
The video stream pushing method, the video stream pushing device, the intelligent device and the storage medium provided by the embodiments of the present application are introduced in detail, and a specific example is applied in the present application to explain the principle and the implementation of the present application, and the description of the above embodiments is only used to help understanding the method and the core idea of the present application; meanwhile, for those skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.

Claims (10)

1. A video streaming method, comprising:
when the intelligent equipment is detected to be in a preset plug flow state, starting a preset plug flow process to keep alive;
when the preset plug flow process is successfully kept alive, acquiring image data sent by image acquisition equipment;
converting the image data into video data by a rendering component;
and pushing the video data to a streaming server in real time.
2. The method of claim 1, wherein converting the image data into video data by a rendering component comprises:
storing the image data into a preset canvas of the rendering component to obtain image stream data corresponding to the image data;
performing texture attachment processing on the image stream data by using the rendering component to obtain texture data corresponding to the image data in the preset canvas;
encoding the texture data into the video data.
3. The method of claim 2, wherein the encoding the texture data into the video data comprises:
calling back the texture data from the rendering component by using a preset call-back function;
acquiring audio data corresponding to the image data;
and coding the texture data and the audio data to obtain the video data.
4. The method according to claim 1, wherein when detecting that the smart device is in a preset push flow state, starting a preset push flow process keep-alive, comprising:
when detecting that the intelligent equipment is in a preset plug flow state, generating process keep-alive starting information, wherein the process keep-alive starting information comprises the preset plug flow process;
adding the process keep-alive starting information into a process white list in a preset application interface layer to obtain an added white list;
and executing the added white list through the preset application interface layer to start the preset push flow process to keep alive.
5. The method according to claim 1, wherein when detecting that the smart device is in a preset push flow state, starting a preset push flow process keep-alive, comprising:
when the intelligent equipment is detected to be in a preset plug flow state, calling a preset list file, wherein the preset list file comprises a priority parameter of the preset plug flow process;
modifying the priority parameter of the preset plug flow process to obtain a modified list file;
and broadcasting the modified manifest file to start a preset stream pushing process to keep alive.
6. The method according to claim 1, wherein before starting a preset push flow process to keep alive when detecting that the intelligent device is in a preset push flow state, the method comprises:
receiving a video plug flow starting request, wherein the video plug flow starting request carries authority verification information;
verifying the authority verification information;
and when the verification passes, detecting the state of the intelligent equipment.
7. The method according to claim 1, wherein before starting a preset push flow process to keep alive when detecting that the intelligent device is in a preset push flow state, the method comprises:
receiving a time configuration instruction, wherein the time configuration instruction carries time configuration information;
generating a device state detection period based on the time configuration information;
and detecting the state of the intelligent equipment in the equipment state detection period.
8. A video streaming apparatus, comprising:
the keep-alive unit is used for starting a preset plug flow process to keep alive when the intelligent equipment is detected to be in a preset plug flow state;
the acquisition unit is used for acquiring image data sent by the image acquisition equipment when the preset plug flow process is successfully kept alive;
a conversion unit for converting the image data into video data by a rendering component;
and the pushing unit is used for pushing the video data to the streaming server in real time.
9. A smart device comprising a memory and a processor; the memory stores a computer program, and the processor is configured to execute the computer program in the memory to perform the steps of the video plug flow method according to any one of claims 1 to 7.
10. A storage medium storing a plurality of computer programs adapted to be loaded by a processor to perform the steps of the video plug flow method according to any one of claims 1 to 7.
CN202111552137.4A 2021-12-17 2021-12-17 Video plug-flow method, device, intelligent equipment and storage medium Pending CN114222164A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111552137.4A CN114222164A (en) 2021-12-17 2021-12-17 Video plug-flow method, device, intelligent equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111552137.4A CN114222164A (en) 2021-12-17 2021-12-17 Video plug-flow method, device, intelligent equipment and storage medium

Publications (1)

Publication Number Publication Date
CN114222164A true CN114222164A (en) 2022-03-22

Family

ID=80703765

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111552137.4A Pending CN114222164A (en) 2021-12-17 2021-12-17 Video plug-flow method, device, intelligent equipment and storage medium

Country Status (1)

Country Link
CN (1) CN114222164A (en)

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103747205A (en) * 2013-12-02 2014-04-23 乐视致新电子科技(天津)有限公司 Method and system for performing monitoring through smart television
CN105893066A (en) * 2014-12-19 2016-08-24 上海涌杰信息科技有限公司 Android system based mobile terminal recovery control method
WO2017219532A1 (en) * 2016-06-24 2017-12-28 乐视控股(北京)有限公司 Video application program control method and video application program control system
CN107690083A (en) * 2017-08-10 2018-02-13 深圳创维-Rgb电子有限公司 A kind of safety protection method based on intelligent TV set, storage medium and intelligent television
CN108173873A (en) * 2018-01-26 2018-06-15 创盛视联数码科技(北京)有限公司 A kind of Android realizes the solution of record screen plug-flow acquired with camera
CN111049908A (en) * 2019-12-12 2020-04-21 中国建设银行股份有限公司 Data pre-caching method and device for mobile equipment
CN111782347A (en) * 2019-04-03 2020-10-16 安克创新科技股份有限公司 Mobile phone application keep-alive method, device and system and storage medium
CN111885407A (en) * 2020-07-30 2020-11-03 深圳创维-Rgb电子有限公司 Standby control method and device, smart television and readable storage medium
CN111966425A (en) * 2020-08-19 2020-11-20 惠州Tcl云创科技有限公司 Process cleaning method and device, storage medium and mobile terminal

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103747205A (en) * 2013-12-02 2014-04-23 乐视致新电子科技(天津)有限公司 Method and system for performing monitoring through smart television
CN105893066A (en) * 2014-12-19 2016-08-24 上海涌杰信息科技有限公司 Android system based mobile terminal recovery control method
WO2017219532A1 (en) * 2016-06-24 2017-12-28 乐视控股(北京)有限公司 Video application program control method and video application program control system
CN107690083A (en) * 2017-08-10 2018-02-13 深圳创维-Rgb电子有限公司 A kind of safety protection method based on intelligent TV set, storage medium and intelligent television
CN108173873A (en) * 2018-01-26 2018-06-15 创盛视联数码科技(北京)有限公司 A kind of Android realizes the solution of record screen plug-flow acquired with camera
CN111782347A (en) * 2019-04-03 2020-10-16 安克创新科技股份有限公司 Mobile phone application keep-alive method, device and system and storage medium
CN111049908A (en) * 2019-12-12 2020-04-21 中国建设银行股份有限公司 Data pre-caching method and device for mobile equipment
CN111885407A (en) * 2020-07-30 2020-11-03 深圳创维-Rgb电子有限公司 Standby control method and device, smart television and readable storage medium
CN111966425A (en) * 2020-08-19 2020-11-20 惠州Tcl云创科技有限公司 Process cleaning method and device, storage medium and mobile terminal

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
08_CARMELO: "Android进程保活实践", Retrieved from the Internet <URL:https://www.jianshu.com/p/53c4d8303e19?hmsr=toutiao.io&utm_medium=toutiao.io> *

Similar Documents

Publication Publication Date Title
CN108491275B (en) Program optimization method, device, terminal and storage medium
US10726068B2 (en) App processing method and apparatus
CN107454416B (en) Video stream sending method and device
US20220167036A1 (en) Live broadcast method and apparatus, and computer device and storage medium
CN111629239B (en) Screen projection processing method, device, equipment and computer readable storage medium
CN109309842B (en) Live broadcast data processing method and device, computer equipment and storage medium
CN111240620A (en) Intelligent terminal screen projection processing method and device, computer equipment and medium
CN111803940A (en) Game processing method and device, electronic equipment and computer-readable storage medium
CN111093094A (en) Video transcoding method, device and system, electronic equipment and readable storage medium
CN113542757A (en) Image transmission method and device for cloud application, server and storage medium
CN112584049A (en) Remote interaction method and device, electronic equipment and storage medium
CN111880865A (en) Multimedia data pushing method and device, electronic equipment and storage medium
CN112799891A (en) iOS device test method, device, system, storage medium and computer device
CN114125482A (en) Live broadcast and microphone connection processing method, electronic equipment and storage medium
CN112351093A (en) Screenshot image sharing method, device and equipment and computer readable storage medium
CN113645476B (en) Picture processing method and device, electronic equipment and storage medium
WO2024051148A1 (en) Cloud game control method and apparatus, electronic device, and storage medium
CN114222164A (en) Video plug-flow method, device, intelligent equipment and storage medium
CN116347158A (en) Video playing method and device, electronic equipment and computer readable storage medium
CN112672035A (en) Thumbnail generation method and terminal
CN110913213B (en) Method, device and system for evaluating and processing video quality
CN109391817A (en) Data texturing transmission method and device
CN111930328A (en) Application data display method, device and system
CN113938457B (en) Method, system and equipment for cloud mobile phone to apply remote camera
CN114390300B (en) Live broadcast control method and device, electronic equipment and computer storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination