CN114051095A - Remote processing method of video stream data and shooting system - Google Patents
Remote processing method of video stream data and shooting system Download PDFInfo
- Publication number
- CN114051095A CN114051095A CN202111341903.2A CN202111341903A CN114051095A CN 114051095 A CN114051095 A CN 114051095A CN 202111341903 A CN202111341903 A CN 202111341903A CN 114051095 A CN114051095 A CN 114051095A
- Authority
- CN
- China
- Prior art keywords
- shooting
- video stream
- stream data
- data
- path
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 34
- 238000004458 analytical method Methods 0.000 claims abstract description 20
- 238000003909 pattern recognition Methods 0.000 claims abstract description 10
- 238000012549 training Methods 0.000 claims abstract description 10
- 238000012545 processing Methods 0.000 claims description 44
- 238000004422 calculation algorithm Methods 0.000 claims description 30
- 238000000034 method Methods 0.000 claims description 21
- 238000005516 engineering process Methods 0.000 claims description 18
- 230000005540 biological transmission Effects 0.000 claims description 10
- 238000013135 deep learning Methods 0.000 claims description 8
- 230000008921 facial expression Effects 0.000 claims description 8
- 230000033001 locomotion Effects 0.000 claims description 8
- 230000008569 process Effects 0.000 claims description 6
- 238000009877 rendering Methods 0.000 claims description 6
- 230000004044 response Effects 0.000 abstract description 2
- 238000000926 separation method Methods 0.000 abstract description 2
- 230000036544 posture Effects 0.000 description 7
- 238000013528 artificial neural network Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 5
- 238000004891 communication Methods 0.000 description 3
- RZVHIXYEVGDQDX-UHFFFAOYSA-N 9,10-anthraquinone Chemical compound C1=CC=C2C(=O)C3=CC=CC=C3C(=O)C2=C1 RZVHIXYEVGDQDX-UHFFFAOYSA-N 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 230000006835 compression Effects 0.000 description 2
- 238000007906 compression Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 241000251468 Actinopterygii Species 0.000 description 1
- 235000014653 Carica parviflora Nutrition 0.000 description 1
- 241000243321 Cnidaria Species 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000009432 framing Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/64—Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
- G06Q10/047—Optimisation of routes or paths, e.g. travelling salesman problem
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/695—Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Business, Economics & Management (AREA)
- Human Resources & Organizations (AREA)
- General Physics & Mathematics (AREA)
- Biophysics (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Biomedical Technology (AREA)
- Signal Processing (AREA)
- Multimedia (AREA)
- Strategic Management (AREA)
- Economics (AREA)
- Game Theory and Decision Science (AREA)
- Development Economics (AREA)
- Entrepreneurship & Innovation (AREA)
- Marketing (AREA)
- Operations Research (AREA)
- Quality & Reliability (AREA)
- Tourism & Hospitality (AREA)
- General Business, Economics & Management (AREA)
- Studio Devices (AREA)
Abstract
The invention provides a remote processing method of video stream data, which comprises the following steps: s11: remotely receiving video stream data of a shooting device; s12: decoding video stream data; s13: analyzing the decoded video stream data for one or more of the following operations: pattern recognition, calculating an optimal shooting path and calculating an optimal shooting parameter of a current scene; s14: an instruction corresponding to the analysis result of step S13 is sent to the photographing apparatus. By adopting the design scheme of the invention, the shooting path can be planned in real time, the shooting parameters can be adjusted, the specialty degree and the shooting quality of automatic shooting can be improved, the operation pressure of an unmanned aerial vehicle or an underwater robot system can be reduced, the hardware configuration cost and the system power consumption can be reduced, the separation of intelligent mode recognition, video stream acquisition and target recognition can be realized, the mode recognition and the response of related business operation can be more accurately carried out, more types of recognition can be rapidly expanded through background training, the requirement of business customization can be met, and the customer experience degree can be greatly improved.
Description
Technical Field
The present disclosure relates to the field of image processing, and in particular, to a method for remotely processing video stream data and a shooting system.
Background
With the rapid development of intelligent hardware, internet, streaming media and artificial intelligence technologies, how to better combine the technologies makes the intelligent hardware products more intelligent and humanized, and brings better experience for users, which is a great trend and expected by users. In the existing products, the hardware configuration almost has no large breakthrough space, so the degree of intelligence of the software directly affects the market competitiveness and the customer experience of the products.
The shooting function aiming at the existing unmanned aerial vehicle and underwater robot is characterized in that the shooting modes such as focus following, intelligent point following and interest point surrounding of a simple scene are included, when shooting is conducted in the face of complex environments and scenes, professional flyers and special robot operators are almost needed to assist in completing the shooting, time and labor are wasted, especially for novice, a relatively professional and good video is needed to be shot in different scenes, and the difficulty in operation is large no matter from path planning, the shooting method selection of different scenes or the camera parameter setting aiming at different shooting scenes.
The traditional path planning has higher requirement on hardware computing power of the robot, the cost for increasing hardware configuration needs to be considered for improving the hardware computing power, the high configuration also increases the power consumption of the system, the traditional path planning and shooting mode can only be used for simple shooting, intelligent professional shooting in any scene cannot be realized, and the expandability cost and difficulty of the system are higher.
How to better assist people to select, plan, track and identify various scenes which a user wants to shoot more intelligently and more accurately, and how to carry out intelligent professional shooting on scenes which the user wants to shoot more intelligently, the intelligent shooting method under more complex shooting scenes is rapidly expanded and upgraded, the hardware configuration cost of the system of the robot is reduced, and the power consumption of the system is reduced, which is a problem to be solved.
In addition, aiming at the current gesture recognition and shooting technology of unmanned aerial vehicles and robots, the traditional gesture recognition is based on a mature gesture recognition algorithm, the requirement on hardware computing power of the unmanned aerial vehicles or the robots is higher, the hardware computing power needs to be improved, the cost of hardware configuration needs to be increased, the power consumption of the system is increased due to the high configuration, and the expandability of the system and the recognition difficulty for increasing other types of modes are higher.
How to better assist people to recognize gestures, postures, mouth shapes and facial expressions more intelligently and more accurately, execute different services (for example, automatically shoot or shoot videos by using different gestures) according to specific recognition results, how to rapidly expand, upgrade and recognize more gestures, postures, mouth shapes and facial expressions, customize corresponding service operations, reduce hardware configuration cost of a robot system, and reduce power consumption of the system is also a problem to be solved.
The statements in this background section merely disclose technology known to the inventors and do not, of course, represent prior art in the art.
Disclosure of Invention
In view of one or more of the above-mentioned drawbacks, the present invention provides a method for remote processing of video stream data, comprising:
s11: remotely receiving video stream data of a shooting device;
s12: decoding the video stream data;
s13: analyzing the decoded video stream data for one or more of the following operations: pattern recognition, calculating an optimal shooting path and calculating an optimal shooting parameter of a current scene;
s14: an instruction corresponding to the analysis result of the step S13 is sent to the photographing apparatus.
According to an aspect of the present invention, the step S11 includes: and based on a streaming media transmission technology, remotely receiving video stream data of the shooting device.
According to an aspect of the present invention, the step S12 includes: and decoding the video stream data, dividing the video stream data into the same first video stream data and second video stream data, rendering the first video stream data and then sending the first video stream data to a display module.
According to an aspect of the present invention, the step S12 includes: and sending the second video stream data to an SDK analysis module to analyze and process the content and the scene of the video stream.
According to an aspect of the present invention, the step S13 includes: the AI algorithm module is used for analyzing and model matching the second video stream data after passing through the SDK analysis module, identifying a specific mode based on a mode identification algorithm model, calculating an optimal shooting path based on a path planning algorithm model, and calculating optimal shooting parameters of the current scene based on a scene shooting algorithm model.
According to an aspect of the invention, the remote processing method further comprises: and training a path planning algorithm model and a scene shooting algorithm model through an automatic scene deep learning technology, an automatic path planning technology and an intelligent scene shooting technology.
According to an aspect of the invention, the remote processing method further comprises: the photographing apparatus starts photographing or stops photographing according to a specific pattern obtained by pattern recognition.
According to an aspect of the invention, the specific pattern comprises one or more of a gesture, a posture, a mouth shape, and a facial expression.
According to an aspect of the invention, the remote processing method further comprises:
s15: and the shooting equipment fuses the optimal shooting path data and binocular vision data and calibrates the flight path.
According to an aspect of the invention, the remote processing method further comprises:
s16: and the shooting equipment automatically adjusts according to the optimal shooting parameters of the current scene.
According to an aspect of the invention, the remote processing method further comprises:
s17: the shooting equipment automatically acquires video stream data based on the adjusted shooting parameters based on the calibrated optimal shooting path movement, and then repeats the steps S11-S17 until shooting is finished;
s18: and the shooting equipment sends an instruction for closing the SDK analysis module and the AI algorithm module.
According to one aspect of the invention, the shooting device is an unmanned aerial vehicle or an underwater robot.
The invention also relates to a shooting system comprising:
a photographing apparatus including:
the shooting module is used for collecting image or video stream data; and
the first data processing module is used for fusing shooting path data and binocular vision data, calibrating a track and adjusting parameters of the shooting module;
a processing device, the processing device comprising:
the second data processing module is used for decoding the image or video stream data, analyzing the content and the scene of the video stream, analyzing the decoded image or video stream data through the AI algorithm module, identifying a specific mode and/or calculating the optimal shooting path and the optimal shooting parameters of the current scene, and rendering the decoded video stream data; and
the display module is used for displaying the rendered video stream;
wherein the first data processing module is configured to send video stream data to the second data processing module, and receive instructions of the specific mode and/or the best shooting path data and the best shooting parameters of the current scene sent back by the second data processing module.
According to an aspect of the present invention, the photographing apparatus further comprises:
and the control module is used for controlling the shooting equipment to start shooting, stop shooting, move based on the calibrated optimal shooting path and acquire images or video stream data based on the adjusted shooting parameters.
According to one aspect of the invention, the shooting device is an unmanned aerial vehicle or an underwater robot, and the processing device is a terminal device.
By adopting the remote processing method or the shooting system designed by the invention, the shooting path can be planned in real time, the shooting parameters can be adjusted, the automatic shooting specialty and the shooting quality can be improved, the operation pressure of an unmanned aerial vehicle or an underwater robot system can be reduced, and the hardware configuration cost and the system power consumption can be reduced. The unmanned aerial vehicle or the underwater robot uses an automatic path planning and intelligent scene shooting method based on high-speed image transmission, the real-time shooting path and the intelligent scene shooting function of AI intelligent analysis planning are realized, the shooting path is more accurately planned, shooting camera parameters are intelligently adjusted through real-time scene analysis, and shooting methods under more environments and scenes can be adapted to be added through background deep learning training, so that the specialty degree and the shooting quality of automatic shooting are improved; in addition, the automatic photographing and video recording method based on gesture, posture, mouth shape and facial expression recognition realizes AI deep learning intelligent recognition with separation of intelligent mode recognition, video stream acquisition and target recognition, more accurately performs mode recognition and response related business operation, can rapidly expand recognition of more types through background training, meets the business customization requirements of customers, and greatly improves the customer experience.
Drawings
The accompanying drawings, which are included to provide a further understanding of the disclosure, illustrate embodiments of the disclosure and together with the description serve to explain the disclosure and are not to limit the disclosure. In the drawings:
FIG. 1 shows a flow diagram of a method for remote processing of video stream data according to one embodiment of the invention;
fig. 2 shows a flow chart of a method of remote processing of video stream data according to another embodiment of the invention;
fig. 3 shows a flow chart of a method of remote processing of video stream data according to another embodiment of the invention;
fig. 4 shows a block schematic diagram of a photographing system according to an embodiment of the present invention.
Detailed Description
In the following, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments may be modified in various different ways, all without departing from the spirit or scope of the present invention. Accordingly, the drawings and description are to be regarded as illustrative in nature, and not as restrictive.
In the description of the present invention, it is to be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", and the like, indicate orientations and positional relationships based on those shown in the drawings, and are used only for convenience of description and simplicity of description, and do not indicate or imply that the device or element being referred to must have a particular orientation, be constructed and operated in a particular orientation, and thus, should not be considered as limiting the present invention. Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, features defined as "first", "second", may explicitly or implicitly include one or more of the described features. In the description of the present invention, "a plurality" means two or more unless specifically defined otherwise.
In the description of the present invention, it should be noted that unless otherwise explicitly stated or limited, the terms "mounted," "connected," and "connected" are to be construed broadly, e.g., as meaning either a fixed connection, a removable connection, or an integral connection, either mechanically, electrically, or in communication with each other; either directly or indirectly through intervening media, either internally or in any other relationship. The specific meanings of the above terms in the present invention can be understood by those skilled in the art according to specific situations.
In the present invention, unless otherwise expressly stated or limited, "above" or "below" a first feature means that the first and second features are in direct contact, or that the first and second features are not in direct contact but are in contact with each other via another feature therebetween. Also, the first feature being "on," "above" and "over" the second feature includes the first feature being directly on and obliquely above the second feature, or merely indicating that the first feature is at a higher level than the second feature. A first feature being "under," "below," and "beneath" a second feature includes the first feature being directly above and obliquely above the second feature, or simply meaning that the first feature is at a lesser level than the second feature.
The following disclosure provides many different embodiments or examples for implementing different features of the invention. To simplify the disclosure of the present invention, the components and arrangements of specific examples are described below. Of course, they are merely examples and are not intended to limit the present invention. Furthermore, the present invention may repeat reference numerals and/or letters in the various examples, such repetition is for the purpose of simplicity and clarity and does not in itself dictate a relationship between the various embodiments and/or configurations discussed. In addition, the present invention provides examples of various specific processes and materials, but one of ordinary skill in the art may recognize applications of other processes and/or uses of other materials.
The preferred embodiments of the present invention will be described in conjunction with the accompanying drawings, and it will be understood that they are described herein for the purpose of illustration and explanation and not limitation.
Fig. 1 shows a remote processing method of video stream data according to an embodiment of the present invention, where the remote processing method 10 includes:
the video stream data of the photographing apparatus is remotely received in step S11. The shooting device is, for example, a camera arranged on an unmanned aerial vehicle or an underwater robot, and video stream data shot by the camera is remotely received through a ground receiving device, such as a remote control terminal or an intelligent terminal. Taking an unmanned aerial vehicle as an example, firstly, the unmanned aerial vehicle acquires a video image through a camera carried by the unmanned aerial vehicle, then codes the video image, and acquires video stream data in a format conforming to a stream media transmission protocol, wherein the video stream data adopts, for example, data in a YUV format compressed by H264 coding; and finally, transmitting the video stream data through a wireless image transmission transmitting module, and remotely receiving the video stream data by ground receiving equipment. The streaming media transport protocol, such as RTSP, RTMP, MMS, HLS, etc., is used to implement the real-time transport function, and the present invention does not limit the streaming media transport protocol.
According to a preferred embodiment of the present invention, step S11 includes: and based on a streaming media transmission technology, remotely receiving video stream data of the shooting device. By taking an unmanned aerial vehicle as an example, the video images are subjected to compression coding and protocol packaging through the data processing module to form individual compression packets, and then continuous and real-time transmission is carried out, and the ground receiving equipment carries out remote receiving. The communication link for image transmission between the shooting device and the ground receiving device can be based on a WIFI or 4G/5G mode, or a special communication link is established.
The video stream data is decoded in step S12. Specifically, the ground receiving device receives the compressed packet data, firstly analyzes the header data of the video stream, acquires relevant parameters of the coded image, such as frame coding type, image width/height and the like, then circularly decodes in units of macro blocks, performs inverse quantization processing, inverse transformation, directional interpolation and motion compensation according to pixel precision and motion direction information and the like in coding, and finally acquires the decoded video stream data. The video stream data consists of a series of spatially and temporally consecutive video frames.
According to a preferred embodiment of the present invention, step S12 further includes: decoding video stream data, dividing the video stream data into the same first video stream data and second video stream data, rendering the first video stream data and then sending the first video stream data to a display module. For example, the decoded video stream data is divided into two paths for different data processing. The video frame in the first video stream data is decomposed into, for example, a value of Y, U, V, and the video frame is rendered and then sent to a display module (for example, a display screen of a mobile phone) for playing, so that a user can view the video frame in real time. Preferably, after the first video stream data is subjected to framing processing, each frame of video image is acquired, and video image enhancement processing, such as denoising processing, picture enhancement, brightness adjustment, and the like, is performed on each frame of video image. The frame image can be output to be played after processing one frame image, and the playing quality of the video image is improved by attaching the streaming media transmission technology.
According to a preferred embodiment of the present invention, step S12 further includes: and sending the second video stream data to an SDK (software Development kit) analysis module to analyze and process the content and the scene of the video stream. And analyzing the content and the scene of the video image in the second video stream data by using an automatic path planning and intelligent scene shooting SDK analysis module based on deep learning. The content to be analyzed is, for example, a target person, an environmental obstacle, and the scene to be analyzed is, for example, a rainy day scene, a sunset scene, a forest scene, or the like.
The decoded video stream data is analyzed in step S13, performing one or more of the following operations: pattern recognition, calculation of the optimal shooting path and calculation of the optimal shooting parameters of the current scene. Pattern recognition includes, among other things, recognizing specific gestures, postures, mouth shapes, facial expressions, etc. The data processing module acquires an optimal shooting path and optimal shooting parameters of the current scene based on the analyzed content and scene of the video stream, and performs automatic path planning and intelligent scene shooting. The shooting path and the shooting scene are shooting with motion trail, shooting along coral reef scene, shooting with tracking fish swarm scene, shooting in rainy day scene, shooting in sunset scene, shooting in grassland scene, shooting in static fixed point scene, or shooting with tracking specified target scene.
According to a preferred embodiment of the present invention, step S13 further includes: the AI algorithm module is used for analyzing and model matching the second video stream data after passing through the SDK analysis module, identifying a specific mode based on a mode identification algorithm model, calculating an optimal shooting path based on a path planning algorithm model, and calculating optimal shooting parameters of the current scene based on a scene shooting algorithm model. Preferably, the recognizing of the specific pattern based on the pattern recognition algorithm model includes connecting to the internet through a ground photographing device, recognizing a gesture, a posture, a mouth shape, a facial expression, etc. rapidly and precisely based on an AI depth image recognition technology, and expanding the recognized pattern type according to product requirements.
An instruction corresponding to the analysis result of step S13 is transmitted to the photographing apparatus at step S14. The intelligently planned optimal shooting path and the optimal shooting parameters of the current scene are transmitted to shooting equipment in an instruction mode, for example, an unmanned aerial vehicle or an underwater robot, and after the unmanned aerial vehicle or the underwater robot obtains the acquisition instruction, the shooting track is corrected and the current shooting parameters are corrected.
In summary, the remote processing method 10 implements AI intelligent analysis and planning of a real-time shooting path by adopting pattern recognition based on high-speed map transmission, automatic path planning and automatic scene shooting through steps S11-S14, and improves the specialty and shooting quality of automatic shooting by intelligently adjusting shooting parameters through real-time scene analysis. The present invention does not limit the sequence of execution of each step, nor the specific operation in the step, for example, in step 12, the video stream data is divided into the same first video stream data and second video stream data, and the subsequent processing of the first video stream data and the second video stream data may be performed synchronously or may have a sequence, which are all within the protection scope of the present invention.
According to a preferred embodiment of the present invention, the remote processing method 10 further comprises: and training a path planning algorithm model and a scene shooting algorithm model through an automatic scene deep learning technology, an automatic path planning technology and an intelligent scene shooting technology. Specifically, an environment map can be constructed in advance, in a training stage, a neural network based on deep learning is constructed firstly, after a training path is set, the deep neural network outputs a planned path according to a video image shot by a camera, and parameters of the deep neural network are optimized according to an execution effect; and then, replacing the training path, and carrying out different path planning training to obtain a path planning algorithm model based on the deep neural network. Similarly, shooting parameters are output according to video images based on different scenes, parameters of the deep neural network are optimized according to shooting effects, and a scene shooting algorithm model based on the deep neural network is obtained. Preferably, the path planning algorithm model and the scene shooting algorithm model can be trained simultaneously, and can be trained through background deep learning, so that the shooting method under more environments and scenes is adapted to be added, and the specialty degree and the shooting quality of automatic shooting are improved. And finally, in an execution stage, analyzing and processing the acquired video stream based on the trained automatic path planning and intelligent scene shooting SDK analysis module.
According to a preferred embodiment of the present invention, the remote processing method 10 further comprises: the photographing apparatus starts photographing or stops photographing according to a specific pattern obtained by the pattern recognition. For example, when a 'V' gesture is recognized, a function of automatically taking a picture or automatically taking a video is started; the capture function is stopped when the "OK" gesture is recognized. Preferably, different services are executed according to the identified specific results, and networking extensions can be upgraded to identify more modes and customize the corresponding service operation.
According to a preferred embodiment of the present invention, the specific mode in the remote processing method 10 includes one or more of a gesture, a posture, a mouth shape, and a facial expression.
Fig. 2 shows a flowchart of a remote processing method of video stream data according to another embodiment of the present invention, where the remote processing method 20 further includes, compared to the remote processing method 10: the photographing apparatus fuses the optimal photographing path data with the binocular vision data and performs track calibration at step S15. The method comprises the steps of obtaining scene three-dimensional information through binocular vision, and further obtaining a scene map of a current position. And the shooting device, such as an unmanned aerial vehicle, generates a main shooting path based on the acquired optimal shooting path and the scene map, and automatically corrects the shooting track and the shooting angle of the camera holder.
According to a preferred embodiment of the present invention, the remote processing method 20 further comprises: the photographing apparatus is automatically adjusted according to the current scene optimum photographing parameters in step S16. And the shooting equipment, such as an unmanned aerial vehicle or an underwater robot, acquires the instruction of the optimal shooting parameter and corrects the current shooting parameter based on the optimal shooting parameter.
According to a preferred embodiment of the present invention, the remote processing method 20 further includes steps S17 and S18. Wherein the photographing apparatus automatically captures video stream data based on the adjusted photographing parameters based on the calibrated optimal photographing path movement in step S17, and then repeats steps S11-S17 until photographing ends. In step S18, the photographing apparatus transmits an instruction to turn off the SDK analysis module and the AI algorithm module.
Fig. 3 shows a flow chart of a remote processing method of video stream data according to another embodiment of the present invention, and the processing device and the drone or the robot jointly execute the remote processing method 30, where the terminal device remotely receives the video stream data shot by the drone or the robot, and the second data processing module of the terminal device decodes the video stream data and divides the decoded video stream data into two paths: rendering one path of video stream data and then sending the rendered video stream data to a display module of the terminal equipment; and analyzing the content and scene of the other path of video stream data to obtain the optimal shooting path and the optimal shooting parameters of the current scene, and sending the analysis result to the unmanned aerial vehicle or the robot in the form of an instruction by the terminal equipment. The following steps are performed at the unmanned aerial vehicle or robot: the first data processing module of the unmanned aerial vehicle or the robot fuses the received command and binocular vision data and calibrates the flight path, on one hand, the flight control or power control parameter data after the fusion and the flight path calibration are transmitted to the flight control module of the unmanned aerial vehicle or the power control module of the robot through the control module interface so as to adjust the flight state or the motion state of the unmanned aerial vehicle or the robot; and on the other hand, shooting parameters are adjusted. The unmanned aerial vehicle or the robot and the terminal device transmit video stream data in real time and process the analyzed data, and the motion parameters and shooting parameters of the unmanned aerial vehicle or the robot are continuously adjusted, for example, shooting tracks of the unmanned aerial vehicle and shooting angles of a camera cloud deck are automatically corrected. By planning the shooting path and adjusting the shooting parameters in real time, the specialty degree and the shooting quality of automatic shooting are improved, and the operation pressure of an unmanned aerial vehicle or a robot system is reduced.
In summary, with the remote processing methods 10 and 20, the computation pressure of the shooting device can be reduced, the hardware configuration cost of the shooting device can be reduced, and the power consumption of the system can be reduced.
According to a preferred embodiment of the invention, the photographing apparatus is an unmanned aerial vehicle or an underwater robot.
The invention also relates to a shooting system 40, with reference to fig. 4, comprising:
a photographing apparatus 41, the photographing apparatus 41 including:
a shooting module 411, configured to collect image or video stream data; and
the first data processing module 412 is configured to perform fusion, track calibration and parameter adjustment on the shooting path data and the binocular vision data;
a processing device 42, said processing device 42 comprising:
a second data processing module 421, configured to decode the image or video stream data, analyze the content and scene of the video stream, analyze the decoded image or video stream data through an AI algorithm module, identify a specific mode and/or calculate an optimal shooting path and an optimal shooting parameter of a current scene, and render the decoded video stream data; and
a display module 422, configured to display the rendered video stream;
wherein, the first data processing module 412 is configured to send video stream data to the second data processing module 421, and receive an instruction of a specific mode and/or best shooting path data and current scene best shooting parameters sent back by the second data processing module 421.
According to a preferred embodiment of the present invention, the photographing apparatus 41 further includes:
a control module 413 for controlling the photographing apparatus 41 to start photographing, stop photographing, move based on the calibrated optimal photographing path, and control the photographing module 41 to capture image or video stream data based on the adjusted photographing parameters.
According to a preferred embodiment of the present invention, the shooting device 41 is an unmanned aerial vehicle or an underwater robot, and the processing device 42 is a terminal device.
According to a preferred embodiment of the invention, the method is implemented by a receiving device as described above.
Finally, it should be noted that: although the present invention has been described in detail with reference to the foregoing embodiments, it will be apparent to those skilled in the art that changes may be made in the embodiments and/or equivalents thereof without departing from the spirit and scope of the invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (15)
1. A method of remote processing of video stream data, comprising:
s11: remotely receiving video stream data of a shooting device;
s12: decoding the video stream data;
s13: analyzing the decoded video stream data for one or more of the following operations: pattern recognition, calculating an optimal shooting path and calculating an optimal shooting parameter of a current scene;
s14: an instruction corresponding to the analysis result of the step S13 is sent to the photographing apparatus.
2. The remote processing method according to claim 1, wherein the step S11 includes: and based on a streaming media transmission technology, remotely receiving video stream data of the shooting device.
3. The remote processing method according to claim 1, wherein the step S12 includes: and decoding the video stream data, dividing the video stream data into the same first video stream data and second video stream data, rendering the first video stream data and then sending the first video stream data to a display module.
4. The remote processing method according to claim 3, wherein the step S12 includes: and sending the second video stream data to an SDK analysis module to analyze and process the content and the scene of the video stream.
5. The remote processing method according to claim 4, wherein the step S13 includes: the AI algorithm module is used for analyzing and model matching the second video stream data after passing through the SDK analysis module, identifying a specific mode based on a mode identification algorithm model, calculating an optimal shooting path based on a path planning algorithm model, and calculating optimal shooting parameters of the current scene based on a scene shooting algorithm model.
6. The remote processing method of claim 5, further comprising: and training a path planning algorithm model and a scene shooting algorithm model through an automatic scene deep learning technology, an automatic path planning technology and an intelligent scene shooting technology.
7. The remote processing method of claim 1, further comprising: the photographing apparatus starts photographing or stops photographing according to a specific pattern obtained by pattern recognition.
8. The method of claim 7, the particular pattern comprising one or more of a gesture, a posture, a mouth shape, and a facial expression.
9. The remote processing method of claim 1, further comprising:
s15: and the shooting equipment fuses the optimal shooting path data and binocular vision data and calibrates the flight path.
10. The processing method of claim 9, the remote processing method further comprising:
s16: and the shooting equipment automatically adjusts according to the optimal shooting parameters of the current scene.
11. The remote processing method of claim 10, further comprising:
s17: the shooting equipment automatically acquires video stream data based on the adjusted shooting parameters based on the calibrated optimal shooting path movement, and then repeats the steps S11-S17 until shooting is finished;
s18: and the shooting equipment sends an instruction for closing the SDK analysis module and the AI algorithm module.
12. The remote processing method according to any one of claims 1 to 11, wherein the photographing apparatus is a drone or an underwater robot.
13. A camera system, comprising:
a photographing apparatus including:
the shooting module is used for collecting image or video stream data; and
the first data processing module is used for fusing shooting path data and binocular vision data, calibrating a track and adjusting parameters of the shooting module;
a processing device, the processing device comprising:
the second data processing module is used for decoding the image or video stream data, analyzing the content and the scene of the video stream, analyzing the decoded image or video stream data through the AI algorithm module, identifying a specific mode and/or calculating the optimal shooting path and the optimal shooting parameters of the current scene, and rendering the decoded video stream data; and
the display module is used for displaying the rendered video stream;
wherein the first data processing module is configured to send video stream data to the second data processing module, and receive instructions of the specific mode and/or the best shooting path data and the best shooting parameters of the current scene sent back by the second data processing module.
14. The photographing system of claim 13, the photographing apparatus further comprising:
and the control module is used for controlling the shooting equipment to start shooting, stop shooting, move based on the calibrated optimal shooting path and acquire images or video stream data based on the adjusted shooting parameters.
15. The shooting system of claim 13 or 14, the shooting device being an unmanned aerial vehicle or an underwater robot, the processing device being a terminal device.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111341903.2A CN114051095A (en) | 2021-11-12 | 2021-11-12 | Remote processing method of video stream data and shooting system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111341903.2A CN114051095A (en) | 2021-11-12 | 2021-11-12 | Remote processing method of video stream data and shooting system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114051095A true CN114051095A (en) | 2022-02-15 |
Family
ID=80208944
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111341903.2A Pending CN114051095A (en) | 2021-11-12 | 2021-11-12 | Remote processing method of video stream data and shooting system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114051095A (en) |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2011114713A (en) * | 2009-11-27 | 2011-06-09 | Canon Inc | Photographing apparatus, photographing control method, and computer program |
| CN106774398A (en) * | 2016-12-20 | 2017-05-31 | 北京小米移动软件有限公司 | Take photo by plane method and device, unmanned plane |
| CN106791420A (en) * | 2016-12-30 | 2017-05-31 | 深圳先进技术研究院 | A kind of filming control method and device |
| CN108965689A (en) * | 2017-05-27 | 2018-12-07 | 昊翔电能运动科技(昆山)有限公司 | Unmanned plane image pickup method and device, unmanned plane and ground control unit |
| CN109062400A (en) * | 2018-07-06 | 2018-12-21 | 深圳臻迪信息技术有限公司 | The control method and device of image acquisition device |
| CN109479090A (en) * | 2017-12-22 | 2019-03-15 | 深圳市大疆创新科技有限公司 | Information processing method, unmanned plane, remote control equipment and non-volatile memory medium |
| CN109815844A (en) * | 2018-12-29 | 2019-05-28 | 西安天和防务技术股份有限公司 | Object detection method and device, electronic equipment and storage medium |
| CN110637269A (en) * | 2018-04-27 | 2019-12-31 | 深圳市大疆创新科技有限公司 | Unmanned aerial vehicle control method and control device and unmanned aerial vehicle |
| CN111279684A (en) * | 2018-05-07 | 2020-06-12 | 合刃科技（武汉）有限公司 | Shooting control method and electronic device |
| CN112348034A (en) * | 2020-10-21 | 2021-02-09 | 中电鸿信信息科技有限公司 | Crane defect detection system based on unmanned aerial vehicle image recognition and working method |
| WO2021052292A1 (en) * | 2019-09-18 | 2021-03-25 | 华为技术有限公司 | Video acquisition method and electronic device |
| CN112857356A (en) * | 2021-01-04 | 2021-05-28 | 上海建科环境技术有限公司 | Unmanned aerial vehicle water body environment investigation and air route generation method |
| CN113038023A (en) * | 2017-05-24 | 2021-06-25 | 深圳市大疆创新科技有限公司 | Shooting control method and device |
-
2021
- 2021-11-12 CN CN202111341903.2A patent/CN114051095A/en active Pending
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2011114713A (en) * | 2009-11-27 | 2011-06-09 | Canon Inc | Photographing apparatus, photographing control method, and computer program |
| CN106774398A (en) * | 2016-12-20 | 2017-05-31 | 北京小米移动软件有限公司 | Take photo by plane method and device, unmanned plane |
| CN106791420A (en) * | 2016-12-30 | 2017-05-31 | 深圳先进技术研究院 | A kind of filming control method and device |
| CN113038023A (en) * | 2017-05-24 | 2021-06-25 | 深圳市大疆创新科技有限公司 | Shooting control method and device |
| CN108965689A (en) * | 2017-05-27 | 2018-12-07 | 昊翔电能运动科技(昆山)有限公司 | Unmanned plane image pickup method and device, unmanned plane and ground control unit |
| CN109479090A (en) * | 2017-12-22 | 2019-03-15 | 深圳市大疆创新科技有限公司 | Information processing method, unmanned plane, remote control equipment and non-volatile memory medium |
| CN110637269A (en) * | 2018-04-27 | 2019-12-31 | 深圳市大疆创新科技有限公司 | Unmanned aerial vehicle control method and control device and unmanned aerial vehicle |
| CN111279684A (en) * | 2018-05-07 | 2020-06-12 | 合刃科技（武汉）有限公司 | Shooting control method and electronic device |
| CN109062400A (en) * | 2018-07-06 | 2018-12-21 | 深圳臻迪信息技术有限公司 | The control method and device of image acquisition device |
| CN109815844A (en) * | 2018-12-29 | 2019-05-28 | 西安天和防务技术股份有限公司 | Object detection method and device, electronic equipment and storage medium |
| WO2021052292A1 (en) * | 2019-09-18 | 2021-03-25 | 华为技术有限公司 | Video acquisition method and electronic device |
| CN112348034A (en) * | 2020-10-21 | 2021-02-09 | 中电鸿信信息科技有限公司 | Crane defect detection system based on unmanned aerial vehicle image recognition and working method |
| CN112857356A (en) * | 2021-01-04 | 2021-05-28 | 上海建科环境技术有限公司 | Unmanned aerial vehicle water body environment investigation and air route generation method |
Non-Patent Citations (2)
| Title |
|---|
| 宋飞;杨扬;: "小型无人机视频实时传输的设计与开发", 电子技术与软件工程, no. 03, 1 February 2017 (2017-02-01) * |
| 赵海盟;张文凯;谷静博;王强;沈路宁;晏磊;: "无人机载荷航拍控制系统设计", 计算机应用, no. 01, 10 January 2015 (2015-01-10) * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7764736B2 (en) | Real-time video object generation for smart cameras | |
| US10484621B2 (en) | Systems and methods for compressing video content | |
| US20190335177A1 (en) | Systems and methods for spatially adaptive video encoding | |
| EP3391639B1 (en) | Generating output video from video streams | |
| WO2017215295A1 (en) | Camera parameter adjusting method, robotic camera, and system | |
| CN110463176A (en) | Image quality measure | |
| EP3404913B1 (en) | A system comprising a video camera and a client device and a method performed by the same | |
| CN105187723B (en) | A kind of image pickup processing method of unmanned vehicle | |
| US8903130B1 (en) | Virtual camera operator | |
| CN107071389A (en) | Take photo by plane method, device and unmanned plane | |
| KR20170128498A (en) | Edit background in video conferences | |
| CN102907092A (en) | Camera parameter-assisted video frame rate up conversion | |
| EP3872744A1 (en) | Method and apparatus for obtaining sample image set | |
| WO2021237616A1 (en) | Image transmission method, mobile platform, and computer readable storage medium | |
| US11373685B2 (en) | Event/object-of-interest centric timelapse video generation on camera device with the assistance of neural network input | |
| CN109716769A (en) | The system and method for the scaling of object-oriented are provided in multimedia messages | |
| JP2024504027A (en) | Pose estimation method and related device | |
| US11949982B2 (en) | Method of forming a processed video stream | |
| KR101540113B1 (en) | Method, apparatus for gernerating image data fot realistic-image and computer-readable recording medium for executing the method | |
| CN113838098A (en) | Intelligent tracking shooting system for remote high-speed moving target | |
| CN112132797B (en) | Short video quality screening method | |
| CN111093077A (en) | Video coding method and device, electronic equipment and storage medium | |
| CN114051095A (en) | Remote processing method of video stream data and shooting system | |
| WO2021164082A1 (en) | Video data acquisition controlling method, transmission control apparatus and wireless transmission system | |
| KR20110123907A (en) | Method for providing contents and internet protocol television system thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |