CN117176705B - Industrial camera video stream display method, device, equipment and medium - Google Patents
Industrial camera video stream display method, device, equipment and medium Download PDFInfo
- Publication number
- CN117176705B CN117176705B CN202311451924.9A CN202311451924A CN117176705B CN 117176705 B CN117176705 B CN 117176705B CN 202311451924 A CN202311451924 A CN 202311451924A CN 117176705 B CN117176705 B CN 117176705B
- Authority
- CN
- China
- Prior art keywords
- camera
- calling
- service
- linux
- platform service
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 57
- 230000004927 fusion Effects 0.000 claims abstract description 8
- 238000012986 modification Methods 0.000 claims description 48
- 230000004048 modification Effects 0.000 claims description 48
- 238000012545 processing Methods 0.000 claims description 21
- 238000012423 maintenance Methods 0.000 abstract description 12
- 230000005540 biological transmission Effects 0.000 description 10
- 238000013024 troubleshooting Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 6
- 230000001965 increasing effect Effects 0.000 description 6
- 238000007726 management method Methods 0.000 description 6
- 238000012360 testing method Methods 0.000 description 6
- 238000004590 computer program Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000001514 detection method Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000000977 initiatory effect Effects 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000010223 real-time analysis Methods 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Abstract
The application provides an industrial camera video stream display method, device, equipment and medium, relates to the technical field of computers, and is used for solving the problems of high cost, high delay, difficult maintenance and the like caused by independent deployment of industrial cameras. The method is applied to a hardware platform comprising an industrial camera and an Android/Linux fusion operating system, and comprises the following steps: sending a camera starting request generated by a Web page to an Android terminal through Linux platform service; the Linux platform service adopts a Linux system, and the Android terminal adopts an Android system; according to the camera starting request, calling the Android end to start the industrial camera; calling the Linux platform service to connect with the RTSP service which is started by the Android terminal; transmitting a camera video stream generated by the industrial camera to the Linux platform service through the RTSP service; and calling the Linux platform service to push the camera video stream to the Web page for display in real time through webRTC connection.
Description
Technical Field
The application relates to the technical field of computers and provides an industrial camera video stream display method, device, equipment and medium.
Background
As is well known, in the conventional scheme, when the AI application performs data transmission and processing with the industrial camera, the AI application needs to be deployed on a separate computing device, and debugging of the industrial camera and test AI reasoning also need to be performed separately, and after the industrial camera is debugged, the image data needs to be transmitted to the AI application device for test verification. Such a procedure is very cumbersome and time consuming. Therefore, the following problems are easily faced, firstly, when the industrial camera and the algorithm reasoning application are separately and independently deployed, the independent industrial camera and the algorithm reasoning equipment need to be purchased and maintained, so that the cost is greatly increased; secondly, when the industrial camera and the algorithm inference application are separately and independently deployed, independent equipment configuration, data transmission and coordination work are required, so that the configuration and management difficulty of the system are increased, and the complexity is greatly improved; thirdly, when the industrial camera and the algorithm inference application are separately and independently deployed, the data is required to be transmitted from the industrial camera to the algorithm inference equipment for processing, so that delay of data processing and response can be possibly caused, and the method is not suitable for an application scene requiring real-time performance, and therefore, the problems of delay and real-time performance can be generated; fourth, when the industrial camera and the algorithm inference application are separately and independently deployed, since the system is composed of different devices, when one of the devices fails, it is necessary to perform independent troubleshooting and maintenance, thus increasing the difficulty of troubleshooting and maintenance.
Disclosure of Invention
The application provides an industrial camera video stream display method, device, equipment and medium, which are used for solving the problems of high cost, high delay, difficult maintenance and the like caused by independent deployment of industrial cameras.
In one aspect, an industrial camera video stream exhibition method is provided, applied to a hardware platform including an industrial camera and an Android/Linux converged operating system, the method includes:
sending a camera starting request generated by a Web page to an Android terminal through Linux platform service; the Linux platform service adopts a Linux system, and the Android terminal adopts an Android system;
according to the camera starting request, calling the Android end to start the industrial camera;
calling the Linux platform service to connect with the RTSP service which is started by the Android terminal;
transmitting a camera video stream generated by the industrial camera to the Linux platform service through the RTSP service;
and calling the Linux platform service to push the camera video stream to the Web page for display in real time through webRTC connection.
Optionally, the step of sending, by the Linux platform service, a camera opening request generated by the Web page to the Android terminal includes:
generating a corresponding camera opening request according to camera opening operation of a user on a Web page aiming at the industrial camera;
invoking the Linux platform service to receive the camera starting request;
and calling the Linux platform service to send the camera starting request to the Android terminal.
Optionally, the step of calling the Linux platform service to connect the Android terminal to start the RTSP service includes:
calling the Android terminal to start the RTSP service;
calling the Linux platform service to receive an RTSP service connection request sent by the Android terminal; wherein the RTSP service connection request includes an RTSP address;
and calling the Linux platform service to connect the RTSP service according to the RTSP service connection request.
Optionally, before invoking the Linux platform service to push the camera video stream to the Web page for presentation in real time through WebRTC connection, the method further includes:
calling the Linux platform service to receive a point-to-point connection request sent by a Web page;
and establishing a WebRTC connection between the Linux platform service and the Web page according to the point-to-point request.
Optionally, the hardware platform further has an AI application integrated therein, and the AI application is set in a Linux environment, and the method further includes:
generating a corresponding parameter modification request according to the parameter modification operation of the user on the Web page aiming at the industrial camera; wherein the parameter modification request contains corresponding camera modification parameters;
according to the camera parameter modification request, calling the Linux platform service to send the camera modification parameters to the Android terminal through a corresponding HTTP interface;
according to the camera modification parameters, calling the Android terminal to perform corresponding operation and update on the industrial camera; at the same time, the method comprises the steps of,
and writing the camera modification parameters into an AI application configuration file.
Optionally, after writing the camera modification parameters into an AI application profile, the method further comprises:
according to the AI application configuration file, calling an Android terminal to start AI application to perform corresponding AI processing, and generating an reasoning result;
and calling the Linux platform service to analyze the reasoning result, and pushing the original image and the image corresponding to the reasoning result to the Web page for display in real time through the connection of WebRTC.
Optionally, the step of calling the Android terminal to start the AI application to perform corresponding AI processing and generating an inference result according to the AI application configuration file includes:
and calling the Android terminal to start the AI application to perform image acquisition and reasoning according to the AI application configuration file, and generating a corresponding reasoning result.
In one aspect, there is provided an industrial camera video stream presentation device, the device comprising:
the request sending unit is used for sending a camera starting request generated by the Web page to the Android terminal through the Linux platform service; the Linux platform service adopts a Linux system, and the Android terminal adopts an Android system;
the camera opening unit is used for calling the Android end to open the industrial camera according to the camera opening request;
the RTSP service starting unit is used for calling the Linux platform service to connect the Android terminal to start the RTSP service;
the video stream sending unit is used for sending the camera video stream generated by the industrial camera to the Linux platform service through the RTSP service;
the video stream display unit is used for calling the Linux platform service to push the camera video stream to the Web page for display in real time through webRTC connection.
In one aspect, an electronic device is provided that includes a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing any of the methods described above when executing the computer program.
In one aspect, a computer storage medium having stored thereon computer program instructions which, when executed by a processor, implement any of the methods described above.
Compared with the prior art, the beneficial effects of this application are:
the embodiment of the application provides an industrial camera video stream display method, which is applied to a hardware platform comprising an industrial camera and an Android/Linux fusion operating system, and further, when an industrial camera video stream is required to be displayed, firstly, a camera opening request generated by a Web page can be sent to an Android end through a Linux platform service; secondly, an Android end can be called to start an industrial camera according to a camera starting request; then, the Linux platform service can be called to connect with the RTSP service which is started by the Android terminal; next, a camera video stream generated by the industrial camera may be transmitted to a Linux platform service through an RTSP service; finally, the Linux platform service can be called to push the video stream of the camera to the Web page for display in real time through webRTC connection. Therefore, in the embodiment of the application, since the industrial camera and the Android/Linux fusion operating system are integrated on the same hardware platform, the cost, delay and difficulty of troubleshooting and maintenance can be greatly reduced by simplifying deployment when the industrial camera video stream is displayed. In addition, since the camera video stream is transmitted by using the WebRTC, the delay can be further reduced and the real-time performance can be improved.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the related art, the drawings that are required to be used in the embodiments or the related technical descriptions will be briefly described below, and it is apparent that the drawings in the following description are only embodiments of the present application, and other drawings may be obtained according to the provided drawings without inventive effort for a person having ordinary skill in the art.
Fig. 1 is a schematic view of an application scenario provided in an embodiment of the present application;
FIG. 2 is a schematic diagram of an overall architecture according to an embodiment of the present disclosure;
fig. 3 is a schematic flow chart of an industrial camera video stream display method according to an embodiment of the present application;
fig. 4 is a schematic flow chart of debugging an industrial camera according to an embodiment of the present application;
fig. 5 is a schematic diagram of an industrial camera video stream display device according to an embodiment of the present application.
The marks in the figure: the system comprises a 10-industrial camera video stream display device, a 101-processor, a 102-memory, a 103-I/O interface, a 104-database, a 50-industrial camera video stream display device, a 501-request sending unit, a 502-camera starting unit, a 503-RTSP service starting unit, a 504-video stream sending unit, a 505-video stream display unit, a 506-connection establishing unit, a 507-camera debugging unit and a 508-algorithm reasoning unit.
Detailed Description
For the purposes of making the objects, technical solutions and advantages of the present application more apparent, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure. Embodiments and features of embodiments in this application may be combined with each other arbitrarily without conflict. Also, while a logical order is depicted in the flowchart, in some cases, the steps depicted or described may be performed in a different order than presented herein.
As is well known, in the conventional scheme, when the AI application performs data transmission and processing with the industrial camera, the AI application needs to be deployed on a separate computing device, and debugging of the industrial camera and test AI reasoning also need to be performed separately, and after the industrial camera is debugged, the image data needs to be transmitted to the AI application device for test verification. Such a procedure is very cumbersome and time consuming. Therefore, the following problems are easily faced, firstly, when the industrial camera and the algorithm reasoning application are separately and independently deployed, the independent industrial camera and the algorithm reasoning equipment need to be purchased and maintained, so that the cost is greatly increased; secondly, when the industrial camera and the algorithm inference application are separately and independently deployed, independent equipment configuration, data transmission and coordination work are required, so that the configuration and management difficulty of the system are increased, and the complexity is greatly improved; thirdly, when the industrial camera and the algorithm inference application are separately and independently deployed, the data is required to be transmitted from the industrial camera to the algorithm inference equipment for processing, so that delay of data processing and response can be possibly caused, and the method is not suitable for an application scene requiring real-time performance, and therefore, the problems of delay and real-time performance can be generated; fourth, when the industrial camera and the algorithm inference application are separately and independently deployed, since the system is composed of different devices, when one of the devices fails, it is necessary to perform independent troubleshooting and maintenance, thus increasing the difficulty of troubleshooting and maintenance.
Based on this, the embodiment of the application provides an industrial camera video stream display method, in the method, firstly, a camera opening request generated by a Web page can be sent to an Android terminal through a Linux platform service; secondly, an Android end can be called to start an industrial camera according to a camera starting request; then, the Linux platform service can be called to connect with the RTSP service which is started by the Android terminal; next, a camera video stream generated by the industrial camera may be transmitted to a Linux platform service through an RTSP service; finally, the Linux platform service can be called to push the video stream of the camera to the Web page for display in real time through webRTC connection. Therefore, in the embodiment of the application, since the industrial camera and the Android/Linux fusion operating system are integrated on the same hardware platform, the cost, delay and difficulty of troubleshooting and maintenance can be greatly reduced by simplifying deployment when the industrial camera video stream is displayed. In addition, since the camera video stream is transmitted by using the WebRTC, the delay can be further reduced and the real-time performance can be improved.
After the design concept of the embodiment of the present application is introduced, some simple descriptions are made below for application scenarios applicable to the technical solution of the embodiment of the present application, and it should be noted that the application scenarios described below are only used to illustrate the embodiment of the present application and are not limiting. In the specific implementation process, the technical scheme provided by the embodiment of the application can be flexibly applied according to actual needs.
Fig. 1 is a schematic view of an application scenario provided in an embodiment of the present application. An industrial camera video stream presentation device 10 may be included in the application scenario.
The industrial camera video stream display device 10 may be used for displaying an industrial camera video stream in real time, for example, may be a personal computer (Personal Computer, PC), a server, a portable computer, and the like. The industrial camera video stream display device 10 can include one or more central processing units 101 (Central Processing Unit, CPU), memory 102, I/O interfaces 103, and a database 104. Specifically, the processor 101 may be a central processing unit (central processing unit, CPU), or a digital processing unit or the like. The memory 102 may be a volatile memory (RAM), such as a random-access memory (RAM); the memory 102 may also be a nonvolatile memory (non-volatile memory), such as a read-only memory (rom), a flash memory (flash memory), a hard disk (HDD) or a Solid State Drive (SSD); or memory 102, is any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited to such. The memory 102 may be a combination of the above. The memory 102 may store therein partial program instructions of the industrial camera video stream display method provided in the embodiments of the present application, which when executed by the processor 101, can be used to implement the steps of the industrial camera video stream display method provided in the embodiments of the present application, so as to solve the problems of high cost, high delay and difficult maintenance caused by separate deployment of industrial camera and algorithm reasoning applications. The database 104 may be used to store data related to the scheme provided in the embodiment of the present application, such as a camera start request, a camera video stream, an RTSP service connection request, a parameter modification request, and an inference result.
In the embodiment of the present application, the industrial camera video stream display device 10 may obtain the camera opening request through the I/O interface 103, and then, the processor 101 of the industrial camera video stream display device 10 may reduce the cost, high latency and difficult maintenance by simplifying the deployment of the industrial camera and algorithm reasoning application according to the program instructions of the industrial camera video stream display method provided in the embodiment of the present application in the memory 102. In addition, data such as a camera on request, a camera video stream, an RTSP service connection request, a parameter modification request, and an inference result may be stored in the database 104.
Fig. 2 is a schematic diagram of an overall architecture provided in an embodiment of the present application, where an industrial camera and an AidLux OS operating system are integrated on the same hardware platform, and the AidLux OS operating system is an Android/Linux fusion operating system, where an Android system is used by RTSP services, and a Linux system is used by Linux platform services and AI applications.
In the embodiment of the application, the Android terminal can directly communicate with the industrial camera on the hardware platform and provide functions and control related to the industrial camera, for example, operations such as opening, closing and modifying camera parameters can be directly performed on the industrial camera.
The Linux platform service is deployed in a Linux environment and can communicate with the Android end to control the behavior of the industrial camera by triggering the operation of the industrial camera by the Android end. In addition, the Linux platform service may also be responsible for importing and managing the lifecycle of AI applications.
When a user wants to access and control the industrial camera through the browser, firstly, the user can perform camera opening operation on the Web page to generate a corresponding camera opening request, then the camera opening request can be sent to the Android terminal through the Linux platform service, and the Android terminal can be called to open the industrial camera according to the camera opening request; then, the Linux platform service can be called to connect with the RTSP service which is started by the Android terminal; then, the camera video stream generated by the industrial camera can be sent to a Linux platform service through an RTSP service; finally, the Linux platform service can be called to push the video stream of the camera to the Web page for display in real time through webRTC connection, so that the deployment of industrial cameras and algorithm reasoning application is simplified, and the cost, high delay and difficult maintenance are reduced.
Of course, the method provided in the embodiment of the present application is not limited to the application scenario shown in fig. 1, but may be used in other possible application scenarios, and the embodiment of the present application is not limited. The functions that can be implemented by each device in the application scenario shown in fig. 1 will be described together in the following method embodiments, which are not described in detail herein. The method according to the embodiment of the present application will be described below with reference to the accompanying drawings.
As shown in fig. 3, a flowchart of an industrial camera video stream display method according to an embodiment of the present application is shown, and the method may be performed by the industrial camera video stream display device 10 in fig. 1, and specifically, the flowchart of the method is described below.
Step 301: and sending a camera starting request generated by the Web page to the Android terminal through the Linux platform service.
In the embodiment of the application, the Linux platform service can adopt a Linux system to run and execute the corresponding command, and the Android terminal can adopt an Android system to run and execute the corresponding command.
Specifically, first, a corresponding camera opening request may be generated according to a camera opening operation performed by a user on a Web page for an industrial camera; then, a Linux platform service can be called to receive a camera starting request; and finally, a Linux platform service can be called to send a camera starting request to the Android terminal.
Step 302: and calling the Android end to start the industrial camera according to the camera starting request.
In the embodiment of the application, since the Android terminal can directly perform operations such as opening, closing and modifying camera parameters and the like on the industrial camera, after the Android terminal receives a camera opening request, the Android terminal can directly open the industrial camera according to the camera opening request.
Step 303: and calling a Linux platform service to connect with an Android terminal to start an RTSP service.
In the embodiment of the application, after the industrial camera is started, the RTSP service may be started to wait for the Linux platform service to connect. Specifically, firstly, an Android terminal can be called to start an RTSP service of the Android terminal; then, the Linux platform service can be called to receive an RTSP service connection request sent by the Android terminal; wherein the RTSP service connection request includes an RTSP address; finally, the Linux platform service can be called to connect with the RTSP service according to the RTSP service connection request.
Step 304: and sending the camera video stream generated by the industrial camera to a Linux platform service through an RTSP service.
In the embodiment of the application, after the Linux platform service and the RTSP service are connected, the camera video stream generated by the industrial camera can be directly sent to the Linux platform service through the RTSP service, and at this time, the camera video stream is forwarded through the RTSP protocol, so that the forwarded camera video stream is the RTSP video stream.
Further, in order to display the camera video stream on the Web page, in the embodiment of the present application, a Linux platform service may be invoked to receive a peer-to-peer connection request sent by the Web page; furthermore, according to the point-to-point request, webRTC connection between the Linux platform service and the Web page can be directly established.
Step 305: and calling a Linux platform service to push the video stream of the camera to the Web page in real time through webRTC connection for display.
In practical application, when the front end wants to play the RTSP video stream, it usually converts the RTSP video stream into a video stream that the front end can support by means of FFmpeg transcoding or WebSocket continuous image pushing, because the front end cannot directly play the RTSP video stream during real-time video stream display. However, these methods have problems of large delay and low real-time performance.
Therefore, in the embodiment of the application, after the Linux platform service obtains the camera video stream, in order to reduce delay and improve real-time performance, webRTC may be used to transmit the RTSP video stream in real time, that is, the Linux platform service may be invoked to push the camera video stream to the Web page in real time for display through WebRTC connection.
Because WebRTC uses the TCP reliable transmission protocol and uses optimized transmission and codec algorithms, lower latency transmission can be achieved, and therefore real-time video data of an industrial camera can be transmitted to a Web page in real time for presentation, and a user can view the real-time video stream in real time, providing smoother and instant viewing experience.
In addition, since WebRTC supports two-way communication, users can interact with industrial cameras on Web pages in real time through data channel functions. The operation of the user on the Web page can be transmitted to the Android terminal in real time, so that the corresponding operation and update are triggered. The real-time interactivity enables a user to remotely control and adjust the camera through the control interface, thereby enhancing the interactivity and flexibility between the user and the camera.
Moreover, the real-time video display can be realized on the browser supporting the WebRTC without additional plug-ins or software. Therefore, the deployment process is simplified, the requirements on users are reduced, and better cross-platform compatibility is provided. Therefore, users can view real-time video on various devices, whether on a desktop, cell phone, or tablet computer, using a WebRTC enabled browser.
In one possible implementation manner, in the embodiment of the present application, the hardware platform may perform algorithm reasoning through an integrated AI application (where the AI application is set in a Linux environment) in addition to performing industrial camera video stream presentation through an integrated industrial camera and an Android/Linux fusion operating system. That is, techniques such as machine learning and deep learning can be utilized to debug industrial cameras and test AI reasoning.
Specifically, as shown in fig. 4, a flowchart of the method for debugging an industrial camera according to an embodiment of the present application may be implemented by the industrial camera video stream display device 10 in fig. 1, and specifically, the flowchart of the method is described below.
Step 401: and generating a corresponding parameter modification request according to the parameter modification operation of the user on the Web page aiming at the industrial camera.
In the embodiment of the present application, the parameter modification request includes corresponding camera modification parameters, such as a shooting mode, an aperture value, a shutter speed, and a sensitivity, and the like.
Step 402: and calling a Linux platform service to send the camera modification parameters to the Android terminal through a corresponding HTTP interface according to the camera parameter modification request.
Step 403: and calling the Android terminal to perform corresponding operation and update on the industrial camera according to the camera modification parameters.
Step 404: the camera modification parameters are written into the AI application profile.
In the embodiment of the application, the third party system (AI application) is integrated in the Linux system, so that data exchange and collaborative work with other devices, databases or management systems can be realized, and the comprehensive performance and functions of the whole system are improved.
In addition, in the embodiment of the application, the data of the AI reasoning result is pushed to the third party management system in the FTP and HTTP modes to realize real-time transmission and synchronous update of the data, so that the data generated by the edge terminal equipment is pushed to the central management system, and therefore the data is very convenient to manage, analyze and trace back in a centralized manner. Meanwhile, the flexibility of FTP and HTTP can also adapt to different data transmission requirements and security requirements.
In a possible implementation manner, in the embodiment of the present application, according to the AI application configuration file, the Android end may also be called to start the AI application to perform corresponding AI processing, so as to generate an inference result. Specifically, according to the AI application configuration file, the Android terminal is called to start the AI application to perform image acquisition and reasoning so as to finally generate a corresponding reasoning result. That is, the Android terminal can be called to start the AI application to perform data transmission and processing with the industrial camera, so as to control the industrial camera to perform image acquisition processing, and transmit the acquired image to the AI application to perform reasoning (perform real-time analysis on the acquired image) so as to finally generate a corresponding reasoning result. Therefore, the problems of complicated flow and time consumption caused by separate debugging of industrial cameras and test AI reasoning due to the fact that AI applications are deployed on independent computing devices in the traditional scheme can be avoided.
Furthermore, after the AI application generates the corresponding reasoning result, the Linux platform service can be called to analyze the reasoning result, and the original image (the image shot before the camera parameter is modified) and the image corresponding to the reasoning result (the image shot after the camera parameter is modified) are pushed to the Web page for display in real time through the WebRTC connection, so that a user can watch the original image and the reasoning result image processed by the algorithm at the same time, the user can more intuitively compare the difference and the improvement effect between the original image and the reasoning result image, the user is helped to better understand the action and the effect of the algorithm, and visual references are provided. In addition, the visual presentation mode of the algorithm reasoning result can more intuitively display the output result of the algorithm, so that a user can clearly observe the processing effect of the algorithm on the image. Meanwhile, the user can evaluate the accuracy, efficiency and practicability of the algorithm by comparing the user with the original image.
In one possible implementation, the entire hardware platform may be defined as an edge device suitable for use in the industrial inspection industry. Therefore, in the embodiment of the application, the hardware platform can also be used for detection, monitoring and analysis tasks in an industrial environment, and therefore, the method has the characteristics of high efficiency and real time. And by combining the results of AI application algorithm reasoning with a hardware platform, a more accurate and rapid industrial detection solution can be provided.
In summary, in the embodiment of the application, since the industrial camera and the Android/Linux fusion operating system are integrated on the same hardware platform, the difficulty of cost, delay and troubleshooting and maintenance can be greatly reduced by simplifying deployment when the industrial camera video stream is displayed. In addition, since the camera video stream is transmitted by using the WebRTC, the delay can be further reduced and the real-time performance can be improved.
In addition, because the AI application is integrated in the Linux system of the hardware platform, the data exchange and the cooperative work with other devices, databases or management systems can be realized, thereby improving the comprehensive performance and the functions of the whole system.
Based on the same inventive concept, an embodiment of the present application provides an industrial camera video stream display device 50, as shown in fig. 5, the industrial camera video stream display device 50 includes:
a request sending unit 501, configured to send a camera opening request generated by a Web page to an Android end through a Linux platform service; the Linux platform service adopts a Linux system, and the Android terminal adopts an Android system;
the camera opening unit 502 is used for calling the Android end to open the industrial camera according to the camera opening request;
the RTSP service starting unit 503 is configured to invoke a Linux platform service to connect an Android terminal to start an RTSP service of the Android terminal;
a video stream transmitting unit 504 for transmitting a camera video stream generated by the industrial camera to the Linux platform service through the RTSP service;
the video stream display unit 505 is configured to invoke a Linux platform service to push a video stream to a Web page for display in real time through WebRTC connection.
Optionally, the request sending unit 501 is further configured to:
generating a corresponding camera opening request according to camera opening operation of a user on the Web page aiming at the industrial camera;
calling Linux platform service to receive a camera starting request;
and calling Linux platform service to send a camera starting request to the Android terminal.
Optionally, the RTSP service initiation unit 503 is further configured to:
calling an Android terminal to start an RTSP service;
calling Linux platform service to receive RTSP service connection request sent by an Android terminal; wherein the RTSP service connection request includes an RTSP address;
and calling the Linux platform service to connect with the RTSP service according to the RTSP service connection request.
Optionally, the industrial camera video stream display device 50 further includes a connection establishment unit 506, where the connection establishment unit 506 is configured to:
calling Linux platform service to receive a point-to-point connection request sent by a Web page;
and establishing a WebRTC connection between the Linux platform service and the Web page according to the point-to-point request.
Optionally, the industrial camera video stream presentation device 50 further comprises a camera commissioning unit 507, the camera commissioning unit 507 being configured to:
generating a corresponding parameter modification request according to the parameter modification operation of the user on the Web page aiming at the industrial camera; wherein the parameter modification request contains corresponding camera modification parameters;
according to the camera parameter modification request, calling a Linux platform service to send the camera modification parameter to an Android terminal through a corresponding HTTP interface;
according to the camera modification parameters, calling an Android terminal to perform corresponding operation and update on the industrial camera; at the same time, the method comprises the steps of,
the camera modification parameters are written into the AI application profile.
Optionally, the industrial camera video stream presentation device 50 further comprises an algorithmic reasoning unit 508, the algorithmic reasoning unit 508 being configured to:
according to the AI application configuration file, calling an Android terminal to start AI application to perform corresponding AI processing, and generating an reasoning result;
and calling a Linux platform service to analyze the reasoning result, and pushing the original image and the image corresponding to the reasoning result to the Web page for display in real time through the connection of the WebRTC.
Optionally, the algorithm inference unit 508 is further configured to:
and calling the Android terminal to start the AI application to collect and infer the image according to the AI application configuration file, and generating a corresponding reasoning result.
The industrial camera video stream display device 50 may be used to perform the method performed by the industrial camera video stream display device in the embodiments shown in fig. 3-4, and thus, the description of the functions that can be implemented by the functional modules of the industrial camera video stream display device 50 and the like may be referred to in the embodiments shown in fig. 3-4, and is not repeated.
In some possible implementations, aspects of the methods provided herein may also be implemented in the form of a program product comprising program code for causing a computer device to carry out the steps of the methods described herein above according to the various exemplary embodiments of the application, when the program product is run on the computer device, e.g. the computer device may carry out the methods as carried out by the industrial camera video stream display apparatus in the examples shown in fig. 3-4.
Those of ordinary skill in the art will appreciate that: all or part of the steps for implementing the above method embodiments may be implemented by hardware associated with program instructions, where the foregoing program may be stored in a computer readable storage medium, and when executed, the program performs steps including the above method embodiments; and the aforementioned storage medium includes: a mobile storage device, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk or an optical disk, or the like, which can store program codes. Alternatively, the above-described integrated units of the present invention may be stored in a computer-readable storage medium if implemented in the form of software functional modules and sold or used as separate products. Based on such understanding, the technical solutions of the embodiments of the present invention may be embodied in essence or a part contributing to the prior art in the form of a software product stored in a storage medium, including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in the embodiments of the present invention. And the aforementioned storage medium includes: a removable storage device, ROM, RAM, magnetic or optical disk, or other medium capable of storing program code.
While preferred embodiments of the present application have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiments and all such alterations and modifications as fall within the scope of the application.
It will be apparent to those skilled in the art that various modifications and variations can be made in the present application without departing from the spirit or scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims and the equivalents thereof, the present application is intended to cover such modifications and variations.
Claims (8)
1. An industrial camera video stream display method, which is applied to a hardware platform containing an industrial camera and an Android/Linux fusion operating system, wherein an AI application is integrated in the hardware platform, and the AI application is set in a Linux environment, the method comprises:
sending a camera starting request generated by a Web page to an Android terminal through Linux platform service; the Linux platform service adopts a Linux system, and the Android terminal adopts an Android system;
according to the camera starting request, calling the Android end to start the industrial camera;
calling the Linux platform service to connect with the RTSP service which is started by the Android terminal;
transmitting a camera video stream generated by the industrial camera to the Linux platform service through the RTSP service;
calling the Linux platform service to push the camera video stream to the Web page for display in real time through webRTC connection;
the method further comprises the steps of:
generating a corresponding parameter modification request according to the parameter modification operation of the user on the Web page aiming at the industrial camera; wherein the parameter modification request contains corresponding camera modification parameters;
according to the camera parameter modification request, calling the Linux platform service to send the camera modification parameters to the Android terminal through a corresponding HTTP interface;
according to the camera modification parameters, calling the Android terminal to perform corresponding operation and update on the industrial camera; at the same time, the method comprises the steps of,
writing the camera modification parameters into an AI application configuration file;
according to the AI application configuration file, calling an Android terminal to start AI application to perform corresponding AI processing, and generating an reasoning result;
and calling the Linux platform service to analyze the reasoning result, and pushing the original image and the image corresponding to the reasoning result to the Web page for display in real time through the connection of WebRTC.
2. The method of claim 1, wherein the step of sending the camera opening request generated by the Web page to the Android terminal through the Linux platform service includes:
generating a corresponding camera opening request according to camera opening operation of a user on a Web page aiming at the industrial camera;
invoking the Linux platform service to receive the camera starting request;
and calling the Linux platform service to send the camera starting request to the Android terminal.
3. The method of claim 1, wherein the step of calling the Linux platform service to connect the Android terminal to start the RTSP service comprises:
calling the Android terminal to start the RTSP service;
calling the Linux platform service to receive an RTSP service connection request sent by the Android terminal; wherein the RTSP service connection request includes an RTSP address;
and calling the Linux platform service to connect the RTSP service according to the RTSP service connection request.
4. The method of claim 1, wherein before invoking the Linux platform service to push the camera video stream to the Web page for presentation in real time via WebRTC connection, the method further comprises:
calling the Linux platform service to receive a point-to-point connection request sent by a Web page;
and establishing a WebRTC connection between the Linux platform service and the Web page according to the point-to-point request.
5. The method of claim 1, wherein the step of calling the Android terminal to start the AI application to perform corresponding AI processing according to the AI application configuration file and generating the reasoning result comprises the following steps:
and calling the Android terminal to start the AI application to perform image acquisition and reasoning according to the AI application configuration file, and generating a corresponding reasoning result.
6. An industrial camera video stream display device, characterized in that it is applied to a hardware platform containing an industrial camera and an Android/Linux converged operating system, in which an AI application is integrated, and the AI application is set in a Linux environment, the device comprises:
the request sending unit is used for sending a camera starting request generated by the Web page to the Android terminal through the Linux platform service; the Linux platform service adopts a Linux system, and the Android terminal adopts an Android system;
the camera opening unit is used for calling the Android end to open the industrial camera according to the camera opening request;
the RTSP service starting unit is used for calling the Linux platform service to connect the Android terminal to start the RTSP service;
the video stream sending unit is used for sending the camera video stream generated by the industrial camera to the Linux platform service through the RTSP service;
the video stream display unit is used for calling the Linux platform service to push the camera video stream to the Web page for display in real time through webRTC connection;
the camera debugging unit is used for generating a corresponding parameter modification request according to the parameter modification operation of the user on the Web page aiming at the industrial camera; wherein the parameter modification request contains corresponding camera modification parameters; according to the camera parameter modification request, calling a Linux platform service to send the camera modification parameter to an Android terminal through a corresponding HTTP interface; according to the camera modification parameters, calling an Android terminal to perform corresponding operation and update on the industrial camera; simultaneously, writing camera modification parameters into an AI application configuration file;
the algorithm reasoning unit is used for calling the Android terminal to start the AI application to perform corresponding AI processing according to the AI application configuration file and generating a reasoning result; and calling a Linux platform service to analyze the reasoning result, and pushing the original image and the image corresponding to the reasoning result to the Web page for display in real time through the connection of the WebRTC.
7. An electronic device, the device comprising:
a memory for storing program instructions;
a processor for invoking program instructions stored in said memory and for performing the method of any of claims 1-5 in accordance with the obtained program instructions.
8. A storage medium having stored thereon computer executable instructions for causing a computer to perform the method of any one of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311451924.9A CN117176705B (en) | 2023-11-03 | 2023-11-03 | Industrial camera video stream display method, device, equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311451924.9A CN117176705B (en) | 2023-11-03 | 2023-11-03 | Industrial camera video stream display method, device, equipment and medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117176705A CN117176705A (en) | 2023-12-05 |
| CN117176705B true CN117176705B (en) | 2024-01-26 |
Family
ID=88939902
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311451924.9A Active CN117176705B (en) | 2023-11-03 | 2023-11-03 | Industrial camera video stream display method, device, equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117176705B (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113448648A (en) * | 2021-06-29 | 2021-09-28 | 技德技术研究所(武汉)有限公司 | Linux-compatible Android camera access method and device |
| CN113709185A (en) * | 2021-10-22 | 2021-11-26 | 浙江云针信息科技有限公司 | Method and system for realizing Android cloud streaming |
| CN113824888A (en) * | 2021-11-23 | 2021-12-21 | 北京鲸鲮信息系统技术有限公司 | Linux-compatible Android camera control method, system, device and medium |
| CN113852854A (en) * | 2021-08-11 | 2021-12-28 | 武汉光庭信息技术股份有限公司 | Video plug-flow and pull-flow realization method and system based on Android system |
| WO2022222537A1 (en) * | 2021-04-24 | 2022-10-27 | 华为云计算技术有限公司 | Ai application deployment method, and related platform, cluster, medium and program product |
| CN115278304A (en) * | 2021-04-14 | 2022-11-01 | 腾讯云计算(长沙)有限责任公司 | Cloud audio and video processing method and system, electronic equipment and storage medium |
| CN115766695A (en) * | 2022-12-23 | 2023-03-07 | 湖北中烟工业有限责任公司 | Websocket-based remote browsing method and device for real-time image of industrial camera |
| CN115865884A (en) * | 2022-11-23 | 2023-03-28 | 上海悠络客电子科技股份有限公司 | Network camera data access device and method, network camera and medium |
| CN116071638A (en) * | 2023-01-12 | 2023-05-05 | 浪潮软件集团有限公司 | Method and device for dynamically analyzing streaming picture data in real time |
| WO2023124002A1 (en) * | 2021-12-29 | 2023-07-06 | 北京字节跳动网络技术有限公司 | Linux-based audio control method and apparatus, device, and storage medium |
-
2023
- 2023-11-03 CN CN202311451924.9A patent/CN117176705B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115278304A (en) * | 2021-04-14 | 2022-11-01 | 腾讯云计算(长沙)有限责任公司 | Cloud audio and video processing method and system, electronic equipment and storage medium |
| WO2022222537A1 (en) * | 2021-04-24 | 2022-10-27 | 华为云计算技术有限公司 | Ai application deployment method, and related platform, cluster, medium and program product |
| CN113448648A (en) * | 2021-06-29 | 2021-09-28 | 技德技术研究所(武汉)有限公司 | Linux-compatible Android camera access method and device |
| CN113852854A (en) * | 2021-08-11 | 2021-12-28 | 武汉光庭信息技术股份有限公司 | Video plug-flow and pull-flow realization method and system based on Android system |
| CN113709185A (en) * | 2021-10-22 | 2021-11-26 | 浙江云针信息科技有限公司 | Method and system for realizing Android cloud streaming |
| CN113824888A (en) * | 2021-11-23 | 2021-12-21 | 北京鲸鲮信息系统技术有限公司 | Linux-compatible Android camera control method, system, device and medium |
| WO2023093715A1 (en) * | 2021-11-23 | 2023-06-01 | 北京字节跳动网络技术有限公司 | Camera control method, system and apparatus for achieving compatibility between linux and android, and medium |
| WO2023124002A1 (en) * | 2021-12-29 | 2023-07-06 | 北京字节跳动网络技术有限公司 | Linux-based audio control method and apparatus, device, and storage medium |
| CN115865884A (en) * | 2022-11-23 | 2023-03-28 | 上海悠络客电子科技股份有限公司 | Network camera data access device and method, network camera and medium |
| CN115766695A (en) * | 2022-12-23 | 2023-03-07 | 湖北中烟工业有限责任公司 | Websocket-based remote browsing method and device for real-time image of industrial camera |
| CN116071638A (en) * | 2023-01-12 | 2023-05-05 | 浪潮软件集团有限公司 | Method and device for dynamically analyzing streaming picture data in real time |
Non-Patent Citations (2)
| Title |
|---|
| 基于AirPlay2的音频流媒体系统设计与实现;李鹏飞;信息化研究;全文 * |
| 基于Android平台的高清监控视频系统的优化设计;蒋达央;信息科技;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117176705A (en) | 2023-12-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR102158557B1 (en) | Method and device for determining response time | |
| TWI629639B (en) | Method and device for realizing screen capture | |
| WO2019242222A1 (en) | Method and device for use in generating information | |
| CN106572139B (en) | Multi-terminal control method, terminal, server and system | |
| US11196962B2 (en) | Method and a device for a video call based on a virtual image | |
| CN111858318B (en) | Response time testing method, device, equipment and computer storage medium | |
| US20190051147A1 (en) | Remote control method, apparatus, terminal device, and computer readable storage medium | |
| EP3852383A2 (en) | Method and apparatus for commenting video, electronic device, storage medium and computer program product | |
| WO2016197590A1 (en) | Method and apparatus for providing screenshot service on terminal device and storage medium and device | |
| US11809882B2 (en) | Interface calling method and apparatus, and computer-readable storage medium | |
| EP2713604A1 (en) | Method for determining number of photos capable of being taken, and photographing device | |
| CN110837474A (en) | Script recording method, device, terminal and storage medium | |
| CN113893524B (en) | Cloud application processing system, method, device and equipment | |
| CN108765084B (en) | Synchronous processing method and device for virtual three-dimensional space | |
| CN111246272A (en) | Method and device for displaying video cover picture | |
| CN110865870A (en) | Application calling method and device based on hook technology | |
| US20180336122A1 (en) | Generating application flow entities | |
| CN117176705B (en) | Industrial camera video stream display method, device, equipment and medium | |
| CN112839065B (en) | Information processing method, information processing device, first equipment and storage medium | |
| CN106909481B (en) | Interface test method, interface test device and electronic equipment | |
| US20170109113A1 (en) | Remote Image Projection Method, Sever And Client Device | |
| CN116758201A (en) | Rendering processing method, device and system of three-dimensional scene and computer storage medium | |
| CN112449243B (en) | Video processing method, device, equipment and storage medium | |
| CN111596980B (en) | Information processing method and device | |
| CN114157626A (en) | Method, system, storage medium and computer equipment for transmitting photos by photographing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |