CN113329205A - Internet of things video data processing system, intelligent retail system, method and device - Google Patents
Internet of things video data processing system, intelligent retail system, method and device Download PDFInfo
- Publication number
- CN113329205A CN113329205A CN202110383522.4A CN202110383522A CN113329205A CN 113329205 A CN113329205 A CN 113329205A CN 202110383522 A CN202110383522 A CN 202110383522A CN 113329205 A CN113329205 A CN 113329205A
- Authority
- CN
- China
- Prior art keywords
- processing result
- video data
- camera
- intelligent terminal
- edge device
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
Landscapes
- Engineering & Computer Science (AREA)
- Signal Processing (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Closed-Circuit Television Systems (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The invention discloses an Internet of things video data processing system, an intelligent retail method and an intelligent retail device. The Internet of things video data processing system comprises edge equipment and an intelligent terminal; the edge device is used for acquiring video data shot in real time from at least one camera connected with the edge device, and an algorithm matched with the camera in the edge device is used for performing corresponding operation analysis on the video data to obtain a processing result and transmitting the processing result to the intelligent terminal; and the intelligent terminal is used for executing corresponding operation according to the processing result. The decoupling of AI operation and a cloud server can be realized, the video data processing speed is not influenced by the number of connected cameras and the network speed, and the offline scene can be dealt with.
Description
Technical Field
The invention relates to the field of Internet of things and intelligent retail, in particular to a video data processing system of the Internet of things, an intelligent retail system, a method and a device.
Background
In the existing intelligent retail, a cloud Artificial Intelligence (AI) algorithm application scheme is mainly used, and as shown in fig. 1, an AI algorithm service is generally set in a cloud, a camera is directly connected to the cloud, a data transmission service is started in the cloud to specially process transactions related to video streams of the camera, and the data transmission service acquires video stream data from each camera through a gateway and provides the video stream data for each AI algorithm service; the AI algorithm service runs at the cloud end, the execution result is output to the VMS data interface service, and the WEB back-end APP acquires relevant data through the VMS data interface service and then transmits the relevant data to the local PC or other intelligent terminals.
The scheme is applied to the intelligent retail scene and has the following disadvantages:
(1) all data need to be transmitted on a public network, certain potential safety hazards exist, and meanwhile, the response speed of intelligent retail is influenced by the number of connected cameras and the network speed;
(2) the coupling between the AI algorithm service and the cloud is high, AI operation cannot be processed in an off-line mode, and the probability of off-line situations in intelligent retail scenes is high, so that the requirements of some intelligent retail scenes are not met, and the risk of data loss exists.
Disclosure of Invention
In view of the above, the present invention has been made to provide an internet of things video data processing system, an intelligent retail system, and a method and apparatus that overcome or at least partially solve the above problems.
In a first aspect, an embodiment of the present invention provides an internet of things video data processing system, which includes an edge device and an intelligent terminal;
the edge device is used for acquiring video data shot in real time from at least one camera connected with the edge device, and algorithm application matched with the camera in the edge device performs corresponding operation analysis on the video data to obtain a processing result and transmits the processing result to the intelligent terminal;
and the intelligent terminal is used for executing corresponding operation according to the processing result.
In a second aspect, an embodiment of the present invention provides an internet of things video data processing method, including:
the method comprises the steps that the edge equipment of the Internet of things acquires video data shot in real time from at least one camera connected with the edge equipment;
and the algorithm application matched with the camera in the edge equipment performs corresponding operation analysis on the video data to obtain a processing result, and the processing result is transmitted to the intelligent terminal of the Internet of things, so that the intelligent terminal performs corresponding operation according to the processing result.
In a third aspect, an embodiment of the present invention provides a method for processing video data of an internet of things, including:
the method comprises the steps that an intelligent terminal of the Internet of things receives a processing result of video data sent by edge equipment, wherein the video data are obtained by the edge equipment from at least one camera connected with the edge equipment and shot in real time, and the processing result is obtained after algorithm application matched with the camera in the edge equipment performs corresponding operation analysis on the video data;
the intelligent terminal requests corresponding information from the cloud server according to the processing result, executes corresponding operation according to the requested corresponding information and the processing result, and sends the processing result and the executed corresponding operation to the cloud server.
In a fourth aspect, an embodiment of the present invention provides an internet of things video data processing apparatus, where the apparatus is disposed on an edge device of an internet of things, and includes:
the acquisition module is used for acquiring video data shot in real time from at least one camera connected with the acquisition module;
and the algorithm application module is matched with the camera and used for performing corresponding operation analysis on the video data to obtain a processing result, and transmitting the processing result to the intelligent terminal of the Internet of things, so that the intelligent terminal performs corresponding operation according to the processing result.
In a fifth aspect, an embodiment of the present invention provides an internet of things video data processing device, where the device is disposed in an internet of things intelligent terminal, and includes:
the edge device comprises a receiving module and a processing module, wherein the receiving module is used for receiving a processing result of video data sent by the edge device, the video data is obtained by the edge device from at least one camera connected with the edge device and shot in real time, and the processing result is obtained by the edge device after algorithm application matched with the camera performs corresponding operation analysis on the video data;
the operation module is used for requesting corresponding information from the cloud server according to the processing result received by the receiving module and executing corresponding operation according to the requested corresponding information and the processing result;
and the sending module is used for sending the processing result received by the receiving module and the corresponding operation executed by the operation module to a cloud server.
The technical scheme provided by the embodiment of the invention has the beneficial effects that at least:
(1) the video data processing system of the Internet of things comprises edge equipment and an intelligent terminal; the edge device is used for acquiring video data shot in real time from at least one camera connected with the edge device, and an algorithm matched with the camera in the edge device is used for performing corresponding operation analysis on the video data to obtain a processing result and transmitting the processing result to the intelligent terminal; and the intelligent terminal is used for executing corresponding operation according to the processing result. The AI algorithm is applied and deployed to the IoT (Internet of Things, IoT) edge equipment, AI operation is operated on the IoT edge equipment, compared with the AI operation operated on a cloud server, the decoupling of the cloud server and the camera is realized, so that the AI operation speed and the response speed of the intelligent terminal are improved, and the algorithm execution speed and the response speed of the intelligent terminal are not influenced by the number of connected cameras and the network speed; meanwhile, the video data can be processed in an off-network state, so that the applicable scene of the embodiment is expanded, and the risk of data loss does not exist.
(2) The video data and the processing result thereof are only transmitted in the Internet of things, and do not need to be transmitted in the public network, thereby improving the safety of the data.
The technical solution of the present invention is further described in detail by the accompanying drawings and embodiments.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention and not to limit the invention. In the drawings:
FIG. 1 is a schematic diagram of a prior art intelligent retail system;
fig. 2 is a schematic structural diagram of a video data processing system of the internet of things in the embodiment of the invention;
fig. 3 is a schematic structural diagram of another video data processing system of the internet of things according to an embodiment of the present invention;
FIG. 4 is a schematic structural diagram of an Internet of things intelligent retail system in an embodiment of the invention;
fig. 5 is a schematic structural diagram of a video data processing apparatus of the internet of things according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of another internet-of-things video data processing apparatus according to an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
In order to solve the problem that the AI algorithm service of the intelligent retail system is arranged on a cloud server in the prior art, so that the response speed of intelligent retail is greatly influenced by the number of connected cameras and the network speed and is not suitable for a network break scene, the embodiment of the invention provides an Internet of things video data processing system, an intelligent retail system, a method and a device, which can realize the decoupling of AI operation and the cloud server, so that the video data processing speed is not influenced by the number of connected cameras and the network speed and can cope with the network break scene.
The intelligent retail generally applies the internet and the internet of things technology, senses consumption habits, predicts consumption trends, guides production and manufacture, and provides diversified and personalized products and services for consumers. The Internet of Things (IoT) Edge device, also called IoT Edge, is an application of Edge computing in The Internet of Things industry, and The IoT Edge device serves as an Internet of Things Edge "cerebellum" and is an open platform that integrates network, computing, storage and application core capabilities on The Edge side near an object or a data source, so as to provide computing and intelligent services nearby, and meet The basic requirements of The industry in The aspects of real-time business, application intelligence, security and privacy protection. The applicable scene of the embodiment of the invention is that at least one camera is connected with edge equipment, and forms an intelligent retail Internet of things together with an intelligent terminal, and the intelligent retail Internet of things is a local area network; the edge device and the intelligent terminal in the Internet of things can be communicated with the cloud server through the wide area network, and the cloud server is in butt joint with at least one group of the Internet of things.
Examples
The embodiment of the invention provides an internet of things video data processing system, the structure of which is shown in fig. 2, and the system comprises edge equipment 21 and an intelligent terminal 22.
The edge device 21 is configured to obtain video data captured in real time from at least one camera 23 connected to the edge device 21, perform corresponding operation analysis on the video data by using an algorithm matched with the camera in the edge device 21 to obtain a processing result, and transmit the processing result to the intelligent terminal 22.
And the intelligent terminal 22 is used for executing the corresponding operation 22 according to the processing result.
Specifically, the connection between the camera and the edge device may be through a local area network, or may be through a data line, and the specific connection mode is not limited in this embodiment.
In one embodiment, the edge device of the internet of things allocates threads for the accessed cameras, and each thread is in butt joint with one connected camera; and requesting the video data shot in real time from the corresponding camera through the local area network through the thread.
The determination of the accessed camera may be that the edge device scans the connected camera when starting, and determines the scanned camera; or, after the edge device is started, detecting the newly accessed camera according to a set interval, and determining the newly accessed camera.
In an embodiment, the intelligent terminal 22 executes a corresponding operation according to the processing result, and is specifically configured to request corresponding information from the cloud server 24 according to the processing result, execute a corresponding operation according to the requested corresponding information and the processing result, and send the processing result and the executed corresponding operation to the cloud server 24.
In the prior art, algorithm application is deployed in a cloud server, so that a processing result is obtained after the cloud server executes corresponding operation analysis, the cloud server determines a corresponding operation to be executed according to the processing result and retrieved stored corresponding information, sends the corresponding operation to be executed to an intelligent terminal for execution, and stores the corresponding operation after receiving an execution result sent by the intelligent terminal; the method and the device for processing the cloud server deployment algorithm deployment are characterized in that the algorithm application is deployed on edge equipment, the edge equipment executes corresponding operation analysis to obtain a processing result, the processing result is sent to the intelligent terminal, corresponding information is requested to the cloud server according to the processing result, corresponding operation needing to be executed is determined according to the requested corresponding information and the processing result, the corresponding operation is executed, and the processing result and the executed corresponding operation are sent to the cloud server to be stored. The data flow of the entire process is substantially changed from the processing result of the video data.
In an embodiment, the edge device 21, after acquiring the video data shot in real time from at least one camera 23 connected thereto, is further configured to combine the received video data, the identification information of the algorithm application matched with the camera 23, and the camera address into a first data packet, publish the first data packet to all current algorithm applications 211 in the edge device 21, and receive the first data packet by the algorithm application 211 subscribing to the data carrying the identification information of the corresponding algorithm application.
Specifically, the identification information of the algorithm application (the algorithm application matched with the camera for shooting the video data) may be used as a theme, a first data packet is composed of the theme, the video data and the camera address, and the first data packet is distributed to the matched algorithm application through a publish-subscribe (pub-sub) network communication mode.
In one embodiment, the algorithm application 211 in the edge device 21 transmits the processing result to the intelligent terminal 22, and is specifically configured to form a second data packet by using the processing result and the camera address, issue the second data packet to the forwarding application 212 in the edge device subscribed to data carrying a corresponding camera address, the forwarding application 212 receives the second data packet and sends the second data packet to the intelligent terminal 22, and the forwarding application 212 and the intelligent terminal 22 subscribe to data carrying a corresponding camera address.
Namely, the processing result is sent to the forwarding application by the algorithm application and sent to the intelligent terminal by the forwarding application through the pub-sub network communication mode. Specifically, the algorithm application 211 may use the camera address as a theme, form a second data packet from the theme and the processing result, send the second data packet to the forwarding application through the pub-sub network communication mode, and send the second data packet to the intelligent terminal through the pub-sub network communication mode.
The camera-matched algorithm application in the edge device may be fixed in the edge device, i.e. the camera-matched algorithm application that is possibly connected to the edge device is predetermined, and these algorithm applications are all fixedly installed in the edge device.
Preferably, the edge device may be temporarily deployed in real time as needed, and the edge device acquires identification information of the accessed camera and sends the identification information to the intelligent terminal or the cloud server; and the edge equipment receives an algorithm program package which is sent by the cloud server and matched with the accessed camera, namely the algorithm program package matched with the identification information, and installs the algorithm program package to obtain algorithm application.
Specifically, the identification information is sent to the intelligent terminal, and the intelligent terminal may determine the identification information of the algorithm application matched with the camera according to the received identification information and the corresponding relationship between the set camera identification and the algorithm application identification, and send the identification information to the cloud server, so that the cloud server sends the matched algorithm program package to the edge device according to the identification information of the algorithm application; and the edge equipment receives an algorithm program package which is sent by the cloud server and matched with the accessed camera, and installs the algorithm program package to obtain algorithm application. Optionally, the edge device may also send the identification information to the cloud server, and the cloud server sends the received identification information to the intelligent terminal again, and the subsequent steps are the same as those described above; the edge device may also send the identification information to a cloud server, where the cloud server stores a correspondence between a camera identification and an algorithm application identification, which are obtained from the intelligent terminal in advance, and determines a matched algorithm application according to the correspondence and the received identification information, and then directly sends an algorithm program package of the matched algorithm application to the edge device.
The obtaining, by the edge device, the identification information of the accessed camera may specifically include performing at least one of the following:
(1) and scanning the connected cameras when the edge equipment is started, and acquiring the identification information of the scanned cameras.
For example, the connected cameras are scanned by the Onvif technique.
(2) And the edge equipment detects the newly accessed camera according to a set interval and acquires the identification information of the newly accessed camera.
When the edge device scans the camera when being started or detects a newly accessed camera according to a set interval in operation, the camera is registered and related attribute information of the camera, such as basic information of the camera, including resolution, identification, address and the like of the camera, is acquired.
In one embodiment, the intelligent terminal is further configured to determine identification information of the algorithm application to be upgraded according to the algorithm application version update information and a version of the algorithm application running in the current edge device, and send the identification information of the algorithm application to be upgraded to the cloud server, so that the cloud server sends a matched algorithm program upgrade package to the edge device according to the identification information; and the edge equipment is used for upgrading the corresponding algorithm application through the received algorithm program upgrading package.
Algorithm application on the edge equipment can be dynamically deployed and updated, so that one edge equipment can be deployed to different intelligent retail scenes according to requirements, the resource utilization rate is high, and the cost is saved.
The algorithm matched with the camera in the edge device is applied to perform corresponding operation analysis on the video data to obtain a processing result, for example, the processing result may be obtained by performing a corresponding algorithm on the video data when a customer selects a commodity by himself, and the processing result includes the determined type or type and number of the commodity selected by the customer.
The video data processing system of the internet of things provided by the embodiment of the invention comprises edge equipment and an intelligent terminal; the edge device is used for acquiring video data shot in real time from at least one camera connected with the edge device, and an algorithm matched with the camera in the edge device is used for performing corresponding operation analysis on the video data to obtain a processing result and transmitting the processing result to the intelligent terminal; and the intelligent terminal is used for executing corresponding operation according to the processing result. The AI algorithm is applied and deployed to the IoT (Internet of Things, IoT) edge equipment, AI operation is operated on the IoT edge equipment, compared with the AI operation operated on a cloud server, the decoupling of the cloud server and the camera is realized, so that the AI operation speed and the response speed of the intelligent terminal are improved, and the algorithm execution speed and the response speed of the intelligent terminal are not influenced by the number of connected cameras and the network speed; meanwhile, the video data can be processed in an off-network state, so that the applicable scene of the embodiment is expanded, and the risk of data loss does not exist.
The video data and the processing result thereof are only transmitted in the Internet of things, and do not need to be transmitted in the public network, thereby improving the safety of the data.
Referring to fig. 3, in a specific example of the video data processing system of the internet of things provided in the embodiment of the present invention, a gateway application is implemented on an IoT edge device, and the gateway application is connected to a cloud server, and the cloud server dynamically deploys and updates an algorithm applied to the IoT edge device through the gateway application; meanwhile, the gateway application schedules all algorithm applications and uploads IoT edge equipment information to a cloud server; the method comprises the following steps of realizing a camera management application, taking charge of camera access, video data acquisition and other camera related tasks, and providing a uniform data acquisition interface for algorithm application; and realizing a forwarding application, which is specially responsible for receiving the execution result data of the algorithm application and then distributing the execution result data to a local PC (local refers to the local of the camera and is in the same local area network with the camera and the edge device) outside the IoT edge device, wherein the forwarding application provides a uniform data output interface for the algorithm application.
After the edge device is started, the camera management application scans the connected cameras through a technology (such as Onvif), registers the cameras and acquires the relevant attributes of the cameras, such as the basic information of the cameras, the application of a matched algorithm and the like. Specifically, the acquisition of the algorithm application matched with the camera may be that identification information of the camera is sent to a cloud server through a gateway application, and the cloud server determines identification information of the matched algorithm application according to the identification information, a preset correspondence between the camera identification information and the algorithm application identification, and sends a matched algorithm program package to the edge device; or the cloud server sends the identification information of the camera to the local PC, the local PC determines the identification information of the matched algorithm application according to the identification information, the preset corresponding relation between the camera identification information and the algorithm application identification, sends the identification information of the matched algorithm application to the cloud server, and the cloud server sends the matched algorithm program package to the edge device. After the edge device is started, the camera management application detects whether a newly accessed camera exists or not according to a set interval, and if the newly accessed camera is detected, the steps are executed, and the matched algorithm application is installed. Real-time deployment and updating of the camera and matched algorithm application are achieved.
The video camera management application creates a thread for each video camera, draws a video stream according to the information of the video camera and publishes the video stream through the same PUB terminal, and the format of a published data packet is as follows: the theme + the video data + the camera address, the theme is specifically identification information of matched algorithm application, and the video data can be a frame of image data, so that the real-time performance of video data transmission and processing is ensured. The PUB end of the camera management application publishes the data packet to an algorithm application group, namely all algorithm applications which are currently running, the algorithm applications subscribe the data packet with the self identification as a theme, and the SUB end of the algorithm applications receives the data packet of the corresponding theme.
Each algorithm application takes the processing result of the video data and the address of a camera for shooting the video data as a theme to form a data packet, and the data packet is issued to the SUB end of the forwarding application through the PUB end.
And forwarding messages subscribed by the application to all algorithm applications, and then publishing the messages to the external of the IoT edge device through the same PUB interface, namely publishing the messages to the local PC.
The gateway application is responsible for connecting to the cloud server to communicate with the cloud server, receiving the algorithm program package or the algorithm program upgrade package sent by the cloud server, and is also responsible for scheduling all algorithm applications on the edge devices and sending information of the edge devices to the cloud server (for example, working state information of all algorithm applications on the edge devices, and the like).
In the whole scheme, the communication inside the edge device and between the edge device and the local PC is based on a PUB-SUB mode, the mode is based on UDP, bidirectional connection is not needed, blocking and waiting are not needed, decoupling is more thorough, and the communication speed is higher.
After the edge device deploys the algorithm application, a complete intelligent retail path is formed from the camera to the edge device to the local PC, a cloud server is not needed, and the intelligent retail can still work normally even if the intelligent retail path is offline. Meanwhile, the edge device supports rapid remote deployment, namely real-time and dynamic increase/decrease cameras and algorithm application, so that the applicability of the edge device is stronger.
Based on the inventive concept of the present invention, an embodiment of the present invention further provides an internet of things intelligent retail system, whose structure is shown in fig. 4, and includes an edge device 41, an intelligent terminal 42, and a cloud server 43;
the edge device 41 is configured to obtain video data captured in real time from at least one camera 44 connected to the edge device 41, perform corresponding operation and analysis on the video data by using an algorithm application 411 matched with the camera 44 in the edge device 41 to obtain a processing result, and transmit the processing result to the intelligent terminal 42;
and the intelligent terminal 42 is configured to request corresponding information from the cloud server 43 according to the processing result, execute a corresponding retail operation according to the requested corresponding information and the processing result, and send the processing result and the executed corresponding retail operation to the cloud server 43.
Based on the inventive concept of the present invention, an embodiment of the present invention further provides an internet of things video data processing method, including: the method comprises the steps that the edge equipment of the Internet of things acquires video data shot in real time from at least one camera connected with the edge equipment; and the algorithm application matched with the camera in the edge equipment performs corresponding operation analysis on the video data to obtain a processing result, and transmits the processing result to the intelligent terminal of the Internet of things, so that the intelligent terminal performs corresponding operation according to the processing result.
In one embodiment, the intelligent terminal executes the corresponding operation according to the processing result, which may specifically include requesting corresponding information from the cloud server according to the processing result, executing the corresponding operation according to the requested corresponding information and the processing result, and sending the processing result and the executed corresponding operation to the cloud server.
In one embodiment, after the edge device obtains video data shot in real time from at least one camera connected to the edge device, the method further includes forming a first data packet by the received video data, identification information of an algorithm application matched with the camera, and an address of the camera, publishing the first data packet to all current algorithm applications in the edge device, and receiving the first data packet by an algorithm application subscribing to data carrying the identification information of the corresponding algorithm application.
In one embodiment, the algorithm application in the edge device transmits the processing result to the intelligent terminal, and specifically includes that the processing result and the camera address form a second data packet, the second data packet is issued to a forwarding application in the edge device subscribing to data carrying a corresponding camera address, the forwarding application receives the second data packet, the second data packet is issued to the intelligent terminal, and the forwarding application and the intelligent terminal subscribe to data carrying a corresponding camera address.
In one embodiment, the method further comprises the steps that the edge device obtains identification information of the accessed camera, sends the identification information to the intelligent terminal or the cloud server, receives an algorithm program package which is sent by the cloud server and matched with the identification information, and installs the algorithm program package to obtain algorithm application.
In one embodiment, when the edge device sends the identification information to the intelligent terminal, the method further comprises the steps that the intelligent terminal determines the identification information of the algorithm application matched with the camera according to the identification information and the corresponding relation between the set camera identification and the algorithm application identification, and sends the identification information to the cloud server; and the cloud server sends the matched algorithm program package to the edge equipment according to the identification information of the algorithm application.
In one embodiment, the method further comprises the steps that the intelligent terminal determines identification information of the algorithm application needing to be upgraded according to the algorithm application version updating information and the version of the algorithm application running in the current edge device, and sends the identification information of the algorithm application needing to be upgraded to the cloud server, so that the cloud server sends a matched algorithm program upgrading package to the edge device according to the identification information; and the edge equipment is used for upgrading the corresponding algorithm application through the received algorithm program upgrading package.
In one embodiment, the edge device obtains the video data shot in real time from at least one camera connected with the edge device, and specifically includes that the edge device allocates a thread for the accessed camera, and requests the video data shot in real time from the corresponding camera through the local area network by the thread.
Based on the inventive concept of the present invention, an embodiment of the present invention further provides an internet of things video data processing method, including:
the method comprises the steps that an intelligent terminal of the Internet of things receives a processing result of video data sent by edge equipment, wherein the video data is obtained by the edge equipment from at least one camera connected with the edge equipment and shot in real time, and the processing result is obtained after algorithm application matched with the camera in the edge equipment performs corresponding operation analysis on the video data; the intelligent terminal requests corresponding information from the cloud server according to the processing result, executes corresponding operation according to the requested corresponding information and the processing result, and sends the processing result and the executed corresponding operation to the cloud server.
In one embodiment, the method further comprises the steps that the intelligent terminal of the internet of things receives identification information of a camera accessed to the edge device of the internet of things; and determining identification information of the algorithm application matched with the camera according to the identification information and the corresponding relation between the set camera identification and the algorithm application identification, and sending the identification information to the cloud server so that the cloud server sends an algorithm program package matched with the identification information of the algorithm application to the edge equipment.
Based on the inventive concept of the present invention, an embodiment of the present invention further provides an apparatus for processing video data of an internet of things, where the apparatus is disposed on an edge device of the internet of things, and a structure of the apparatus is shown in fig. 5, and the apparatus includes:
an acquisition module 51 for acquiring video data shot in real time from at least one camera connected thereto;
and the at least one algorithm application module 52 matched with the camera is used for performing corresponding operation analysis on the video data to obtain a processing result, and transmitting the processing result to the intelligent terminal of the internet of things, so that the intelligent terminal performs corresponding operation according to the processing result.
Based on the inventive concept of the present invention, an embodiment of the present invention further provides an internet of things video data processing apparatus, where the apparatus is disposed at an internet of things intelligent terminal, and a structure of the apparatus is shown in fig. 6, and the apparatus includes:
the receiving module 61 is configured to receive a processing result of video data sent by an edge device, where the video data is obtained by the edge device from at least one camera connected to the edge device and shot in real time, and the processing result is obtained by performing corresponding operation analysis on the video data by using an algorithm application matched with the camera in the edge device;
an operation module 62, configured to request corresponding information from the cloud server according to the processing result received by the receiving module 61, and execute corresponding operation according to the requested corresponding information and the processing result;
a sending module 63, configured to send the processing result received by the receiving module 61 and the corresponding operation executed by the operating module 62 to the cloud server.
With regard to the method and apparatus in the above embodiments, the specific manner in which each step or each module performs an operation has been described in detail in the embodiments related to the system, and will not be described in detail herein.
Unless specifically stated otherwise, terms such as processing, computing, calculating, determining, displaying, or the like, may refer to an action and/or process of one or more processing or computing systems or similar devices that manipulates and transforms data represented as physical (e.g., electronic) quantities within the processing system's registers and memories into other data similarly represented as physical quantities within the processing system's memories, registers or other such information storage, transmission or display devices. Information and signals may be represented using any of a variety of different technologies and techniques. For example, data, instructions, commands, information, signals, bits, symbols, and chips that may be referenced throughout the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof.
It should be understood that the specific order or hierarchy of steps in the processes disclosed is an example of exemplary approaches. Based upon design preferences, it is understood that the specific order or hierarchy of steps in the processes may be rearranged without departing from the scope of the present disclosure. The accompanying method claims present elements of the various steps in a sample order, and are not intended to be limited to the specific order or hierarchy presented.
In the foregoing detailed description, various features are grouped together in a single embodiment for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the claimed embodiments of the subject matter require more features than are expressly recited in each claim. Rather, as the following claims reflect, invention lies in less than all features of a single disclosed embodiment. Thus, the following claims are hereby expressly incorporated into the detailed description, with each claim standing on its own as a separate preferred embodiment of the invention.
Those of skill would further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. Of course, the storage medium may also be integral to the processor. The processor and the storage medium may reside in an ASIC. The ASIC may reside in a user terminal. Of course, the processor and the storage medium may reside as discrete components in a user terminal.
For a software implementation, the techniques described herein may be implemented with modules (e.g., procedures, functions, and so on) that perform the functions described herein. The software codes may be stored in memory units and executed by processors. The memory unit may be implemented within the processor or external to the processor, in which case it can be communicatively coupled to the processor via various means as is known in the art.
What has been described above includes examples of one or more embodiments. It is, of course, not possible to describe every conceivable combination of components or methodologies for purposes of describing the aforementioned embodiments, but one of ordinary skill in the art may recognize that many further combinations and permutations of various embodiments are possible. Accordingly, the embodiments described herein are intended to embrace all such alterations, modifications and variations that fall within the scope of the appended claims. Furthermore, to the extent that the term "includes" is used in either the detailed description or the claims, such term is intended to be inclusive in a manner similar to the term "comprising" as "comprising" is interpreted when employed as a transitional word in a claim. Furthermore, any use of the term "or" in the specification of the claims is intended to mean a "non-exclusive or".
Claims (10)
1. The video data processing system of the Internet of things is characterized by comprising edge equipment and an intelligent terminal;
the edge device is used for acquiring video data shot in real time from at least one camera connected with the edge device, and algorithm application matched with the camera in the edge device performs corresponding operation analysis on the video data to obtain a processing result and transmits the processing result to the intelligent terminal;
and the intelligent terminal is used for executing corresponding operation according to the processing result.
2. The system of claim 1, wherein the intelligent terminal, according to the processing result, executes a corresponding operation, specifically configured to:
and requesting corresponding information from a cloud server according to the processing result, executing corresponding operation according to the requested corresponding information and the processing result, and sending the processing result and the executed corresponding operation to the cloud server.
3. The system of claim 1, wherein the edge device, after acquiring the real-time captured video data from the at least one camera coupled thereto, is further configured to:
and forming a first data packet by the received video data, the identification information of the algorithm application matched with the camera and the address of the camera, publishing the first data packet to all current algorithm applications in the edge device, and receiving the first data packet by the algorithm application subscribing the data carrying the identification information of the corresponding algorithm application.
4. The system of claim 3, wherein the algorithm application in the edge device transmits the processing result to the intelligent terminal, and is specifically configured to:
and forming a second data packet by the processing result and the camera address, issuing the second data packet to a forwarding application in the edge device which subscribes to data carrying the corresponding camera address, receiving the second data packet by the forwarding application, issuing the second data packet to the intelligent terminal, and subscribing the data carrying the corresponding camera address by the forwarding application and the intelligent terminal.
5. The system of claim 1, wherein the edge device is further to:
the method comprises the steps of obtaining identification information of an accessed camera, sending the identification information to an intelligent terminal or a cloud server, receiving an algorithm program package which is sent by the cloud server and matched with the identification information, and installing the algorithm program package to obtain algorithm application.
6. The system of claim 5, wherein when the edge device sends the identification information to the intelligent terminal, the intelligent terminal is further configured to determine identification information of the algorithm application matched with the camera according to the identification information and a corresponding relationship between a set camera identification and an algorithm application identification, and send the identification information to a cloud server;
and the cloud server is further used for sending the matched algorithm program package to the edge device according to the identification information of the algorithm application.
7. An Internet of things video data processing method is characterized by comprising the following steps:
the method comprises the steps that the edge equipment of the Internet of things acquires video data shot in real time from at least one camera connected with the edge equipment;
and the algorithm application matched with the camera in the edge equipment performs corresponding operation analysis on the video data to obtain a processing result, and the processing result is transmitted to the intelligent terminal of the Internet of things, so that the intelligent terminal performs corresponding operation according to the processing result.
8. An Internet of things video data processing method is characterized by comprising the following steps:
the method comprises the steps that an intelligent terminal of the Internet of things receives a processing result of video data sent by edge equipment, wherein the video data are obtained by the edge equipment from at least one camera connected with the edge equipment and shot in real time, and the processing result is obtained after algorithm application matched with the camera in the edge equipment performs corresponding operation analysis on the video data;
the intelligent terminal requests corresponding information from the cloud server according to the processing result, executes corresponding operation according to the requested corresponding information and the processing result, and sends the processing result and the executed corresponding operation to the cloud server.
9. The utility model provides a thing networking video data processing apparatus, its characterized in that, the device sets up in thing networking edge device, includes:
the acquisition module is used for acquiring video data shot in real time from at least one camera connected with the acquisition module;
and the algorithm application module is matched with the camera and used for performing corresponding operation analysis on the video data to obtain a processing result, and transmitting the processing result to the intelligent terminal of the Internet of things, so that the intelligent terminal performs corresponding operation according to the processing result.
10. The utility model provides a thing networking video data processing apparatus, its characterized in that, the device sets up in thing networking intelligent terminal, includes:
the edge device comprises a receiving module and a processing module, wherein the receiving module is used for receiving a processing result of video data sent by the edge device, the video data is obtained by the edge device from at least one camera connected with the edge device and shot in real time, and the processing result is obtained by the edge device after algorithm application matched with the camera performs corresponding operation analysis on the video data;
the operation module is used for requesting corresponding information from the cloud server according to the processing result received by the receiving module and executing corresponding operation according to the requested corresponding information and the processing result;
and the sending module is used for sending the processing result received by the receiving module and the corresponding operation executed by the operation module to a cloud server.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110383522.4A CN113329205B (en) | 2021-04-09 | 2021-04-09 | Internet of things video data processing system, method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110383522.4A CN113329205B (en) | 2021-04-09 | 2021-04-09 | Internet of things video data processing system, method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113329205A true CN113329205A (en) | 2021-08-31 |
| CN113329205B CN113329205B (en) | 2023-05-05 |
Family
ID=77414667
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110383522.4A Active CN113329205B (en) | 2021-04-09 | 2021-04-09 | Internet of things video data processing system, method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113329205B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114137634A (en) * | 2021-12-07 | 2022-03-04 | 中兴飞流信息科技有限公司 | Platform end intrusion detection method and system based on radar video fusion |
| CN114844873A (en) * | 2022-04-11 | 2022-08-02 | 神马人工智能科技(深圳)有限公司 | Real-time processing system for audio-visual stream of Internet of things equipment based on artificial intelligence |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105531715A (en) * | 2013-06-26 | 2016-04-27 | 亚马逊科技公司 | Detecting item interaction and movement |
| US20200007460A1 (en) * | 2018-06-29 | 2020-01-02 | Intel Corporation | Scalable edge computing |
| CN111260525A (en) * | 2020-01-16 | 2020-06-09 | 深圳市广道高新技术股份有限公司 | Community security situation perception and early warning method, system and storage medium |
| CN111479048A (en) * | 2020-04-22 | 2020-07-31 | 安徽大学 | Intelligent video image processing equipment based on edge calculation |
| CN111698470A (en) * | 2020-06-03 | 2020-09-22 | 河南省民盛安防服务有限公司 | Security video monitoring system based on cloud edge cooperative computing and implementation method thereof |
-
2021
- 2021-04-09 CN CN202110383522.4A patent/CN113329205B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105531715A (en) * | 2013-06-26 | 2016-04-27 | 亚马逊科技公司 | Detecting item interaction and movement |
| US20200007460A1 (en) * | 2018-06-29 | 2020-01-02 | Intel Corporation | Scalable edge computing |
| CN111260525A (en) * | 2020-01-16 | 2020-06-09 | 深圳市广道高新技术股份有限公司 | Community security situation perception and early warning method, system and storage medium |
| CN111479048A (en) * | 2020-04-22 | 2020-07-31 | 安徽大学 | Intelligent video image processing equipment based on edge calculation |
| CN111698470A (en) * | 2020-06-03 | 2020-09-22 | 河南省民盛安防服务有限公司 | Security video monitoring system based on cloud edge cooperative computing and implementation method thereof |
Non-Patent Citations (2)
| Title |
|---|
| 刘伟 等: "移动边缘计算中资源受限的串行任务卸载策略" * |
| 程渤 等: "《物联网服务与应用》", 31 January 2018 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114137634A (en) * | 2021-12-07 | 2022-03-04 | 中兴飞流信息科技有限公司 | Platform end intrusion detection method and system based on radar video fusion |
| CN114844873A (en) * | 2022-04-11 | 2022-08-02 | 神马人工智能科技(深圳)有限公司 | Real-time processing system for audio-visual stream of Internet of things equipment based on artificial intelligence |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113329205B (en) | 2023-05-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113329205B (en) | Internet of things video data processing system, method and device | |
| CN101616132B (en) | Method for acquiring data, device and system thereof | |
| US20020032754A1 (en) | Method and apparatus for profiling in a distributed application environment | |
| US20070165615A1 (en) | Apparatus and method for notifying communication network event in application server capable of supporting open API based on Web services | |
| CN111984408B (en) | Data cooperative processing method based on big data and edge computing and edge cloud platform | |
| CN113127168A (en) | Service distribution method, system, device, server and medium | |
| CN104243611A (en) | Distribution thought based message service middleware system | |
| CN113709810B (en) | Method, equipment and medium for configuring network service quality | |
| CN113220715A (en) | Data processing method, system, computer and readable storage medium | |
| CN112379963B (en) | Remote application window control method and device and computer equipment | |
| CN112910744A (en) | Intelligent device control method and device, storage medium and electronic device | |
| CN111917838A (en) | Processing method and device based on micro service, storage medium and electronic device | |
| CN101707622A (en) | Method for realizing remote data monitoring | |
| CN108512889B (en) | Application response pushing method based on HTTP and proxy server | |
| CN110730197A (en) | Service discovery method and system | |
| CN113835789A (en) | Rendering method, rendering device, electronic equipment and computer storage medium | |
| US10643039B2 (en) | Location based situation awareness system and method thereof | |
| CN111092952A (en) | Data interaction method and related product | |
| CN113454969B (en) | Information interaction method, device, equipment, system and storage medium | |
| CN114785861A (en) | Service request forwarding system, method, computer equipment and storage medium | |
| CN114501374A (en) | Dynamic service deployment method, system, device and storage medium for Internet of vehicles | |
| CN113542424A (en) | Data processing method, device, equipment and computer program product | |
| CN114070889A (en) | Configuration method, traffic forwarding method, device, storage medium, and program product | |
| CN106874004A (en) | The method and client and server of lowest version software compatibility highest version file | |
| CN114077409A (en) | Screen projection method and device, electronic equipment and computer readable medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |