CN112990174A - Visual tracking platform and method for multi-application scene - Google Patents
Visual tracking platform and method for multi-application scene Download PDFInfo
- Publication number
- CN112990174A CN112990174A CN202110171894.0A CN202110171894A CN112990174A CN 112990174 A CN112990174 A CN 112990174A CN 202110171894 A CN202110171894 A CN 202110171894A CN 112990174 A CN112990174 A CN 112990174A
- Authority
- CN
- China
- Prior art keywords
- camera
- visual tracking
- module
- application scene
- hardware
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000000007 visual effect Effects 0.000 title claims abstract description 26
- 238000000034 method Methods 0.000 title claims abstract description 16
- 238000004364 calculation method Methods 0.000 claims abstract description 14
- 230000010354 integration Effects 0.000 claims abstract description 7
- 102100036790 Tubulin beta-3 chain Human genes 0.000 claims description 3
- 102100036788 Tubulin beta-4A chain Human genes 0.000 claims description 3
- 238000012545 processing Methods 0.000 abstract description 24
- 238000001514 detection method Methods 0.000 abstract description 8
- 238000013461 design Methods 0.000 abstract description 6
- 238000005516 engineering process Methods 0.000 description 7
- 238000011161 development Methods 0.000 description 5
- 238000004458 analytical method Methods 0.000 description 3
- 241000282412 Homo Species 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 238000009825 accumulation Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 210000004556 brain Anatomy 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 210000000056 organ Anatomy 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000012958 reprocessing Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/10—Image acquisition
- G06V10/12—Details of acquisition arrangements; Constructional details thereof
- G06V10/14—Optical characteristics of the device performing the acquisition or on the illumination arrangements
- G06V10/147—Details of sensors, e.g. sensor lenses
Abstract
The invention discloses a visual tracking platform and a visual tracking method for multiple application scenes, which comprise the following steps: step S1: setting parameters according to an actual application scene; step S2: setting camera parameters; step S3: a camera acquires information; step S4: optimizing software and hardware; step S5: performing resource integration; step S6: adjusting the camera; in step S2, the set camera parameters include the number of frames and the resolution. The invention can switch and debug the application of the complex scene, can carry out different detections on the same system platform, and reduces the hardware cost and difficulty; the user only needs to carry out debugging design according to the current required scene, so that the working efficiency is improved; the double-flow asynchronous algorithm of the invention makes continuous calculation processing independent, starts a new processing flow for each processing, can fully utilize the parallel capability of calculation hardware, and reduces the mutual restriction of data processing before and after time domain; maximum multiplexing of hardware modules is achieved.
Description
Technical Field
The present invention relates to a visual tracking system, and more particularly, to a visual tracking platform and method for multiple application scenes.
Background
Vision is one of the important ways in which humans perceive the world, with 80% of the external information acquired by humans coming from the visual system. The computational vision is to use an imaging system to replace human visual organs and use a computational grade to replace human brain to complete the processing and understanding of the input image on the basis of knowing the human vision. Meanwhile, with the development of information technology and intelligent science, computer vision is one of popular subjects in the field of artificial intelligence and one of important technologies of the perception layer of the internet of things.
The visual tracking technology, one of the popular subjects in the field of computer vision, is to perform moving target detection, feature extraction, classification and recognition, tracking filtering and behavior recognition on a continuous image sequence to obtain accurate moving information parameters (such as position, speed and the like) of a target, perform corresponding processing and analysis on the accurate moving information parameters, and realize behavior understanding of the target.
The visual tracking refers to detecting, extracting, identifying and tracking a moving target in an image sequence to obtain motion parameters of the moving target, such as position, speed, acceleration, motion track and the like, so that the next step of processing and analysis is performed to realize behavior understanding of the moving target to complete a higher-level detection task.
The existing image processing technology can basically have mature technologies in the aspects of object identification, face detection, face identification and object tracking; however, in the current artificial intelligence system, only visual identification and tracking can be performed for a single situation, and it is not easy to switch and debug the application of a complex scene, for example, a given scene is used for face identification of a user, but an actual application scene needs to detect a person who does not wear a mask, different system platforms need to be used for different detections, so that hardware cost and development difficulty are increased, and a user needs to perform debugging design for different scenes, thereby reducing working efficiency.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provides a visual tracking platform and a visual tracking method for a multi-application scene.
In order to solve the technical problems, the invention provides the following technical scheme:
the invention discloses a visual tracking method of a multi-application scene, which comprises the following steps:
step S1: setting parameters according to an actual application scene;
step S2: setting camera parameters;
step S3: a camera acquires information;
step S4: optimizing software and hardware;
step S5: performing resource integration;
step S6: the camera is adjusted.
As a preferred embodiment of the present invention, in step S2, the set camera parameters include the number of frames and the resolution.
As a preferred embodiment of the present invention, in step S5,
the resource integration content includes OpenCV and YoLo algorithms.
As a preferred embodiment of the present invention, in step S6,
the adjustments made to the camera include adjustments to a camera carrier.
As a preferred embodiment of the present invention, in the steps S1 to S6, the camera includes an RGB-D camera.
As a preferred technical solution of the present invention, the step S6 further includes a dual-stream asynchronous algorithm, including the following steps:
step SS 1: simultaneously reading data and calculating the data;
step SS 2: carrying out asynchronous calculation;
step SS 3: selecting a message node;
step SS 4: carrying out priority matching;
step SS 5: a continuous treatment is performed.
The invention also provides a visual tracking platform of a multi-application scene, which comprises the camera, the OpenCV module, the YoLo algorithm module, the gazebo module, the rviz module, the main control system module, the mobile chassis module, the laser radar module and the mechanical arm.
The invention has the following beneficial effects: the invention can switch and debug the application of the complex scene, can carry out different detections on the same system platform, and reduces the hardware cost and difficulty; the user only needs to carry out debugging design according to the current required scene, so that the working efficiency is improved; the double-flow asynchronous algorithm of the invention makes continuous calculation processing independent in time dimension, starts a new processing flow for each processing, can fully utilize the parallel capability of calculation hardware, and reduces the mutual restriction of data processing before and after time domain; the maximum multiplexing of the hardware modules is realized; the difficulty in development of the existing visual processing can be solved; the resources are integrated through resource packages such as Opencv and Yolo, a main control system is carried by a system platform, physical systems such as a mobile chassis, a laser radar and a mechanical arm are written in virtual control of gazebo and rviz, and a user can decide whether to perform and apply a visual tracking technology to a wider combination.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention and not to limit the invention. In the drawings:
FIG. 1 is one of the flow diagrams of the present invention;
FIG. 2 is a second schematic flow chart of the present invention.
Detailed Description
The preferred embodiments of the present invention will be described in conjunction with the accompanying drawings, and it will be understood that they are described herein for the purpose of illustration and explanation and not limitation.
Examples
As shown in fig. 1-2, the present invention provides a visual tracking method for multiple application scenes, comprising the following steps:
step S1: setting parameters according to an actual application scene; firstly, the corresponding design scheme is known through product data, and then design analysis is carried out according to the actual application scene. Calling different libraries for problem solving under different conditions; step S2: setting camera parameters; in step S2, the set camera parameters include the number of frames and resolution; step S3: a camera acquires information; step S4: optimizing software and hardware; step S5: performing resource integration; step S6: the camera is adjusted. In step S5, the resource integration content includes OpenCV and YoLo algorithms.
In step S6, the adjustment performed on the camera includes adjustment of a camera carrying device, in this embodiment, the carrying device is a steering engine, and the adjustment range is 180 degrees in the horizontal direction and 90 degrees in the pitch direction.
In the steps S1 through S6, the camera includes an RGB-D camera.
In step S6, the method further includes a dual-stream asynchronous algorithm, including the following steps: step SS 1: simultaneously reading data and calculating the data; step SS 2: carrying out asynchronous calculation; step SS 3: selecting a message node; step SS 4: carrying out priority matching; step SS 5: carrying out continuous treatment;
and (3) a double-flow asynchronous real-time reasoning calculation framework. Under the condition of limited computational performance, in order to solve the problems of synchronous blocking, multi-model data redundancy and high model switching overhead possibly existing in the inference neural network operation, the multi-core parallel capability of the computation hardware is fully excavated, a double-flow asynchronous real-time inference computation architecture is designed, and a polymorphic model is supported to realize real-time inference. The designed double-flow structure separates data reading from a data calculation thread, avoids the influence of data reading delay and invalidation on subsequent processing, and also isolates data accumulation and data loss caused by data processing delay; the asynchronous calculation framework realizes the independence of continuous calculation processing in the time dimension, starts a new processing flow for each processing, can fully utilize the parallel capability of calculation hardware, and reduces the mutual restriction of data processing before and after time domain.
A multi-application scene vision tracking platform comprising a camera as claimed in the preceding claims and an OpenCV module, a YoLo algorithm module, a gazebo module, a rviz module, a main control system module, a mobile chassis module, a lidar module, and a robotic arm.
Further, the method is simple and convenient to operate.
Specifically, the existing data parameters are first acquired by the RGB-D camera. Adjusting the range of image acquisition through a steering engine; the development direction is then determined from the existing scene. And selecting and combining the resources of object recognition, face detection, face recognition, object tracking and the like which are carried by the system platform. And optimizing the software and hardware of the whole system architecture. And then, acquiring the information of the image through an ROS system, reprocessing the acquired information sample, and carrying out secondary development and calling on the existing resources according to actual requirements.
The invention can switch and debug the application of the complex scene, can carry out different detections on the same system platform, and reduces the hardware cost and difficulty; the user only needs to carry out debugging design according to the current required scene, so that the working efficiency is improved; the double-flow asynchronous algorithm of the invention makes continuous calculation processing independent in time dimension, starts a new processing flow for each processing, can fully utilize the parallel capability of calculation hardware, and reduces the mutual restriction of data processing before and after time domain; maximum multiplexing of hardware modules is achieved.
The difficulty in developing the existing visual processing can be solved. The resources are integrated through resource packages such as Opencv and Yolo, a main control system is carried by a system platform, physical systems such as a mobile chassis, a laser radar and a mechanical arm are written in virtual control of gazebo and rviz, and a user can decide whether to perform and apply a visual tracking technology to a wider combination.
Finally, it should be noted that: although the present invention has been described in detail with reference to the foregoing embodiments, it will be apparent to those skilled in the art that changes may be made in the embodiments and/or equivalents thereof without departing from the spirit and scope of the invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (7)
1. A visual tracking method for a multi-application scene is characterized by comprising the following steps:
step S1: setting parameters according to an actual application scene;
step S2: setting camera parameters;
step S3: a camera acquires information;
step S4: optimizing software and hardware;
step S5: performing resource integration;
step S6: the camera is adjusted.
2. The method for visually tracking a multi-application scene as claimed in claim 1, wherein the camera parameters set in step S2 include frame number and resolution.
3. A visual tracking method of multi-application scene as claimed in claim 1, wherein in said step S5,
the resource integration content includes OpenCV and YoLo algorithms.
4. A visual tracking method of multi-application scene as claimed in claim 1, wherein in said step S6,
the adjustments made to the camera include adjustments to a camera carrier.
5. A method for visual tracking of a multi-application scene as recited in claim 1, wherein said camera comprises an RGB-D camera in said steps S1 through S6.
6. The visual tracking method for multiple application scenes of claim 1, characterized in that in the step S6, the method further comprises a dual-stream asynchronous algorithm, comprising the following steps:
step SS 1: simultaneously reading data and calculating the data;
step SS 2: carrying out asynchronous calculation;
step SS 3: selecting a message node;
step SS 4: carrying out priority matching;
step SS 5: a continuous treatment is performed.
7. A visual tracking platform of a multi-application scenario according to claim 1, characterized by comprising a camera as described in claims 1-6 and OpenCV module, YoLo algorithm module, gazebo module, rviz module, main control system module, mobile chassis module, lidar module and robotic arm.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110171894.0A CN112990174A (en) | 2021-02-04 | 2021-02-04 | Visual tracking platform and method for multi-application scene |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110171894.0A CN112990174A (en) | 2021-02-04 | 2021-02-04 | Visual tracking platform and method for multi-application scene |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112990174A true CN112990174A (en) | 2021-06-18 |
Family
ID=76347446
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110171894.0A Pending CN112990174A (en) | 2021-02-04 | 2021-02-04 | Visual tracking platform and method for multi-application scene |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112990174A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114911537A (en) * | 2022-05-10 | 2022-08-16 | 声呐天空资讯顾问有限公司 | Parameter configuration method and system, and storage medium |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2008278515A (en) * | 2008-06-02 | 2008-11-13 | Hitachi Ltd | Image processing camera system and image processing camera control method |
| US20130050517A1 (en) * | 2011-08-26 | 2013-02-28 | International Business Machines Corporation | Visual content-aware automatic camera adjustment |
| US20170161579A1 (en) * | 2014-09-30 | 2017-06-08 | Qualcomm Incorporated | Single-processor computer vision hardware control and application execution |
| CN108274466A (en) * | 2018-01-17 | 2018-07-13 | 广州威沃电子有限公司 | Intelligent vision flexible wires system based on industrial robot |
| CN108733368A (en) * | 2017-05-16 | 2018-11-02 | 研祥智能科技股份有限公司 | Machine vision general software development system |
| CN109922247A (en) * | 2019-04-17 | 2019-06-21 | 浙江禾川科技股份有限公司 | A kind of smart camera and a kind of image processing method |
| CN111698418A (en) * | 2020-04-17 | 2020-09-22 | 广州市讯思视控科技有限公司 | Industrial intelligent camera system based on deep learning configuration cloud platform |
| CN211906310U (en) * | 2018-10-19 | 2020-11-10 | 成都信息工程大学 | Zynq-based machine vision detection system |
-
2021
- 2021-02-04 CN CN202110171894.0A patent/CN112990174A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2008278515A (en) * | 2008-06-02 | 2008-11-13 | Hitachi Ltd | Image processing camera system and image processing camera control method |
| US20130050517A1 (en) * | 2011-08-26 | 2013-02-28 | International Business Machines Corporation | Visual content-aware automatic camera adjustment |
| US20170161579A1 (en) * | 2014-09-30 | 2017-06-08 | Qualcomm Incorporated | Single-processor computer vision hardware control and application execution |
| CN108733368A (en) * | 2017-05-16 | 2018-11-02 | 研祥智能科技股份有限公司 | Machine vision general software development system |
| CN108274466A (en) * | 2018-01-17 | 2018-07-13 | 广州威沃电子有限公司 | Intelligent vision flexible wires system based on industrial robot |
| CN211906310U (en) * | 2018-10-19 | 2020-11-10 | 成都信息工程大学 | Zynq-based machine vision detection system |
| CN109922247A (en) * | 2019-04-17 | 2019-06-21 | 浙江禾川科技股份有限公司 | A kind of smart camera and a kind of image processing method |
| CN111698418A (en) * | 2020-04-17 | 2020-09-22 | 广州市讯思视控科技有限公司 | Industrial intelligent camera system based on deep learning configuration cloud platform |
Non-Patent Citations (2)
| Title |
|---|
| 屠大维等: "智能机器人视觉体系结构研究", 《机器人》 * |
| 金桂银等: "视觉识别机器人在物流作业中的智能应用", 《制造业自动化》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114911537A (en) * | 2022-05-10 | 2022-08-16 | 声呐天空资讯顾问有限公司 | Parameter configuration method and system, and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2018065158A1 (en) | Computer device for training a deep neural network | |
| CN109740590B (en) | ROI accurate extraction method and system based on target tracking assistance | |
| US20220101640A1 (en) | Method of processing information from an event-based sensor | |
| WO2023040146A1 (en) | Behavior recognition method and apparatus based on image fusion, and electronic device and medium | |
| CN112016461A (en) | Multi-target behavior identification method and system | |
| CN112037142B (en) | Image denoising method, device, computer and readable storage medium | |
| WO2024001123A1 (en) | Image recognition method and apparatus based on neural network model, and terminal device | |
| US20210397860A1 (en) | Object detection for event cameras | |
| CN112990174A (en) | Visual tracking platform and method for multi-application scene | |
| CN113065568A (en) | Target detection, attribute identification and tracking method and system | |
| Kyrkou | C 3 Net: end-to-end deep learning for efficient real-time visual active camera control | |
| CN113095199B (en) | High-speed pedestrian identification method and device | |
| CN114359554A (en) | Image semantic segmentation method based on multi-receptive-field context semantic information | |
| Duan et al. | A more accurate mask detection algorithm based on Nao robot platform and YOLOv7 | |
| CN111382705A (en) | Reverse behavior detection method and device, electronic equipment and readable storage medium | |
| Liu et al. | Abnormal behavior analysis strategy of bus drivers based on deep learning | |
| CN115984124A (en) | Method and device for de-noising and super-resolution of neuromorphic pulse signals | |
| Schraml et al. | A real-time pedestrian classification method for event-based dynamic stereo vision | |
| CN112019723B (en) | Big data target monitoring method and system of block chain | |
| Lyu et al. | Roadnet-v2: A 10 ms road segmentation using spatial sequence layer | |
| Li et al. | Improved edge lightweight YOLOv4 and its application in on-site power system work | |
| Xia et al. | Unsupervised Optical Flow Estimation with Dynamic Timing Representation for Spike Camera | |
| CN117079079B (en) | Training method of video anomaly detection model, video anomaly detection method and system | |
| CN117789255A (en) | Pedestrian abnormal behavior video identification method based on attitude estimation | |
| CN116721124A (en) | Method and device for three-dimensional object detection by means of a road-side sensor unit |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210618 |
|
| RJ01 | Rejection of invention patent application after publication |