CN105516601B - Device and method for real-time processing of gesture images - Google Patents
Device and method for real-time processing of gesture images Download PDFInfo
- Publication number
- CN105516601B CN105516601B CN201610010666.4A CN201610010666A CN105516601B CN 105516601 B CN105516601 B CN 105516601B CN 201610010666 A CN201610010666 A CN 201610010666A CN 105516601 B CN105516601 B CN 105516601B
- Authority
- CN
- China
- Prior art keywords
- core
- gesture
- input end
- image processing
- video
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/66—Remote control of cameras or camera parts, e.g. by remote control devices
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/10—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths
- H04N23/12—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths with one sensor only
Abstract
The invention provides a device and a method for real-time processing of gesture images. The embedded multi-core computing module is embedded in the FPGA and comprises a soft-core processor, an integrated circuit bus configuration module and the like, wherein a camera signal of an external PAL system is sent to the input end of a gesture image acquisition module, the output end of the gesture image acquisition module is respectively connected with the input end of the integrated circuit bus configuration module and the input end of an image processing IP core, the input end of the integrated circuit bus configuration module and the input end of the image processing IP core are connected with the input end of the soft-core processor, and the output end of the soft-core processor is connected with a video graphic array controller and a double-rate synchronous dynamic random access memory controller. The invention can effectively improve the real-time performance of gesture image processing, reduce the power consumption of system operation and can be flexibly embedded into other systems.
Description
Technical Field
The invention belongs to the technical field of image hardware processing, particularly relates to a device and a method for real-time processing of gesture images, and belongs to the innovative technology of the device and the method for real-time processing of the gesture images.
Background
With the development of image hardware processing technology and the wide application in the fields of industrial detection, medical equipment, national defense and military and the like, higher requirements are put forward on the real-time performance and low power consumption of the image processing technology. However, most of the current image processing technologies are realized by a software programming mode on a PC, and the processing speed is low, so that the real-time requirement cannot be met.
Disclosure of Invention
In order to solve the defects of the prior art, the invention aims to provide a device for real-time processing of gesture images, which can preprocess gesture images captured by a camera, effectively improve the real-time performance of image processing, reduce the power consumption of system operation, and be flexibly embedded into other systems.
Another object of the present invention is to provide a method for real-time processing of gesture images. The invention is simple and convenient to control. The invention can not only accelerate the real-time image processing, but also run with low power consumption.
The purpose of the invention is realized by adopting the following technical scheme:
the device for real-time processing of the gesture image comprises an embedded multi-core computing module and a gesture image acquisition module, wherein the output end of the gesture image acquisition module is connected with the input end of the embedded multi-core computing module.
The embedded multi-core computing module is embedded in the FPGA and comprises a soft-core processor, an integrated circuit bus configuration module, an image processing IP core, a video graphic array controller and a double-rate synchronous dynamic random access memory controller, wherein a camera signal of an external PAL system is directly sent to the input end of a gesture image acquisition module, the output end of the gesture image acquisition module is respectively connected with the input end of the integrated circuit bus configuration module and the input end of the image processing IP core, the input end of the integrated circuit bus configuration module and the input end of the image processing IP core are connected with the input end of the soft-core processor, and the output end of the soft-core processor is connected with the video graphic array controller and the double-rate synchronous dynamic random access memory controller.
The invention relates to a processing method of a device for real-time processing of gesture images, which comprises the following steps:
1) after the system is powered on, the embedded multi-core computing module initializes a video decoding chip TVP5150PBS for gesture image acquisition through the integrated circuit bus configuration module, and converts PAL/NTSC analog video output by the analog camera into digital color difference signals in ITU-R BT.656 standard format and outputs the digital color difference signals to an image processing IP core;
2) after an image processing IP core in the embedded multi-core computing module acquires a digital color difference signal, sequentially performing the steps of time sequence detection, color space conversion, gesture image preprocessing, line caching and the like to obtain a processed RGB format gesture image; the specific processes of time sequence detection, color space conversion, gesture image preprocessing and line caching are as follows: firstly, separating a signal stream including a field flag bit, a field state flag bit, a line state flag bit and YCbCr video data from a digital video stream after time sequence detection is completed, then converting the YCbCr video data into RGB video data through a color space, preprocessing the video data through a gesture image to obtain binary RGB data, and finally temporarily storing the binary RGB data in a line cache constructed by a Block RAM;
3) under the scheduling of the soft-core processor, data processed by the image processing IP core can be transmitted to the video graphic array controller through the PLB bus, the video graphic array controller is responsible for generating a field synchronizing signal and a line synchronizing signal and transmitting RGB data to external video graphic array display equipment to realize the online display of the gesture image processing effect.
The invention introduces a field programmable logic device (FPGA) with a parallel processing mechanism, proposes to customize an image processing system-on-chip (SoC) on the FPGA, and utilizes the mode that all IP cores work cooperatively, compared with the prior art, the technical scheme of the invention has the beneficial effects that:
1) the invention adopts the modularized design idea, and specially customizes an image processing IP core for preprocessing the gesture image input by the gesture image acquisition in the embedded multi-core control system, thereby effectively improving the gesture image processing speed and ensuring the real-time property of the image processing system;
2) because the embedded multi-core control system adopts the FPGA chip, the high-speed processing capability of the embedded multi-core control system is determined by the intrinsic parallel mechanism to be far better than that of a serial execution architecture adopted by the traditional PC. Therefore, the speed of real-time image processing can be greatly improved, and the processing efficiency is improved;
3) the programmable logic device technology adopted by the invention can carry out targeted software and hardware cutting on the gesture image processing control system according to the actual application scene, and can be flexibly embedded into other systems, thereby realizing the advantages of low power consumption, good stability and the like.
Drawings
FIG. 1 is a system block diagram of the device for real-time processing of gesture images of the present invention.
Detailed Description
The technical solutions of the present invention will be further described with reference to the accompanying drawings and examples, wherein the drawings are used for providing further understanding of the present invention and form a part of the specification, and the accompanying drawings and examples are used together for explaining the present invention and do not limit the present invention.
Example 1:
as shown in fig. 1, the device for real-time processing of gesture images of the present invention includes an embedded multi-kernel computing module 1 and a gesture image acquisition module 2, wherein an output end of the gesture image acquisition module 2 is connected to an input end of the embedded multi-kernel computing module 1.
In this embodiment, the embedded multi-core computing module 1 is embedded in an FPGA, and includes a soft-core processor 11, an integrated circuit bus configuration module 12, an image processing IP core 13, a video graphics array controller 14, and a double-rate synchronous dynamic random access memory controller 15, where a camera signal of an external PAL system is directly sent to an input end of a gesture image acquisition module 2, an output end of the gesture image acquisition module 2 is connected to an input end of the integrated circuit bus configuration module 12 and an input end of the image processing IP core 13, an input end of the integrated circuit bus configuration module 12 and an input end of the image processing IP core 13 are connected to an input end of the soft-core processor 11, and an output end of the soft-core processor 11 is connected to the video graphics array controller 14 and the double-rate synchronous dynamic random access memory controller 15. The gesture image acquisition module 2 sends the processed signal to the input end of the image processing IP core 13.
In this embodiment, the soft core processor 11 is connected to the integrated circuit bus configuration module 12, the image processing IP core 13, the video graphics array controller 14, and the double-rate synchronous dynamic random access memory controller 15 through the PLB bus.
In this embodiment, the gesture image acquisition module 2 is connected to the integrated circuit bus configuration module 12 through an integrated circuit bus.
In this embodiment, the output terminal of the video graphic array controller 14 is connected to an external video graphic array display device through a general video graphic array interface, and the output terminal of the ddr sdram controller 15 is connected to an external ddr sdram memory. After receiving the storage signal and the content sent by the soft core processor 11, the ddr sdram controller 15 stores the content in the external ddr sdram memory.
In this embodiment, the camera is a camera of an external PAL system.
In this embodiment, the gesture image collection module 2 adopts a video decoding chip TVP5150 PBS. The video signal conversion device is used for converting a video signal input to an external analog camera into a digital color difference signal and supporting two composite videos or one S terminal input.
The invention relates to a processing method of a device for real-time processing of gesture images, which comprises the following steps:
1) after the system is powered on, the embedded multi-core computing module 1 initializes a video decoding chip TVP5150PBS for gesture image acquisition through the integrated circuit bus configuration module 12, and converts PAL/NTSC analog video output by the analog camera into digital color difference signals in ITU-R BT.656 standard format and outputs the digital color difference signals to the image processing IP core 13.
2) After the image processing IP core 13 in the embedded multi-core computing module 1 obtains a digital color difference signal (YUV), the steps of timing detection, color space conversion, gesture image preprocessing, line caching and the like are sequentially performed to obtain a processed RGB format gesture image. The specific process comprises the following steps: firstly, separating a signal stream including a field flag bit, a field state flag bit, a line state flag bit and YCbCr video data from a digital video stream after time sequence detection is completed, then converting the YCbCr video data into RGB video data through a color space, preprocessing the video data through a gesture image to obtain binary RGB data, and finally temporarily storing the binary RGB data in a line cache constructed by a Block RAM.
3) Under the scheduling of the soft-core processor 11, the data processed by the image processing IP core 13 can be transmitted to the video graphic array controller 14 through the PLB bus, and the video graphic array controller 14 is responsible for generating a field synchronization signal and a line synchronization signal and transmitting the RGB data to an external video graphic array display device to realize the online display of the gesture image processing effect.
In this embodiment, under the scheduling of the soft-core processor 11, the data processed by the image processing IP core 13 can also be transmitted to the double-rate synchronous dynamic random access memory controller 15 through the PLB bus, and the double-rate synchronous dynamic random access memory controller 15 is responsible for implementing three functions of state conversion, instruction decoding, and clock generation, and completing the storage of the gesture image in the external double-rate synchronous dynamic random access memory.
The state conversion is to complete the switching of different working states in the double-rate synchronous dynamic random access memory; the instruction decoding is to finish decoding the input instruction so that the controller can correctly execute the command; the clock generation is finished to provide a clock for the external double-rate synchronous dynamic random access memory to ensure that data can be normally stored. The different working states comprise reading and writing working states.
Claims (3)
1. A processing method of a device for real-time processing of gesture images is disclosed, wherein the device for real-time processing of gesture images comprises an embedded multi-kernel computing module (1) and a gesture image acquisition module (2), and the output end of the gesture image acquisition module (2) is connected with the input end of the embedded multi-kernel computing module (1); the embedded multi-core computing module (1) is embedded in the FPGA and comprises a soft core processor (11), an integrated circuit bus configuration module (12) and an image processing IP core (13), the system comprises a video graphic array controller (14) and a double-rate synchronous dynamic random access memory controller (15), wherein a camera signal of an external PAL system is directly sent to an input end of a gesture image acquisition module (2), an output end of the gesture image acquisition module (2) is respectively connected with an input end of an integrated circuit bus configuration module (12) and an input end of an image processing IP core (13), an input end of the integrated circuit bus configuration module (12) and an input end of the image processing IP core (13) are connected with an input end of a soft-core processor (11), and an output end of the soft-core processor (11) is connected with the video graphic array controller (14) and the double-rate synchronous dynamic random access memory controller (15); the soft-core processor (11) is connected with an integrated circuit bus configuration module (12), an image processing IP core (13), a video graphic array controller (14) and a double-rate synchronous dynamic random access memory controller (15) through a PLB bus; the method is characterized by comprising the following steps:
1) after a system is powered on, an embedded multi-core computing module (1) initializes a video decoding chip TVP5150PBS for gesture image acquisition through an integrated circuit bus configuration module (12), and converts PAL/NTSC analog video output by an analog camera into a digital color difference signal in an ITU-R BT.656 standard format and outputs the digital color difference signal to an image processing IP core (13);
2) after an image processing IP core (13) in the embedded multi-core computing module (1) acquires a digital color difference signal, sequentially carrying out time sequence detection, color space conversion, gesture image preprocessing and line caching to obtain a processed RGB format gesture image; the specific processes of time sequence detection, color space conversion, gesture image preprocessing and line caching are as follows: firstly, separating a signal stream including a field flag bit, a field state flag bit, a line state flag bit and YCbCr video data from a digital video stream after time sequence detection is completed, then converting the YCbCr video data into RGB video data through a color space, preprocessing the video data through a gesture image to obtain binary RGB data, and finally temporarily storing the binary RGB data in a line cache constructed by a Block RAM;
3) under the scheduling of the soft-core processor (11), data processed by the image processing IP core (13) can be transmitted to the video graphic array controller (14) through a PLB bus, the video graphic array controller (14) is responsible for generating a field synchronization signal and a line synchronization signal, and RGB data is transmitted to external video graphic array display equipment to realize the online display of gesture image processing effects.
2. The processing method of the device for real-time processing of gesture images according to claim 1, wherein under the scheduling of the soft-core processor (11), the data processed by the image processing IP core (13) can be further transmitted to the double-rate synchronous dynamic random access memory controller (15) through a PLB bus, and the double-rate synchronous dynamic random access memory controller (15) is responsible for realizing three functions of state conversion, instruction decoding and clock generation, and completing the storage of the gesture images into an external double-rate synchronous dynamic random access memory.
3. The processing method of the device for real-time processing of gesture images according to claim 1, characterized in that the state transition is to complete the switching of different working states in the double rate synchronous dynamic random access memory controller (15); the instruction decoding is to finish decoding the input instruction so that the controller can correctly execute the instruction; after the clock generation is finished, providing a clock for the double-rate synchronous dynamic random access memory controller (15) to ensure that data can be normally stored; the different working states comprise reading and writing working states.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610010666.4A CN105516601B (en) | 2016-01-08 | 2016-01-08 | Device and method for real-time processing of gesture images |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610010666.4A CN105516601B (en) | 2016-01-08 | 2016-01-08 | Device and method for real-time processing of gesture images |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105516601A CN105516601A (en) | 2016-04-20 |
| CN105516601B true CN105516601B (en) | 2020-01-17 |
Family
ID=55724201
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610010666.4A Active CN105516601B (en) | 2016-01-08 | 2016-01-08 | Device and method for real-time processing of gesture images |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105516601B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2011056224A1 (en) * | 2009-11-04 | 2011-05-12 | Pawan Jaggi | Switchable multi-channel data transcoding and transrating system |
| CN102075758A (en) * | 2011-02-24 | 2011-05-25 | 山东大学 | Motion joint photographic experts group (MJPEG) video coding and decoding system based on system on chip (SOC) and method thereof |
| CN202058090U (en) * | 2011-01-18 | 2011-11-30 | 上海理工大学 | Non-contact control system based on gesture recognition |
| CN202121711U (en) * | 2011-06-09 | 2012-01-18 | 南京觅踪电子科技有限公司 | Fog-penetrating monitoring system based on machine vision |
| CN102946529A (en) * | 2012-10-19 | 2013-02-27 | 华中科技大学 | Image transmission and processing system based on FPGA (Field Programmable Gate Array) and multi-core DSP (Digital Signal Processor) |
| CN206195927U (en) * | 2016-01-08 | 2017-05-24 | 广东工业大学 | A serial communication port, |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102663758A (en) * | 2012-04-20 | 2012-09-12 | 北京工业大学 | Image acquiring and processing method based on FPGA (field programmable gate array) serving as control core |
-
2016
- 2016-01-08 CN CN201610010666.4A patent/CN105516601B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2011056224A1 (en) * | 2009-11-04 | 2011-05-12 | Pawan Jaggi | Switchable multi-channel data transcoding and transrating system |
| CN202058090U (en) * | 2011-01-18 | 2011-11-30 | 上海理工大学 | Non-contact control system based on gesture recognition |
| CN102075758A (en) * | 2011-02-24 | 2011-05-25 | 山东大学 | Motion joint photographic experts group (MJPEG) video coding and decoding system based on system on chip (SOC) and method thereof |
| CN202121711U (en) * | 2011-06-09 | 2012-01-18 | 南京觅踪电子科技有限公司 | Fog-penetrating monitoring system based on machine vision |
| CN102946529A (en) * | 2012-10-19 | 2013-02-27 | 华中科技大学 | Image transmission and processing system based on FPGA (Field Programmable Gate Array) and multi-core DSP (Digital Signal Processor) |
| CN206195927U (en) * | 2016-01-08 | 2017-05-24 | 广东工业大学 | A serial communication port, |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105516601A (en) | 2016-04-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102025934B (en) | Digital television system on a chip (SoC) storage and control method based on automatic X-ray inspection (AXI) bus | |
| CN205726099U (en) | The video matrix system that a kind of multi-format video signal is switched fast | |
| CN107710757B (en) | Adaptive bulk coding for slow motion video recording | |
| CN102075758B (en) | Motion joint photographic experts group (MJPEG) video coding and decoding system based on system on chip (SOC) and method thereof | |
| CN102857703A (en) | High-definition video character superimposing system and control method | |
| CN108200359A (en) | A kind of multi-standard video frequency superimposer for airborne indicator | |
| CN106817545B (en) | A kind of fast multiresolution video image mirror image rotation processing system | |
| CN108492242B (en) | Implementation of 2D desktop hybrid operation based on GPGPU | |
| CN109873998B (en) | Infrared video enhancement system based on multi-level guide filtering | |
| CN108134912B (en) | Video stream conversion method | |
| CN204539339U (en) | Based on video acquisition and the detection system of SOPC | |
| CN105516601B (en) | Device and method for real-time processing of gesture images | |
| CN115002304A (en) | Video image resolution self-adaptive conversion device | |
| CN206195927U (en) | A serial communication port, | |
| CN211959380U (en) | Image processing module | |
| CN107911610A (en) | A kind of data handling system applied to image capture module | |
| CN204575844U (en) | A kind of New Type Radar information acquisition apparatus | |
| CN202110329U (en) | Digital microscope | |
| CN107623834A (en) | A kind of moving object detection system based on FPGA | |
| CN103716636A (en) | TMS320DM642-based video image processing system | |
| CN204883833U (en) | Image processing device | |
| CN203675196U (en) | Network control 3G-SDI high-definition characters superimposer | |
| Pandey et al. | An embedded architecture for implementation of a video acquisition module of a smart camera system | |
| CN203722733U (en) | Image format converter | |
| Yuan et al. | Study on Dual-port RAM-based Image Capture and Storage |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |