CN1950846A - Image processing apparatus and method - Google Patents
Image processing apparatus and method Download PDFInfo
- Publication number
- CN1950846A CN1950846A CNA2005800136763A CN200580013676A CN1950846A CN 1950846 A CN1950846 A CN 1950846A CN A2005800136763 A CNA2005800136763 A CN A2005800136763A CN 200580013676 A CN200580013676 A CN 200580013676A CN 1950846 A CN1950846 A CN 1950846A
- Authority
- CN
- China
- Prior art keywords
- dimension
- data
- image processing
- processing equipment
- fit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T1/00—General purpose image data processing
- G06T1/20—Processor architectures; Processor configuration, e.g. pipelining
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
An image processing apparatus and method identifies a region of interest 22a, for example a line segment, in an image frame. Preferably, the region of interest is defined within a course bounding box 21a, which is used to define a 'best-fitting' boundary box 23a. The best-fitting bounding box 23a is then rescanned to an orthogonal grid 25a. The line segment 22a is therefore represented by a two dimensional array, with the entire length of the line segment 22a being in a first dimension, and pixel data to the left and/or right of the line segment 22a being in a second dimension. The orthogonal grid 25 is then processed by the SIMD processor 26 for pixel parallel processing, for example edge finding, enhancement, interpolation or line data. Since the line segment 22a has been rescanned such that its entire length is found in one dimension, the image data is more suited for line-region based processing by the SIMD processor 26. Preferably, the first dimension corresponds to a row in an image and the second dimension to a column in an image. Once the pixel data has been processed by the SIMD processor, scan information stored during the rescanning operation is used to scan the image data back into original image frame.
Description
The present invention relates to image processing equipment and method, particularly use the image processing equipment and the method for single instruction multiple data (SIMD), wherein pixel is rescaned, so that utilize the parallel processing capability of SIMD processor better.
The SIMD processing is the strong computation paradigm to the application with large-scale parallel (parallelism).Such application taking to use SIMD to handle is a Flame Image Process.The SIMD processor, for example Xetal carries out their operation (for example, to Xetal each pixel in delegation) for each data item, no matter whether they need.In other words, whole row is carried out the processing operation, and no matter whether need to handle operation.Depend on DATA DISTRIBUTION or sparse property, use this technology therefore can waste many computing powers.
The some parts that increasing image processing algorithm begins to do image works.For example, handle at TV, in industrial vision or the imaging of medical, (that is, line is handled) handled in the image border is known.In addition, presenting in such application such as Image Communication or 3D, it is known that the object that separates in the image is handled (that is, object handles), reduces the unnecessary processing operational ton thus.
In other is used, processing be entire frame, and use very powerful and effective SIMD processor possibly for this reason.Yet, handle for object or line, because the layout dispersed or object in image of object, SIMD does not provide effective disposal route.This means that SIMD is uneconomic for handling such treatment technology, because it has handled most of uninterested data.
For effective use SIMD computational resource, there are several solutions.For example, method is the balancing the load on a plurality of SIMD processors.Another method provides uses special data structure to act on effectively in the algorithm of sparsity structure.For example, such technology is at " Massive parallelism for sparse images " (large-scale parallel that is used for the sparse graph picture), people such as Shankar, IEEE International Conference on Decision Aiding forComplex Systems describes in 1991.
Yet above-mentioned method will suffer hardware overheads, and they have such shortcoming: the mode of image data processing can not be handled compatible mutually with SIMD fully.
The purpose of this invention is to provide the image processing equipment and the method that can not suffer above-mentioned shortcoming.
According to one aspect of the present invention, a kind of image processing equipment is provided, comprising: the device that is used for recognition image signal interesting areas; Be used for the view data corresponding to interesting areas is limited in an orthogonal area with first peacekeeping, second dimension; And processor array, be used for handling the view data of this orthogonal area.
By a zone is rescaned in the orthogonal grid, the present invention makes processor array can carry out effectively efficiently based on line or based on the processing of rectangle by assembling data block on the line.
According to another aspect of the present invention, the method for handling picture signal in the processor of the array with parallel processing element is provided, this method may further comprise the steps:
Interesting areas in the recognition image signal; Interesting areas is limited in the orthogonal area with first peacekeeping second dimension; And the view data of in this processor, handling this orthogonal area.
In order to understand the present invention better and, to come as an example with reference to the accompanying drawings now, wherein in order more clearly to show how to implement the present invention:
Fig. 1 shows the basic element of character of known intelligent camera;
Fig. 2 shows the image processing method that is used for edge or line that has the processing of rescaning according to the present invention;
Fig. 3 shows the image processing method that is used for object that has the processing of rescaning according to the present invention;
Fig. 4 is a process flow diagram of describing step of the present invention in detail;
Fig. 5 shows how object line is created to storer; And
How Fig. 6 display image data stores the storer of Fig. 5 into.
Fig. 1 shows the block diagram of the critical piece of known intelligent camera 1.Intelligent camera 1 comprises chip 3, and it has sensor array 4, cmos sensor array for example, and this sensor array receives the image that will obtain.SIMD handles array just as on sensor array 4 the is integrated in same chip 3 on the chip, and comprises SIMD pe array (PE) 5 and the storer 7 that distributes.SIMD processing array on chip utilizes the general processor 8 outside the chip and is expanded.Intelligent camera 1 comprises that also other is used to handle the functional element of picture signal, such as data RAM 9, instruction RAM 11, ILP (instruction-level parallelism) processor 13 and input and output networking unit 15.
General processor 8 provides high-level image processing function, such as feature extraction, object detection and tracking.The present invention is according to the image-capable that enhancing is provided by the object detection functions of describing such intelligent camera 1 such as Fig. 1 and providing.
Fig. 2 shows according to processing edge of image of the present invention and or the method for line.Picture frame 20 has some interesting areas that are identified therein, and for example line segment 22a is to 22c.It is in orthogonal area that each line segment 22a is prescribed to 22c, promptly in the distance confinement block (bounding block).For example, Fig. 2 is shown as the distance confinement block 21a of line segment 22a regulation.Then, distance confinement block 21a is used to regulation " best-fit " boundary block (boundary block) 23a.Preferably, the boundary block 23a of best-fit is perpendicular to the edge of line segment 22a, and has for example span of 5 pixels.
The boundary block 23a of best-fit is rescanned among the orthogonal grid 25a then.This involves the confinement block 23a that grid is applied to best-fit, shown in 24a.Then, by using interpositioning to calculate each pixel value of orthogonal grid 25a according to the orthogonal grid among the 24a.As a result, line segment 22a thereby quilt " bending ", thus it will be represented by two-dimensional array, the whole length of line segment 22a is in first dimension, is in second dimension on the left side of line segment 22a and the pixel data on the right side.Rescaning operating period, image processing equipment remains on storer to scanning information and is provided with the back use, as what describe below.
Handle orthogonal grid 25a to carry out the pixel parallel processing by SIMD processor 26 then, for example edge searching, enhancing, the interpolation of line data.Because line segment 22a is rescaned and makes its whole length try to achieve on a dimension, so view data is more suitable for the processing undertaken based on the line zone by SIMD processor 26.Preferably, first dimension is corresponding to the row in the image, and second dimension is corresponding to the row in the image.Provide orthogonal grid to have such advantage, promptly guarantee each processor to pixel carry out similar instruction from but the SIMD compatibility.Therefore, according to the present invention, the instruction stream that is used for the line algorithm typically is quadrature on line.
In case pixel data is by the SIMD processor processing, just be used for view data is rescaned in original grid 27 rescaning the scanning information that operating period stores.Original grid 27 is then by being transformed into normal line segment 22a to line segment from the crooked more original form of getting back to it of first dimension.If necessary, line segment is re-inserted into picture frame 20 after can having been done to handle by the SIMD processor then.In this stage, carry out inverse scan, so that being placed, the line of handling gets back to original image.Alternatively, if do not need image (for example because only need measured value), then final stage can be omitted.
Fig. 3 shows according to the object in the processing image of the present invention or the method for spot.Object 32a is identified in picture frame 30 to 32c.Each object 32a has the zone of quadrature to 32c, promptly with its distance confinement block of relation regulation.For example, Fig. 3 shows the distance confinement block 31a for object 32a regulation, and then, according to an embodiment, distance confinement block 31a is used to the confinement block 33a of regulation best-fit.Though Fig. 3 has only shown the confinement block 33a of a best-fit, the confinement block 33a of one or more best-fits can be used to come the regulation object according to the feature of object 32a.The confinement block 33a of best-fit uses known algorithm to determine, or by determining when determine by best-fit under different rotations for object.Then, in rescaning into two-dimensional quadrature grid 36a before, grid is applied to the confinement block 33a of best-fit, shown in 35a.Rescaning operating period, image processing equipment is preserved scanning information and is provided with the back use, as what describe below.
SIMD processor processing orthogonal grid 36a then, so that carry out the pixel parallel processing, for example filtering, identification or the like.View data is more effectively handled, because object data is transformed into two-dimensional grid, thereby makes it be more suitable in by the SIMD processor processing.Because the result of above-mentioned operation, object was remapped before by the SIMD processor processing effectively, made object be in than in still less the row in the past, avoided handling the needs of entire frame thus.In case pixel data is by the SIMD processor processing, with regard to being used to image 39a is rescaned in the output of SIMD processor 38a at the scanning information that rescans the operating period storage, if desired, it is inserted the picture frame of having been crossed by the SIMD processor processing 30 again.Alternatively, if do not need picture signal,,, then rescan step and can omit with what 38a and 39a represented such as zone or colored if for example only need numeral or measured value from object.
Above-mentioned a kind of image processing equipment and the method for the invention provides wherein more effectively uses SIMD to handle during the object on handling image.
Should be pointed out that in the embodiments of figure 3 some stage that involves is only chosen wantonly when process object, thus can omit, and still keep processing power very high in the SIMD processor effectively.For example,, rescan stage 33a owing to be quadrature by the zone of distance confinement block 31a regulation, 35a, and 36a (together with corresponding back to scanning (scan back) stage 39a) can be omitted.In other words, though the preferred embodiment of Fig. 3 shows that the confinement block 33a of best-fit is determined, it is rescanned to quadrature piece 36a then, and the present invention also can only utilize the orthogonal area by distance confinement block 31a regulation.Though this alternative embodiment does not comprise that the embodiment in all stages is effective like that, in any case it still provides the validity of height, because the pixel data in quadrature distance confinement block 31a is processed, rather than entire frame.
Fig. 4 is summarized in the step that involves in the method according to the object in the processing picture signal of the present invention.At first, determine distance confinement block, step 401 for interesting areas in the picture signal that receives.Use the distance confinement block to come the confinement block of regulation best-fit, step 403 then.View data in the confinement block of best-fit is rescanned to orthogonal grid, step 405 then.In response to the instruction that receives from program source 408, the view data through rescaning is by SIMD processor processing, step 407.By using the scanning information of during rescaning process, storing 406, scan treated view data backward, step 409 then.
As mentioned above, step 403,405,409th, choose wantonly, because the SIMD processor can be configured to directly handle the pixel data in quadrature distance confinement block.
Fig. 5 shows how object line is created to storer 503.Processor 501 is carried out the identification of objects and line and mark and the signal that receives is carried out general processing.In the second level 502, determine for object and line that confinement block and execution are rescaned respectively and/or crooked.Output data 502a and control information 502b from the second level 502 are sent to storer 503.Storer is with the mode storing image data (below with reference to Fig. 6 discussion) of the conversion done in subordinate phase.SIMD processor 504 is handled the view data that is stored in the storer 503.The output of SIMD processor or be used to measure 505 perhaps is sent to and carries out contrary rescaning and/or the level 506 of bending operation.Contrary rescan and/or bending is to object or line execution by using the 502 control information 502b that receive from the second level.Treated object/line from level 506 is sent to combiner 507 then, and picture signal that the combiner combination is original and treated picture signal are to produce the picture signal 508 that finally obtains.
How Fig. 6 display image data stores the storer of Fig. 5 into.Can see, the processing of being undertaken by the SIMD processor be limited to the row of the corresponding picture frame of object data and with the corresponding row of line data (these line data have been transformed into straight-line segment, promptly are not first dimensions).As a result, be not to handle entire frame, but the SIMD processor is only handled through conversion and data line processor quadrature.
Therefore, above-mentioned the present invention rescans view data become in the predetermined dimension corresponding to the row or column in the image, makes view data be more suitable in being handled by simd array thus.
The present invention can be used to many different application, comprising: the processing of television image is to improve picture quality; In using, carries out computer video object identification; The image that execution is used for computer game, education or CAD/CAM presents; Be MPEG4, H263+ carries out object-based coding; Be the medical system carries out image processing.
Though should be pointed out that preferred embodiment relates to rescans into the two-dimensional quadrature grid, should see that when handling such as the such 3 d image data of video data, grid can be three-dimensional, rather than two dimension.
Should be pointed out that the above embodiments are explanation rather than restriction the present invention, those skilled in the art can design many alternative embodiments, and not deviate from the scope of the present invention as the claims regulation.In the claims, any label that is placed in the bracket should not seen the restriction claim as.Individual character " comprises " not to be got rid of and those the different unit integrally listed in any claim or patent specification or the existence of step.The singular reference of unit is not got rid of most labels of such unit, and vice versa.The present invention can be implemented by means of the hardware that comprises several different unit with by means of the computing machine of suitably programming.In enumerating the claim of several means, several such devices can be implemented by same hardware branch.The fact that some measure is set forth in different mutually appended claims does not represent that the combination of these measures can not be used for benefiting.
Claims (19)
1. method of in the processor of array, handling picture signal with parallel processing element, this method may further comprise the steps:
Interesting areas in the recognition image signal;
Interesting areas is limited in orthogonal area with first peacekeeping, second dimension; And
In processor, handle the view data of this orthogonal area.
2. the method as requiring in the claim 1 also comprises the segment data in the picture signal is transformed to step in first dimension.
3. the method as requiring in the claim 2, also comprise the left side of segment data and or the pixel data on the right side transform to second the step in tieing up.
4. as the method for requirement in claim 2 or 3, further comprising the steps of:
Channel data in the picture signal is determined the confinement block of best-fit; And
Confinement block best-fit before treatment step rescans in the orthogonal grid.
5. as the method for requirement in the claim 1, further comprising the steps of:
Object in the picture signal is determined the confinement block of best-fit; And
Confinement block best-fit before treatment step rescans in the orthogonal grid.
6. the method as requiring in claim 4 or 5 also is included in and rescans the step that information is rescaned in storage during the step.
7. the method as requiring in the claim 6 also comprises the following steps: to use during rescaning step canned data that the processed images data are carried out the contrary step that rescans.
8. the method as requiring in the claim 7 also comprises segment data from remap back dimension before its conversion of first dimension.
9. the method that requires in each of claim as described above, wherein first dimension is corresponding to the row of picture frame.
10. the method that requires in each of claim as described above, wherein second dimension is corresponding to the row of picture frame.
11. an image processing equipment comprises:
The device that is used for recognition image signal interesting areas;
Be used for interesting areas is limited in the device of orthogonal area with first peacekeeping, second dimension; And
Processor array is used for handling the view data of orthogonal area.
12., also comprise the device that is used for the segment data of picture signal is transformed into first dimension as the image processing equipment that requires in the claim 1.
13. as the image processing equipment that requires in the claim 12, also comprise be used for the left side of segment data and or the pixel data on the right side be transformed into second device of tieing up.
14. the image processing equipment as requiring in claim 12 or 13 also comprises:
Be used for the segment data of picture signal is determined the device of the confinement block of best-fit; And
Be used for the confinement block of best-fit is rescaned into the device of orthogonal grid.
15. the image processing equipment as requiring in the claim 11 also comprises:
Be used for the object of picture signal is determined the device of the confinement block of best-fit; And
Be used for the confinement block of best-fit is rescaned into the device of orthogonal grid.
16., also comprise the memory storage that is used to store by rescaning the information that rescans that device provides as the image processing equipment that requires in each of claim 11 to 15.
17., also comprise being used for the processed images data being carried out the contrary device of operation that rescans by using at the memory storage canned data as the image processing equipment that requires in the claim 16.
18., also comprise being used for segment data from remap the back device of the dimension before its conversion of first dimension as the image processing equipment that requires in the claim 17.
19. as the image processing equipment that requires in each of claim 11 to 18, wherein processor is the SIMD processor.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| EP04101843 | 2004-04-29 | ||
| EP04101843.3 | 2004-04-29 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN1950846A true CN1950846A (en) | 2007-04-18 |
Family
ID=34965517
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CNA2005800136763A Pending CN1950846A (en) | 2004-04-29 | 2005-04-26 | Image processing apparatus and method |
Country Status (4)
| Country | Link |
|---|---|
| EP (1) | EP1745433A1 (en) |
| JP (1) | JP2007535066A (en) |
| CN (1) | CN1950846A (en) |
| WO (1) | WO2005106786A1 (en) |
Families Citing this family (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9451142B2 (en) | 2007-11-30 | 2016-09-20 | Cognex Corporation | Vision sensors, systems, and methods |
| US8570393B2 (en) | 2007-11-30 | 2013-10-29 | Cognex Corporation | System and method for processing image data relative to a focus of attention within the overall image |
| US8200025B2 (en) | 2007-12-07 | 2012-06-12 | University Of Ottawa | Image classification and search |
| EP2248122A1 (en) * | 2008-02-25 | 2010-11-10 | Inventive Medical Limited | Medical training method and apparatus |
| US8260002B2 (en) * | 2008-09-26 | 2012-09-04 | Axis Ab | Video analytics system, computer program product, and associated methodology for efficiently using SIMD operations |
| US9189670B2 (en) | 2009-02-11 | 2015-11-17 | Cognex Corporation | System and method for capturing and detecting symbology features and parameters |
| WO2014117108A1 (en) * | 2013-01-25 | 2014-07-31 | Duke University | Segmentation and identification of closed-contour features in images using graph theory and quasi-polar transform |
| WO2016127140A1 (en) | 2015-02-05 | 2016-08-11 | Duke University | Compact telescope configurations for light scanning systems and methods of using the same |
| US10238279B2 (en) | 2015-02-06 | 2019-03-26 | Duke University | Stereoscopic display systems and methods for displaying surgical data and information in a surgical microscope |
| US10694939B2 (en) | 2016-04-29 | 2020-06-30 | Duke University | Whole eye optical coherence tomography(OCT) imaging systems and related methods |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CA1309198C (en) * | 1987-12-10 | 1992-10-20 | Carlo J. Evangelisti | Parallel rendering of smoothly shaded color triangles with anti-aliased edges for a three dimensional color display |
| EP0349630A1 (en) * | 1987-12-18 | 1990-01-10 | Digital Equipment Corporation | Method of drawing in graphics rendering system |
| US5808623A (en) * | 1996-10-07 | 1998-09-15 | Adobe Systems Incorporated | System and method for perspective transform in computer using multi-pass algorithm |
-
2005
- 2005-04-26 WO PCT/IB2005/051365 patent/WO2005106786A1/en not_active Application Discontinuation
- 2005-04-26 JP JP2007510221A patent/JP2007535066A/en active Pending
- 2005-04-26 CN CNA2005800136763A patent/CN1950846A/en active Pending
- 2005-04-26 EP EP05718810A patent/EP1745433A1/en not_active Withdrawn
Also Published As
| Publication number | Publication date |
|---|---|
| EP1745433A1 (en) | 2007-01-24 |
| WO2005106786A1 (en) | 2005-11-10 |
| JP2007535066A (en) | 2007-11-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN1950846A (en) | Image processing apparatus and method | |
| US8023725B2 (en) | Identification of a graphical symbol by identifying its constituent contiguous pixel groups as characters | |
| Fernandes et al. | Real-time line detection through an improved Hough transform voting scheme | |
| US10558844B2 (en) | Lightweight 3D vision camera with intelligent segmentation engine for machine vision and auto identification | |
| Wang et al. | High-fidelity view synthesis for light field imaging with extended pseudo 4DCNN | |
| CN110060230B (en) | Three-dimensional scene analysis method, device, medium and equipment | |
| KR20160106045A (en) | A light field processing method | |
| US20230048234A1 (en) | Real-time video overlaying display | |
| CN1950843A (en) | Image processing apparatus and method | |
| Baygin et al. | A new image stitching approach for resolution enhancement in camera arrays | |
| CN111179166A (en) | Image processing method, device, equipment and computer readable storage medium | |
| Iwatsuki et al. | Unsupervised disparity estimation from light field using plug-and-play weighted warping loss | |
| KR100640761B1 (en) | Method of extracting 3 dimension coordinate of landmark image by single camera | |
| Javan Hemmat et al. | Real-time planar segmentation of depth images: from three-dimensional edges to segmented planes | |
| Huang et al. | An improved RANSAC algorithm of color image stitching | |
| Gao et al. | A new image mosaics algorithm based on feature points matching | |
| WO2008138802A1 (en) | Device for object detection in an image, and method thereof | |
| Nasrin et al. | Partially occluded object reconstruction using multiple Kinect sensors | |
| Ren et al. | Rgb-d salient object detection: a review | |
| Schaeferling et al. | ASTERICS-An Open Toolbox for Sophisticated FPGA-Based Image Processing | |
| Ammar et al. | Visual saliency in MPEG-4 AVC video stream | |
| Limonova et al. | Mobile and embedded fast high resolution image stitching for long length rectangular monochromatic objects with periodic structure | |
| Kareem et al. | Real-Time Panoramic Video Construction Using Harris Corners Detector | |
| Mosqueron et al. | Smart camera with embedded co-processor: a postal sorting application | |
| Jin et al. | Dual-Fisheye Image Stitching via Unsupervised Deep Learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C02 | Deemed withdrawal of patent application after publication (patent law 2001) | ||
| WD01 | Invention patent application deemed withdrawn after publication |