Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N5/00—Details of television systems
  • H04N5/14—Picture signal circuitry for video frequency region

Definitions

• the invention relates to a cellular network based video signal processing computer, a cellular chip and a method for converting one or more input video signals into one or more output video signals, where the conversion, i.e. the video signal processing, is carried out in real time without digitalization.
• a programmable analog computer is disclosed, which is based on a special type of cellular networks, the cellular neural network, called CNN-UM (CNN Universal Machine).
• the CNN-UM comprises a cell matrix consisting of locally interconnected programmable analog/logic cells. Each programmable cell, of the cell matrix comprises an analog core and local analog memory elements connected to the core via switches. Furthermore, the CNN-UM comprises a control unit for controlling the analog cores and the switches.
• “locally interconnected” means that an analog core can only communicate directly with surrounding analog cores located within a predetermined distance.
• the CNN-UM comprising the above cell matrix could already be suitable for real time processing of video signals without digitalizing.
• a video signal processing computer containing such an analog/logic cell matrix is only capable of processing video frames represented by one or more input video signals, if each pixel in the frame corresponds to a respective cell in the cell matrix.
• There are several hundred thousand pixels in one video frame but due to the limitations of the silicon technology, one silicon chip can only comprise up to some tens of thousand analog cells. Therefore, based on this known solution, it is not possible to design a cellular chip suitable for processing video signals in real time without digitalizing.
• a cellular chip since a cellular chip can not have as many programmable cells as the number of pixels in a whole frame, a cellular chip is to be designed that processes the frames in a segmented manner. Since the frames are received line by line, it is advisable to perform a horizontal segmentation of the frames. According to the invention, the video frames are segmented into horizontal stripes extending over, i.e. covering a number of video lines, and these stripes are then processed one by one. According to the invention, each of the programmable cells of the cellular chip is assigned to a relevant pixel in a given stripe.
• the video frames are processed in overlapping stripes. Due to the overlaps, the video frame stripes to be processed - because the input video signals are received on an ongoing basis in real time processing - are alternately read in separate memory areas, and a separate image processing operation is carried out for each stripe, using the stored values of the stripes.
• these memory areas are formed as input memory banks comprising local analog memory elements in each programmable cell. The result of the image processing operation is loaded alternately in separate memory areas, hereinafter output memory banks, in each programmable cell, and then the one or more output video signals can be generated by an alternating scheduled read-out of these output memory banks.
• the programmable cells in the cellular chip comprise a core being characteristic to the relevant cellular network, and in the given case they also comprise other processing units, for example those described in connection with the CNN-UM, as well as the input and output memory banks.
• the part of the programmable cell without the inventive input and output memory banks will be called a local processor.
• the design of the local processors is irrelevant from the aspect of the invention, and the solution offered by the invention can be used in any appropriate cellular network. Principally, the only difference between the cellular networks is in the analog cores of the local processors.
• the invention is a video signal processing computer for converting one or more input video signals into one or more output video signals, the computer comprising a cellular network consisting of a plurality of programmable cells, each of the programmable cells comprising a local processor core and local analog memory elements attached thereto via controllable internal switches, wherein the video signal processing computer comprises a control unit for controlling the local processor cores and the internal switches, characterized in that each of the programmable cells is assigned to a respective pixel of a horizontal frame stripe extending over a number of video lines, wherein the stripe comprises a useful area and an overlapping area, the local analog memory elements comprise input memory elements connected via controllable input switches to conductor lines carrying the input video signals, and output memory elements connected via controllable output switches to conductor lines carrying the output video signals, wherein stripes located one below and overlapping one another in the overlapping areas are processed successively in such a way that by means of the control of the input switches, input analog values associated with the pixels of the stripe are
• the invention is a cellular chip for a video signal processing computer converting one or more input video signals into one or more output video signals
• the cellular chip comprising a cellular network formed by a plurality of programmable cells, each of the programmable cells comprising a local processor core, local analog memory elements and an internal bus connecting the local analog memory elements with the local processor core, wherein the local processor core and the local analog memory elements are connected to the internal bus via controllable internal switches, characterized in that the programmable cells forming the cellular network are arranged in a stripe consisting of several lines, which stripe comprises a useful area and an overlapping area, the programmable cells in the useful area comprise at least two input memory banks and at least two output memory banks, the programmable cells in the overlapping area comprise at least two input memory banks and at least one output memory bank, wherein each input memory bank comprises input memory elements in a number corresponding to that of the input video signals, the input memory elements storing sampled values of the input video signals, and each output
• the invention is a method for converting one or more input video signals into one or more output video signals, wherein a cellular network comprising a plurality of programmable cells is used for the conversion, and each of the programmable cells comprises a local processor core and local memory elements, characterized in that the input video signals are processed in stripes one below and overlapping one another and extending over a number of video lines, the stripe comprising pixels in a useful area not overlapping the neighboring stripe and pixels in an overlapping area overlapping with the neighboring stripe, wherein the cellular network comprises one programmable cell for each of the pixels in the stripe, and each of the programmable cells is assigned to the same respective pixel for each stripe, the programmable cells comprising local input and output memory elements alternately assigned to the successive stripes, wherein input analog values associated with each pixel in a given stripe and obtained by a scheduled sampling of the input video signals are stored in the local input memory elements assigned to the given stripe of the programmable cell assigned to the pixel
• the invention is a method for converting one or more input video signals into one or more output video signals, wherein the video signals represent frames consisting of video lines, characterized in that the one or more input video signals are processed in stripes one below and overlapping one another and extending over a number of video lines, the stripe comprising pixels in a useful area not overlapping the neighboring stripe and pixels in an overlapping area overlapping the neighboring stripe, wherein by a scheduled sampling of the one or more input video signals, input analog values associated with the pixels of a first stripe are stored in a first memory unit assigned to the stripe, and after storing the input analog values of an entire stripe, an image processing operation is carried out, the image processing operation resulting output analog values associated with the pixels, the output analog values are loaded into a second memory unit assigned to the stripe, and then the output analog values associated with the pixels in the useful area of the stripe are read out in a scheduled way from the second memory unit, thereby generating a part of one or more output video signals.
• Fig. 1 is a schematic diagram of a video signal processing computer according to the invention
• Fig. 2 is a schematic stripe configuration with a symmetric boundary condition
• Fig. 3 is a schematic stripe configuration with an asymmetric boundary condition
• Fig. 4 is a schematic stripe configuration with a pre-calculated asymmetric boundary condition
• Fig. 5 is a timing diagram of a video signal processing computer designed with two input memory banks for processing stripes with a symmetric boundary condition
• Fig. 6 is a timing diagram of a video signal processing computer designed with three input memory banks for processing stripes with a symmetric boundary condition
• Fig. 7 is a timing diagram of a video signal processing computer designed with two input memory banks for processing stripes with an asymmetric boundary condition
• Fig. 8 is a timing diagram of a video signal processing computer designed with three input memory banks for processing stripes with an asymmetric boundary condition
• Fig. 9 is a schematic diagram of a preferred embodiment of a programmable cell according to the invention.
• Fig. 10 is a diagram of the I/O module of the programmable cell shown in Fig. 9, Fig. 11 is an example showing the locations of sampling and result read-out points at a given moment within a stripe,
• Fig. 12 is a schematic diagram of a video signal processing computer implemented with a single cellular chip according to the invention.
• Fig. 13 is a schematic diagram of a video signal processing computer implemented with two cellular chips according to the invention.
• Figs. 14 and 15 are schematic diagrams of video signal processing computers suitable for "inter frame” image processing
• Fig. 16 is a schematic diagram of the digital control unit in the video signal processing computer as shown in Figs. 12 to 15.
• the video signal processing computer has been designed for real time processing without digitalizing the interlaced or progressive scan video signals, but of course it is also suitable for processing a video signal according to any current or future standard.
• N Assuming an N number of line and frame synchronized Y in , ⁇ ... Yj n , N input video signals as well as an M number of line and frame synchronized Y 0 ut, ⁇ , --- Yout.M output video signals, the video signal processing computer 10 comprising the cellular chip according to the invention is shown in Fig. 1.
• input and output video signals are primarily understood as monochromatic luminance signals obtained from standard video signals by removing the synchron signal and separating into RGB or YUV components the composite video signals.
• the input video signals YJ ⁇ ,I ... Yin. N are generated for example by a video input module 72 to be described below, and from the output video signals Y out, ⁇ • ⁇ Y o u t .
• the standard output video signals are generated for example by a video output module 73 to be described below.
• the programmable cells of the cellular network which is implemented preferably in the form of a cellular chip, are assigned one by one to the pixels in the stripe, therefore the design of the selected frame stripe determines the design of the cellular chip. Therefore, the description below of preferred stripe designs also serves as a description of cellular chip designs.
• the preferred stripe designs disclosed in the specification have a full video line width (PAL: 768 pixels; NTSC: 640 pixels), consequently they include or cover full video lines.
• PAL 768 pixels
• NTSC 640 pixels
• Fig. 2 depicts the design of a stripe 20 with an inventive symmetric boundary condition (SBC) , which stripe comprises a useful area 21 , an upper overlapping area 22 arranged above the useful area 21 , and a lower overlapping area 23 arranged below the useful area 21.
• the useful area 21 contains an 'h' number of video lines, and the overlapping areas 22 and 23 comprise an 'o' number of video lines each.
• the analog values in the upper overlapping area 22 and in the lower overlapping area 23 are always processed, and hence when applying an appropriate overlapping, the problem caused by the boundary areas can be eliminated.
• Useful output information is only generated in the 'h' number of lines of the useful area 21 , while the 'o' number of lines in each of the upper and lower overlapping areas 22 and 23 are only processed in order to avoid boundary effects. Therefore, the result captured in these lines is not read out from the cellular chip.
• Length L s of the stripe 20 in the figure corresponds to the width of the video frame, and width W s of the stripe 20 is the sum of the number of lines in the stripe 20, i.e. in the current case o+h+o.
• the stripe 20 depicted in Fig. 3 has an asymmetric boundary condition (ABC).
• ABS asymmetric boundary condition
• the useful area 21 in addition to the useful area 21 , it is the 'o' number of lines in the lower overlapping area 23 which are processed, because the upper overlapping area has already been processed in the previous step.
• the image processing problems caused by the boundary areas can be avoided.
• Stripe 20 shown in Fig. 4 has a pre-calculated asymmetric boundary condition (PABC).
• PABC asymmetric boundary condition
• the number of lines in the overlapping area 23 may even be one half of that in the ABC case.
• the lower boundary line of the lower overlapping area 23 represents a single accumulator line 25 for each input video signal.
• the first line in the boundary area 26 is sampled in the accumulators in the accumulator lines 25, and then the next lines in the boundary area 26 are added in a weighted way to the first one. Consequently, in the PABC system, ⁇ +o 2 number of lines are involved in the calculation of the boundary condition.
• Processing time t p is the maximum time available to the video signal processing computer to process a stripe of a video frame.
• the optimal stripe design can be unambiguously determined. However, in the case of smaller series, permanent costs - for example the costs of the design work - are more dominant. Therefore, preference can be given to a cellular chip design which consists of the same type of programmable cells, i.e. a structure with a higher number of programmable cells could be optimal.
• Fig. 5 depicts the timings of a preferred embodiment having a stripe design with a symmetric boundary condition, where the programmable cells of the cellular chip in the video signal processing computer receive the analog values associated with pixels of the one or more input video signals in two input memory banks MBA and MBB, alternately for each stripe, and the analog values generated as a result of the image processing operation are loaded into two output memory banks MB ⁇ and MB ⁇ , from which the one or more output video signals can be generated by a scheduled read-out, wherein the output memory banks MB ⁇ and MBp are read out alternately stripe by stripe.
• the output memory banks MB ⁇ and MBp are read out alternately stripe by stripe.
• Fig. 5 depicts a time diagram in which time runs from top to bottom contrary to usual diagrams.
• the passing of time is indicated by consecutively received input video lines consisting of one or more video signals, which are depicted by horizontal dashed lines, and by consecutively sent output video lines consisting of one or more video signals and indicated by horizontal dashed lines.
• the time difference between two video lines, i.e. line period t ⁇ is 64 ⁇ s in the case of a PAL system, and 63.55 ⁇ s in the case of an NTSC system.
• FIG. 5 It can be counted in Fig. 5, how many lines the useful area 21 of the stripe configuration and its upper and lower overlapping areas 22 and 23 consist of.
• the figure is to be treated as depicting a general embodiment, where the useful area 21 always consists of an 'h' number of lines, and the overlapping areas 22 and 23 always comprise 'o' number of lines each.
• thin arrows t ⁇ , t * and t o depict time ranges in which the memory banks play an external I/O role
• thick arrows t p show the time ranges when in the framework of the image processing operation, the memory banks communicate with the local processors in the cells.
• the loading of the input memory banks associated with the relevant stripe and located in the programmable cells is always followed by a processing step, the duration of which is indicated by the thick arrow t p .
• the relevant input memory banks are connected to the local processors. During this time, the information stored in the input memory banks is read out and processed.
• the input memory banks are returned to data acquisition status, and they are again loaded with video lines from the beginning to the end of the stripe. Consequently, when a stripe appears repeatedly within one column, this means that the same input memory banks are repeatedly loaded.
• the operation of the embodiment shown in Fig. 5 is the following.
• the first input video line is loaded into the input memory banks MB A of the programmable cells associated with the first line of the useful area 21 in the first stripe, to make sure that the output video line obtained by processing the input video line can appear in the output image. This is because it is always the result generated in the useful area 21 , which is read out as explained above. If a video line is not loaded into the useful area 21 , this line would not appear on the output.
• the upper overlapping area 22 above the first video line can be loaded with a constant value or with copies of the first video line. The same can be done when processing the bottom of the frame, in the stripe in the area below the last video line.
• the video lines represented by the one or more input video signals are sampled in the programmable cells either by both types of the input memory banks MB A , MB B on a simultaneous basis or only by one type of those.
• the video lines stored in the other type of input memory banks are processed by the video signal processing computer. Because of the errors appearing at the lower and upper edges, only the 'h' number of lines in the middle of the total processed image, i.e. the lines corresponding to the pixels in the useful area 21 , can be used as an output, because these inner pixels are not deteriorated by errors.
• the loading of the input memory banks MBB associated with the useful area of the next stripe must be started immediately.
• the lower overlapping area 23 of the fist stripe must also be loaded into the input memory banks MB A of the programmable cells in the overlapping area, i.e. for a time both types of input memory banks are sampling.
• the input memory banks MB A associated with the first stripe are full, the input memory banks MB A are separated in a way described later from the conductor lines carrying the input video signals, and they are connected to the inner bus in the cellular chip, and hence the processing can be started.
• the loading period of a useful area of a stripe is h-tj.
• this entire h-t ⁇ period while the data is collected in the useful area of the other memory bank, may not be devoted to processing, because the processing may only commence when the lower overlapping area of the stripe is read in, and the processing is to be stopped when the sampling of the upper overlapping area of the next stripe associated with the input memory banks is started. Therefore, the time that can be dedicated to processing is the following:
• the local processors can only process the data in one type of input memory banks at a time. This means that no more time is available for processing the data of a stripe than the total cycle period divided by the number of memory banks. If a 'k' number of input memory banks are assumed in the programmable cells, the total cycle period is k-h-ti, because this is the frequency at which the system is in the same phase within one frame, therefore the time that can be devoted to processing may not exceed k h-t
• /k h-t
• the utilization factor of the local processors can be calculated in a way that the processing time is divided by the number of lines in the useful area:
• the output video lines can be read out with the following delay:
• This delay t d is indicated in the figure by an arrow with a dashed line.
• the read-out time of the video lines of one stripe from the output memory banks of the programmable cells in useful area 21 is
• FIG. 6 depicts an embodiment where in the programmable cells there are three input memory banks MB A , MB B) MB C for receiving the input video signals. The operation of this embodiment is very similar to that discussed above. Again, each video line is to be sampled into the input memory banks of programmable cells associated with a useful area 21 of a stripe.
• Symbols 'x' in Fig. 6 show in a given instant the programmable cell positions in which the input/output memory banks are currently busy in sampling the input video signals and generating the output video signals.
• input memory banks MB A and output memory banks MB ⁇ are connected to the local processors for the processing operation.
• the output video lines can be read out with the following delay:
• the number of input memory banks per programmable cell can be further increased. Assuming that a 'k' number of input memory banks are used per programmable cell, the processing time can be calculated on the basis of the following formula:
• the timing of the configurations with asymmetric and pre-calculated asymmetric boundary condition is principally the same.
• the image processing operation can be started in both cases after loading an 'o' or Oi' number of lines, respectively, in the lower overlapping area 23.
• a time equal to loading an o 2 lines is to be lapsed for generating the pre-calculated boundary condition.
• This means that the calculations using this boundary condition can only be started o 2 -t ⁇ time after the starting of image processing. All this of course also indicates that these calculations must fit into the t p - ⁇ 2 -t ⁇ time frame, which must always be positive. In such cases, consequently, the following condition must be satisfied:
• the input memory banks associated with a stripe, of the programmable cells in the useful area of the stripe are loaded with data for a period h-t
• the restriction relating to local processors is in this case the following:
• the utilization factor of the local processors is the following:
• the output video lines may be read out with a delay of
• the processing time can be calculated on the basis of the following formula:
• the restriction in association with the local processors in this case is the following:
• the utilization factor of the processors is the following:
• the number of memory banks per programmable cell can be increased further. Assuming that a 'k' number of memory banks are applied per programmable cell, the processing time can be calculated on the basis of the following formula:
• the utilization factor of the processors in the general case is: h - t, h ( 3)
• Fig. 9 shows the internal structure of a preferred embodiment of a programmable cell 30 in the cellular chip according to the invention.
• the cellular chip according to the invention can be implemented in any suitable cellular network.
• the programmable cells are located in a standard square grid and they are locally connected to each other.
• the cellular chip according to the invention is designed as a stripe, where each of the programmable cells 30 implemented on the cellular chip is assigned to a respective pixel of the stripe 20.
• the programmable cell 30 comprises a local processor 31.
• the local processor 31 comprises a programmable analog forward convolution local processor core 32.
• the latter in the given case may also be a reverse convolution core for example, depending on the application.
• the local processor core 32 as indicated by arrows 33, is connected to the other local processor cores 32 in the cellular chip, and together with them it is suitable to carry out for example convolution operations.
• the local processor 31 may also comprise in accordance with the requirements of the given application a local analog output unit (LAOU) 34 known from the CNN-UM technology, which LAOU 34 is an analog arithmetic unit, and by which in the course of the image processing operations for example various operations can be carried out using analog values stored in an 'a' number of local analog memory elements (LAM) 36 in the programmable cell 30.
• LAOU local analog output unit
• the local processor 31 may also comprise a local logic unit (LLU) 35 also known from the CNN-UM technology, which LLU 35 is designed for carrying out logic operations using logic values in a 'b' number of local logic memories (LLM) in the programmable cell 30.
• LLU local logic unit
• the application of the LAOU 34 could be necessary when processing grayscale images, and the application of the LLU 35 could be advantageous when processing black and white images.
• the LAOU 34 and/or the LLU 35, as well as the LAMs 36 and/or LLMs 37 can be omitted, i.e. these elements are not necessarily parts of the known local processor 31.
• the elements of the local processor 31 are linked to each other via an internal bus 40, which is used for forwarding analog and logic values.
• the elements are connected via controllable internal switches 41 to the internal bus 40, which internal switches 41 are preferably controlled globally, i.e. in an identical way in all programmable cells 30 of the cellular chip.
• a global conductor line 55 can be used to establish connection with the inner bus 40, which enables the loading of the programmable cells 30 with initial values.
• the programmable cells 31 also comprise an I/O module 50, which comprises local analog memory elements.
• the local analog memory elements in the I/O module 50 are arranged in memory banks, and these memory banks perform different functions at different times, as described earlier.
• the external and internal access to as well as the scheduling of the local analog memory elements in the memory banks are designed in a way that they enable a continuous receiving of one or more input video signals Y, n , ⁇ ... Y m , N and a continuous sending of one or more output video signals Y ou t, ⁇ • ⁇ Yout.M.
• all the LAMs of one and the same input memory bank and output memory bank are connected to the internal bus 40 in the programmable cells. These LAMs are preferably shown memory mapped. In such a way, when switching to a different memory bank, the addresses of the mapped LAMs will not change, which simplifies the programming of the cellular chip.
• the I/O modules 50 in the cellular chip operate synchronously with one another, i.e. at a given moment the same memory banks are activated for example to processing or sampling in each programmable cell 30.
• the I/O module 50 there are two types of memory banks as described above.
• the embodiment depicted in Fig. 10 comprises three input memory banks MB A , MB B and MBc, and two output memory banks MB ⁇ , MB ⁇ .
• the number of local analog memory elements in the input memory banks MB A , MB B and MB C called hereinafter as the input memory elements, is identical with that of the input video signals. Due to the image processing of overlapping stripes, at least two input memory banks are required, but three or more input memory banks may also be applied.
• each programmable cell 30 of the cellular chip There are the same number of input memory banks in each programmable cell 30 of the cellular chip.
• the number of local analog memory elements in the output memory banks is identical with that of the output video signals.
• the output memory elements In the programmable cells 30 corresponding to the pixels in the useful area 21 of the stripe, there are at least and preferably two output memory banks, and in the overlapping areas 22 and 23 there is preferably and at least one. The reason for this is that the LAMs in the output memory banks are used for temporary storage during the calculation, and then the final results are transferred here after performing the calculation. However, these final results are only of relevance in the useful area 21 , and not in the overlapping areas 22 and 23.
• the read-out of the output video signals begins from the output memory banks, while the other output memory banks are connected to the local processors 31.
• the final result generated in the overlapping areas 22 and 23 can be discarded, and hence the LAMs located in the output memory banks of these areas can immediately take part in processing the next frame stripe.
• the input memory banks in the cellular chip together are called the first memory unit, and the output memory banks in the cellular chip are called the second memory unit.
• the same input memory bank and output memory bank in each programmable cell 30, for example in Fig. 10 the input memory bank MB A and the output memory bank MB ⁇ are connected to the internal bus 40 of the local processor 31.
• the input memory bank connected to the internal bus 40 results in the appearance of an N number of read-write LAMs on the internal bus 40, which contain the last read stripe at the time of starting the image processing.
• the local analog memory elements in the input memory banks can also be used as temporary memories once the algorithm no longer needs the frame stripe in its original form.
• the relevant output memory bank with its M number of LAMs becomes visible on the internal bus 40. In these LAMs, there is no useful data at the time of starting the calculation.
• each local processor 31 has altogether a+N+M number of LAMs, including the LAMs 36, available for performing the calculations.
• the input and output memory banks respectively, which are currently not connected to the local processors 31, perform the sampling of the input video signals Yj n , ⁇ ... Y ⁇ n ,N and the generating of output video signals Y ou t, ⁇ ⁇ •• Y o u t , N , respectively.
• An input memory bank which is associated with a frame stripe, and which is located in a programmable cell 30 assigned to a given pixel of the frame stripe to be read in, is only connected at the moment of sampling via controllable input switches 53 to conductor lines 51 carrying the input video signals Y iri
• Fig. 11 provides an example of the location of sampling and result readout points at one moment within one stripe. The figure refers to the symmetric boundary condition arrangement shown in Fig.
• the input memory bank MB C of the programmable cell 30 in position 60 samples the upper overlapping area 22.
• the output memory bank MB ⁇ of the programmable cell 30 in position 61 the result is read out, and the input memory bank MBB of the programmable cell in position 62 samples the useful area 21.
• the input memory banks MB A and the output memory banks MBp are connected to the local processors 31.
• the video signal processing computer described above can be implemented preferably by using the stripe based cellular chip according to the invention. Through the use of such cellular chips, a high capacity video signal processing computer can be built with a minimum number of elements.
• a simple video signal processing computer as shown in Fig. 12, has a single cellular chip 70 of a stripe design according to the invention. In this case, in the stripe structured cellular chip 70, the number of programmable cells in one row is identical with the horizontal resolution of the input and output video signals.
• the standard line and frame synchronized incoming video signals are received by a known video input module 72, which removes the synchron signal from them, separates eventual composite video signals to components, and only sends the monochromatic luminance signals to the cellular chip 70.
• This N number of signals will be the input video signals Y, ⁇ , ⁇ ... Y, ⁇ , N for the cellular chip 70.
• the video input module 72 can also carry out level shifts, if necessary.
• a video output module 73 known per se and shown in the figure supplies synchron signals for the luminance signals read out from the cellular chip 70, i.e. for the output video signals Y out , ⁇ ••• Y o ut. N described above, and sends these through a standard video signal output amplifier stage.
• the video signal processing computer comprises a digital control unit 71 , which carries out the digital control and synchronizing of the whole system.
• the frame stripe can be segmented vertically into two or more sections overlapping each other in a lateral direction, where the processing of each such section can be implemented by a separate stripe structured cellular chip.
• Fig. 13 depicts the schematical structure of a video signal processing computer having two cellular chips 70a, 70b, which video signal processing computer is designed for processing frames divided into two sections. The vertical overlap between the sections must at least be the same size as the horizontal overlap between two stripes which are one below the other. By applying such an overlap, communication (data transfer) between the cellular chips 70a and 70b can be avoided.
• the video signal processing computers described above are only capable of so-called 'intra frame' processing. Through such type of image processing, it is not possible to implement numerous basic so-called inter frame image processing functions, e.g. motion detection, noise filtering by averaging in time etc. If, however, an analog memory of the size capable to store an entire frame, for example an ARAM 74 shown in Fig. 14, is introduced into the system, the implementation of these functions also becomes possible by means of the video signal processing computer according to the invention.
• an ARAM as described in the publication "A 0.5 ⁇ m CMOS CNN Analog Random Access Memory Chip for Massive Image Processing" by R. Carmone, S. Espejo, R. Dominguez-Castro, A. Rodriguez-Vazquez, T.
• the sensors of certain cameras for example long wave IR cameras and image intensified cameras have an offset error which changes in space but is constant in time.
• the video signal is digitized in the known video signal processing computer systems, and then after the compensation of the offset error, an analog signal is generated again. If the ARAM 74 shown in Fig. 14 is replaced by a non-volatile memory, the system will be able to compensate the offset error.
• other image improving and processing functions necessary for these noisy cameras can also be carried out by the same video signal processing computer.
• the design of the digital control unit 71 depicted in Figs. 12 to 15 is shown in Fig. 16.
• the control unit 71 comprises a digital processor 80, which is preferably a DSP, a microprocessor or a microcontroller, a memory 81 , a serial port handling unit 82 and bi-directional digital ports 83a, 83b, 83c.
• the elements of the control unit 71 are connected to each other via an internal digital bus 84.
• the memory 81 also comprises a RAM, and a boot unit, preferably a boot EPROM or a flash memory.
• the control unit 71 can be programmed, tuned or diagnosed via an external serial bus 85.
• These functions can be implemented preferably from a PC or a notebook connected to the external serial bus 85.
• the bi-directional digital port 83a serves for communication with the cellular chip 70 and in the given case with the ARAM 74
• the bi-directional digital port 83b serves for the communication with the video input module 72
• the bi-directional digital port 83c serves for the communication with the video output module 73.
• the speed of the digital ports 83a, 83b and 83c must be matched to the speed of the cellular chip 70, the video input module 72 and the video output module 73, and is usually approx. 15 to 50 MHz.
• the control unit 71 may also be formed as part of the cellular chip 70.
• the control unit 71 issues the commands preferably in a coded form to the cellular chip 70.
• the cellular chip 70 preferably comprises a decoder unit (not shown) for transforming commands from the control unit 71 into signals for conductor lines that operate the local processor cores 32, the internal switches 41 , the input switches 53 and the output switches 54. If the control unit 71 is designed on the cellular chip 70, the decoder unit may be omitted.
• the internal switches 41 , the input switches 53 and the output switches 54 are preferably controlled CMOS switches.
• the control unit 71 executes a program stored in the memory 81.
• This program usually includes a larger cycle, which has to be carried out in each frame stripe. Consequently, the period of such a cycle is equal to the following: the number of lines in the cellular chip 70, multiplied by the line period of the video line.
• the video signal processing computer is able to receive complete video lines, and collect, process and issue them on a continuous basis.
• the video signal processing computer can be made suitable for processing video signals coming from different modality cameras (black and white, color RGB, long wave IR, image intensified etc.), compiling them in an intelligent way, and the output may also be of multi-channel type, for example color RGB.
• a stripe structured video signal processing computer can be built.
• the SBC, ABC and PABC cellular chip arrangements described as preferred embodiments eliminate the problems arising from the lack of overlaps on the one hand, and on the other they can be built cost efficiently on a silicon basis.
• the joint 'inter frame' processing of consecutive frames as described above is also possible by stripe structured cellular chips, which also allow to carry out image processing tasks in time, e.g. motion detection.
• the video signal processing computer based on the cellular chip according to the invention may also be of a PCMCI size, and its computational output could be several hundred times higher than the current highest capacity digital signal processing systems (e.g. TMS320C6x).
• the video signal processing computer, cellular chip and method according to the invention may be applied according to the description above not only in the cellular network used in the described embodiments, but in any suitable cellular network, for example in the convolution neural network described in the introduction, in a network having general analog processor cores or in a cellular network implemented with a two-dimensional resistive grid.

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Image Processing (AREA)
• Television Systems (AREA)

Priority Applications (1)

Applications Claiming Priority (2)

Publications (2)

Family

ID=89978763

Family Applications (1)

Country Status (3)

Cited By (1)

Family Cites Families (1)

• 2000
  • 2000-11-15 HU HU0004500A patent/HUP0004500A2/hu unknown
• 2001
  • 2001-11-15 AU AU2002218423A patent/AU2002218423A1/en not_active Abandoned
  • 2001-11-15 WO PCT/HU2001/000113 patent/WO2002041246A2/en not_active Application Discontinuation

Also Published As

Similar Documents

Legal Events