WO2001041448A1 - Systeme d'acquisition de donnees, oeil artificiel, dispositif de vision, capteur d'images, et appareil associe - Google Patents
Systeme d'acquisition de donnees, oeil artificiel, dispositif de vision, capteur d'images, et appareil associe Download PDFInfo
- Publication number
- WO2001041448A1 WO2001041448A1 PCT/JP2000/008334 JP0008334W WO0141448A1 WO 2001041448 A1 WO2001041448 A1 WO 2001041448A1 JP 0008334 W JP0008334 W JP 0008334W WO 0141448 A1 WO0141448 A1 WO 0141448A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- image
- array operation
- operation unit
- moving
- interlocking
- Prior art date
Links
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S3/00—Direction-finders for determining the direction from which infrasonic, sonic, ultrasonic, or electromagnetic waves, or particle emission, not having a directional significance, are being received
- G01S3/78—Direction-finders for determining the direction from which infrasonic, sonic, ultrasonic, or electromagnetic waves, or particle emission, not having a directional significance, are being received using electromagnetic waves other than radio waves
- G01S3/782—Systems for determining direction or deviation from predetermined direction
- G01S3/785—Systems for determining direction or deviation from predetermined direction using adjustment of orientation of directivity characteristics of a detector or detector system to give a desired condition of signal derived from that detector or detector system
- G01S3/786—Systems for determining direction or deviation from predetermined direction using adjustment of orientation of directivity characteristics of a detector or detector system to give a desired condition of signal derived from that detector or detector system the desired condition being maintained automatically
- G01S3/7864—T.V. type tracking systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T1/00—General purpose image data processing
- G06T1/0007—Image acquisition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/10—Image acquisition
- G06V10/12—Details of acquisition arrangements; Constructional details thereof
- G06V10/14—Optical characteristics of the device performing the acquisition or on the illumination arrangements
- G06V10/147—Details of sensors, e.g. sensor lenses
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/24—Aligning, centring, orientation detection or correction of the image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/32—Normalisation of the pattern dimensions
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/06—Recognition of objects for industrial automation
Definitions
- the present invention relates to an information collection system using an artificial eyeball that tracks an object and generates an image showing only the object, and more particularly, a moving force camera having a mechanism for panning, tilting, rolling, and zooming.
- the visual device generates a normalized image, type, position, size, and number of the object from the moving image, and the visual device controls the movement measurable so that the object is photographed largely in the center of the moving image.
- the present invention relates to a device for transmitting an image of an object area separated from a background to a user terminal via a wired or wireless communication network.
- a spherical multi-degree-of-freedom mode is effective (for example, see Japanese Patent Laid-Open No. 6-210585; Nikkan Kogyo Shimbun, January 15, 2001 Day, see page 6).
- a multi-degree-of-freedom motor by using an ultrasonic motor instead of the conventional motor, the mechanism and control of the multi-degree-of-freedom motor become simpler. Shigeru Shigeru, "Development of Spherical Ultrasonic Motor", Tripologist, Vol. 40, No. 8, pp. 6 27- 631, 1995; Japanese Patent Application Laid-Open No. 7-80793; Japanese Patent Application Laid-Open No.
- Sen sory P ro c ess e ng E l emen t) Vision chip (Takashi Komuro, Shinsuke Suzuki, Takashi Ishii, Masatoshi Ishikawa, "Design of massively parallel and ultra-high-speed vision chips using general-purpose processing elements", Electronic Information and Communication) Journal Transactions, Vo 1. J 81-D-I, No. 2, pp. 70-76, 1998), Artificial Retina LSI (JP-A-6-139361, JP-A-11-17)
- GVP P Generic Visua l Pe rcepti on Processor
- Data processor architecture or inconsistent image processing architecture and algorithm makes it difficult to implement multiple image processing with a single vision chip.
- S 3 PE is connected to the moving camera also on it can be carried out tracking of fast moving objects, S 3 PE is to be affected to the speed of the shooting environment and moving objects, for stationary object and a plurality objects Image processing must be performed using another device.
- a wireless image communication device that transmits an image of an object extracted using an artificial retinal LSI has been developed, but since this wireless image communication device does not have a function for controlling a mobile camera, wireless image communication is not possible. The use of the device is limited to a monitoring device using a fixed camera.
- LSIs become more and more highly integrated, so digital circuits conventionally mounted on separate LSIs can be easily mounted on a single LSI.
- LSI designers Become like Technologies for stacking more chips (eg, Nikkei Microdevices, June 2000, pages 62-79; Nikkei Microdevices, June 2000, pages 157-164; Nikkei Microdevices, June 2000) No., page 176), it has become possible for LSI designers to design electronic circuits more easily than with conventional 3D LSI technology.
- the use of 3D LSI technology also allows LSI designers to shorten the wiring length, so that LSI designers can easily design LSIs.
- the inventor has developed a visual device as a device for controlling a moving camera and performing image processing (for example, see Japanese Patent Application No. 11-253634, International Publication No. WO 00/16259, Japanese Patent Application No. 2000-269500).
- This vision device searches for an object and performs image processing of the object by controlling the pan, tilt, roll, and zoom mechanisms of the moving camera.
- Most of the image processing performed by the visual device is local processing, and these local processings are executed in parallel by an array operation unit arranged in a two-dimensional lattice.
- each array operation unit is designed to be able to asynchronously communicate with an adjacent array operation unit.
- an image sensor that combines a visual device and an imaging device such as a photodiode based on the currently available large-scale integrated circuit technology and three-dimensional large-scale integrated circuit technology. It is expected to be possible. By mounting this image sensor on a multi-degree-of-freedom motor, it is expected that it is possible to manufacture a moving force lens controlled by a visual device.
- the moving force camera equipped with a multi-degree-of-freedom motor controlled by the vision device searches for the object, and the normalized image, type, position, size and position change of the captured object are directly reflected in the brain. Is expected to be able to produce artificial eyeballs that can be transmitted to
- the moving device mounted on the multi-degree-of-freedom camera controlled by the visual device captures the object, and the visual device determines the type of the captured object. It is expected that such a moving image can be transmitted to a remote place.
- the user can control the visual device by voice, and can know the type of the object identified by the visual device by voice. Be expected.
- the present invention described in the claims uses a visual device for controlling a moving camera, and transmits a moving image showing an object to a user terminal at a remote place even if the mobile camera is not constantly controlled from a remote place.
- the purpose is to: Another object is to mount the visual device and the image sensor on the LSI and incorporate it into the mobility camera.
- the invention according to claim 1 is an information collection system including one or more artificial eyes and one or more user terminals connected to a communication network, wherein the artificial eyes include a mobile camera, a visual device, and a communication device.
- the moving camera has at least one of pan, tilt, roll, and zoom mechanisms; Acquiring the image of the object; generating the type, position, and number of objects in the digital image; controlling the mechanism for performing the pan, the tilt, the roll, and the zoom;
- the communication device transmits at least one of the digital image, the type, the position, and the number to the user terminal, and corresponds to the pan, the tilt, the roll, and the zoom.
- the user terminal includes a central controller, a display, and a control button, and the central controller is the digital image, the type, Receiving at least one of the location and the number from the communication device; and Displaying at least one of the type, the position and the number; the user terminal generating the control command assigned to the control button; and the central controller communicating the control command in the communication.
- An information collection system characterized by transmitting to an apparatus.
- the artificial eyeball includes the pan (horizontal movement), the tilt (vertical movement), the roll (twisting), and the zoom (telephoto). The object can be photographed at an appropriate size at the center of the moving image.
- the visual device generates the type, the position, and the number of the object from the digital image, and controls the moving camera according to the position.
- the communication network includes USB (Universa 1 Serial Bus), Ethernet, ISDN (Integrated Services Digital Network), xDSL (Digita 1 Subscriber Line), and ATM (Asynchr and a wireless technology such as IMT-2000 (DS-CDMA and MC-CDMA) and B1ue tooth, and the communication device is the communication network.
- a computer, a mobile phone, a portable information terminal, a navigation system, or the like having the display and the control button is used.
- the central control device has a communication function to the communication network, outputs the type, the position, and the number to the display, and further includes the control command. Is input from the control button.
- the visual device since the visual device always controls the moving camera, the user does not need to finely control the moving camera. Further, in the present invention, since the visual device recognizes the type of the object captured by the mobile camera, the artificial eyeball can transmit information on only the object that the user wants to know to the user terminal. . That is, since the user terminal can output the information to the display only when the object that the user wants to know is photographed, the user does not need to always pay attention to the display. However, the artificial eye does not need to always occupy the communication network. Therefore, the present invention releases the user from the control of the moving camera, so that various problems relating to the photographing and communication of the object are suitably solved.
- the invention according to claim 2 is an information collection system including one or more artificial eyes, a database, and one or more user terminals connected to a communication network, wherein the artificial eyes are a mobile camera, a visual device, and a communication device.
- a moving device having at least one of a mechanism for panning, tilting, rolling, and zooming, wherein the visual device acquires a digital image from a moving image captured by the moving camera; Generating a type, a position, and a number of objects in the digital image, wherein the communication device transmits at least one of the digital image, the type, the position, and the number to the database.
- the database stores the digital image, the type, the position, and the number.
- the user terminal includes a central control unit, a display and a control button, and the central control unit transmits at least one of the digital image, the type, the position, the number, and the system.
- the central control unit transmits at least one of the digital image, the type, the position, the number, and the system.
- the artificial eyeball By using the moving camera having at least one of the mechanisms for performing the pan (horizontal movement), the tilt (vertical movement), the roll (twisting), and the zoom (telephoto), The image can be taken at an appropriate size in the center of the image.
- the visual device generates the type, the position, and the number of the object from the digital image, and controls the moving camera according to the position.
- the communication network includes USB (Universal Serial Bus), Ethernet, ISDN (Integrated Services Digital Network), xDSL (Digital Subscriber Line), and ATM (Asynchronous Online). And a wireless technology such as IMT-2000 (DS-CDMA and MC-CDMA) and B1uetooth, wherein the communication device is connected to the communication network by the digital communication device.
- the database calculates the statistical data from the digital image, the type, the position, and the number using mathematical methods such as mean, variance, principal component analysis, finite element method, linear programming, and Bayesian estimation.
- a relational database and an object-oriented database having a database description language such as SQL (Structure Query Language) are used.
- a combination screen, a mobile phone, a portable information terminal, a navigation system, or the like having the display and the control button is used.
- the central control device has a communication function to the communication network, outputs the type, the position, the number, and the statistical data to the display, and further inputs the search key from the control button. .
- the database stores the digital image, the type, the position, and the number over a long period of time, so that the user can obtain the search key for the type, area, location, season, date, and the like. Can be obtained. Therefore, according to the present invention, the various problems relating to the counting and prediction of the object are suitably solved.
- the invention according to claim 3 is the information collection system according to claim 1 or 2, wherein the user terminal includes at least one of a voice synthesizer including a speaker and a voice recognition device including a microphone, and the voice synthesizer. Sounds the type, the position, the number, and the statistical data. A voice signal; and the voice recognition device converting the voice collected by the microphone into the control command and the search key.
- a computer equipped with the display, the speaker, the control buttons, and the microphone a mobile phone, a portable information terminal, a navigation system, and the like are used.
- the central control device has a communication function to the communication network, outputs the type, the position, the number, and the statistical data to the display and the voice synthesizer, and further includes the control command and the search key. Input from the control button and the voice recognition device.
- the voice synthesizer converts the type, the position, the number, and the statistical data into the voice signal and outputs the voice signal to the speaker.
- the voice recognition device converts the voice collected by the microphone into the control command and the search key.
- the invention according to claim 4 is an artificial eyeball provided with a moving camera and a visual device, wherein the moving camera has at least one of a mechanism for performing pan, tilt, roll, and zoom, and the visual device includes: Means for acquiring a digital image from a moving image captured by the moving camera, means for identifying the type of an object in the digital image, means for detecting the position, size and inclination of the object, and the number of the objects A means for controlling the mechanism for performing the pan, the tilt, the roll, and the zoom, and a means for outputting the digital image, the type, the position, and the number. It is an artificial eyeball that is a feature.
- the moving camera uses a combination of a plurality of cylinders and a single-axis motor or a multi-degree-of-freedom motor to provide the pan (horizontal movement), the tilt (vertical movement), the roll (twisting), and the zoom (telephoto).
- the above-described mechanism for performing the above can be realized. Therefore, the moving camera can photograph the object at the center of the moving image and at a size necessary for recognizing the object.
- the visual device performs local parallel image processing such as edge information generation means, edge information formation means, object Z background separation means, position and size detection means, position Z size inclination detection means, and pattern matching means.
- the visual device controls the pan, the tilt, the roll, and the zoom of the moving camera using the position, and controls the pan, the tilt, and the roll using a control command input from outside. And controlling the zoom.
- the moving camera can be pointed at the object even when the position of the object is not known in advance, and when the object is moving. Therefore, the present invention relieves the user of the present invention from the control of the moving camera, so that various problems relating to the photographing of the object are suitably solved.
- the invention according to claim 5 is the artificial eyeball according to claim 4, further comprising a communication device, wherein the communication device includes: a unit configured to compress the digital image; and the digital image compressed through a communication network.
- Means for transmitting the type, the position, and the number, the artificial eyeball comprising:
- the communication device transmits the compressed digital image, the type, the position, and the number to the communication network, and receives the control command from the communication network.
- the communication device uses the type identified by the visual device to screen the digital image showing the object of the specific type. Further, the communication device may transmit the selected digital image by masking a region other than an area occupied by the object.
- the communication device may be an MPEG (moving)
- the data amount of the digital image can also be reduced by using an image compression technique such as Picture ExpoRentsGroup) and JPEG (JointPhotogrApIcCodinG ExpErtsGroup).
- the present invention can transmit only the digital image in which the object is photographed to the communication network. Therefore, the present invention can reduce the load on the communication network and inform the user of the present invention of the presence / absence of the object, so that various problems relating to imaging and communication of the object can be suitably solved.
- the invention of claim 6 is the artificial eyeball according to claim 4, comprising a stimulus generator, an electrode array terminal, a connector, and a battery, wherein the stimulus generator is a normalized image of the object generated by the visual device. Converting the type, the position, the size, and the change in the position into electrical stimulation; An artificial eyeball, wherein the electrode array terminal transmits the electrical stimulation to the brain via the connector.
- the stimulus generator uses a DZA conversion circuit to determine the normalized image, the type, the position, the size, and the change in the position of the object to such an extent that nerve cells can appropriately excite. Convert to the electrical stimulus.
- Platinum is used for the electrodes of the electrode array terminals, and the electrode array terminals are embedded in the occipital lobe, temporal lobe, parietal lobe, frontal lobe, and the like of the brain.
- a rechargeable battery such as a lithium ion battery and a battery having a long discharge time such as a fuel cell are used.
- the present invention allows the user to recognize the object in a short time and with a high probability, so that various problems relating to artificial vision are suitably solved.
- the moving camera includes an image sensor in which a plurality of image sensors, a plurality of AZD conversion circuits, and the visual device are integrated. It is an artificial eyeball that is a feature.
- the imaging device is realized using a photodiode and a CCD (charge coupled device) or a CMOS (complementary metal oxide semiconductor).
- the imaging element, the AZD conversion circuit, and the visual device are integrated in the image sensor using LSI (Large Scale Integrated Circuit) technology and three-dimensional LSI technology.
- the AZD conversion circuit and the visual device include an electronic circuit for transmitting the digital image. It can be simplified. Therefore, according to the present invention, the number of parts can be reduced, and the size of the artificial eyeball can be reduced, so that various problems relating to the production of the artificial eyeball can be suitably solved.
- a visual device comprising: means for calculating an inclination angle from all of the band pixel values of the information image; and means for outputting the band pixel value and the inclination angle of the overlapping information image.
- a value image is input as appropriate in pixel units, and the processes from the calculation of the center of gravity to the output of the overlapping information image and the tilt angle are sequentially performed, and the process is repeated until the binary image is no longer input.
- the array operation units can be operated in parallel, various problems relating to the detection of position, size, and inclination can be suitably solved.
- each of the array operation units arranged in a lattice comprises a processor, a memory, and a plurality of controllers, and the controller is a source of the data transmitted from the array operation unit that is transferred between the array operation units.
- a visual device characterized by communicating a plurality of the data simultaneously with a unit.
- the processor controls the plurality of controllers, so that the array operation unit efficiently transfers the plurality of data in the memory to the memory of the adjacent array operation unit.
- the present invention can transfer a large amount of the data in a short time, various problems relating to the performance of the visual device that executes various kinds of local parallel image processing are suitably solved.
- a tenth aspect of the present invention is the visual processing device according to the ninth aspect, wherein all of the controllers are stacked so that the signal lines between the array operation units do not intersect.
- LSI Large Scale Integrated Circuit
- designers can stack all the controllers using the three-dimensional LSI technology, so that each of the stacked controllers is arranged in a grid. At this time, by connecting the adjacent controllers mounted on the same height layer with the signal lines, the signal lines between the controllers do not intersect.
- the wiring patterns of the signal lines in all the layers can be made the same, and the wiring length of the signal lines can be shortened. It is preferably solved.
- the invention according to claim 11 is a visual device configured by a plurality of virtual array operation units arranged in a lattice, wherein the virtual array operation unit includes a processor, a memory, and a plurality of controllers, and the controller is Each of the virtual array operation units includes a front input register group, the front input register group includes one or more front input registers and one or more counters, and the front input register is a shift register.
- the processor executes a program of a plurality of array operation units arranged in a grid
- the controller communicates with the controller of the adjacent virtual array operation unit
- the shift register includes Inputting and storing a plurality of result data arranged in a lattice in order; and And counting the evening number of a visual device, wherein the processor is characterized, and to enter by selecting one of the plurality of the result data from said shift register.
- the controller is preferably arranged such that the array operation units are arranged in a lattice. More than the maximum number of vertical and horizontal numbers.
- the previous input register includes the shift register having a storage capacity enough to store the result data output from all the array operation units, and the counter, and the shift register is arranged in a predetermined order. The input result data is stored in order.
- the order of the result data can be maintained.
- An initial value of the power calculation unit is set to the number of the array operation units, and each time the array operation unit outputs the result data, the counts are counted in descending order. Therefore, when all the result data are output from the array operation unit, the value of the count becomes zero.
- the controller sets the previous input delivery to 1. Thereafter, when the processor reads all the result data during the shift register, the processor sets the previous input delivery status to zero.
- a plurality of the array operation units can be implemented as a program in one virtual array operation unit. Therefore, according to the present invention, since many array operation units can be realized with a small amount of hardware, various problems relating to the manufacture of the visual device can be suitably solved.
- the invention according to claim 12 is an image sensor in which a functional block including at least one image sensor, at least one AZD conversion circuit, and at least one array operation unit is arranged in a two-dimensional lattice.
- the functional block is configured so that the image sensor and the AZD conversion circuit are connected in a one-to-one correspondence, and that all of the array operation units are at least among all the A / D conversion circuits and all of the array operation units.
- the imaging device An image sensor wherein the array operation unit performs image processing on a captured moving image.
- the imaging device is realized using a photodiode and a CCD (charge coupled device) or a CMOS (complementary metal oxide semiconductor).
- the image sensor, the AZD conversion circuit, and the array operation unit are integrated in the image sensor using LSI (Large Scale Integrated Circuit) technology and three-dimensional LSI technology.
- LSI Large Scale Integrated Circuit
- the designer of the image sensor can stack the array operation unit in the functional block, so that the designer can connect the signal line of the array operation unit to the function adjacent to the array unit. There is no need to cross between blocks.
- the AZD conversion circuit and the array operation unit include an electronic circuit for transmitting the digital signal.
- the digital signal can be transmitted at a high speed.
- the array operation unit since the array operation unit communicates asynchronously with the array operation unit of the adjacent function block, regardless of the number of the function blocks of the image sensor, the clock signal is generated in the function block. Only the array operation units need to be synchronized. Therefore, according to the present invention, the number of components and the amount of wiring can be reduced, and the image sensor can be operated at high speed, so that various problems relating to the design and manufacture of the image sensor can be suitably solved.
- the invention according to claim 13 is a functional block having a plurality of image sensors, a plurality of A / D conversion circuits, at least one parallel-serial conversion circuit, and at least one virtual array operation unit.
- An image sensor arranged in a child shape wherein the functional block includes: a one-to-one connection between the image sensor and the AZD conversion circuit; and a plurality of the AZD converter circuits for each band of the image sensor.
- the parallel Z-serial conversion circuit is connected, and all the virtual array operation units input digital signals from at least one of all the parallel-serial conversion circuits and all the virtual array operation units. Wherein all of the virtual array operation units receive a clock signal, and the virtual array operation units of each of the adjacent functional blocks are connected to each other.
- the one-to-one connection allows the virtual array operation unit to perform image processing on a moving image captured by the image sensor.
- Image sensor The imaging device is realized using a CCD (charge coupled device) and a CMOS (complementary metal oxide semiconductor).
- the image sensor, the AZD conversion circuit, the parallel-serial conversion circuit, and the virtual array operation unit are integrated on the image sensor using LSI (Large Scale Integrated Circuit) technology and three-dimensional LSI technology.
- LSI Large Scale Integrated Circuit
- the designer of the image sensor can stack the virtual array operation units in the function block, so that the designer can connect the signal lines of the virtual array operation unit adjacent to the virtual array operation unit. There is no need to cross between functional blocks.
- the parallel-serial conversion circuit and the virtual array operation unit transmit the digital signal. Therefore, the electronic circuit can be simplified, and the digital signal can be transmitted at high speed.
- the virtual array operation unit communicates asynchronously with the virtual array operation unit of the adjacent function block, so that the clock signal is within the function block regardless of the number of the function blocks of the image sensor. Only the virtual array operation unit described above needs to be synchronized. Therefore, according to the present invention, the number of components and the amount of wiring can be reduced, and the image sensor can be operated at high speed. Therefore, various problems relating to the design and manufacture of the image sensor can be suitably solved.
- the invention according to claim 14 is the image sensor according to claim 12 or 13, wherein the function block includes: a synchronous oscillation circuit that generates the clock signal; and an interlocking counter that counts the number of pulses of the clock signal.
- a synchronous oscillating circuit arranged in a two-dimensional lattice, and connected to an adjacent synchronous oscillating circuit to form a first interlocking device; and And a second interlocking device configured by connecting the arrayed interlocking power terminals to the adjacent interlocking power counters, thereby providing an image sensor.
- the synchronous oscillation circuit an astable multivibrator and a crystal oscillator are used.
- the present invention reduces the problem of fan-out and clock skew, because the click signal is provided only within the functional block. can do.
- a binary counter is used for the interlocking counter.
- the interlocking counters are interlocked with the adjacent interlocking counters, so that the number of pulses counted by all the interlocking counters is the same.
- the interlocking counter can gate the input of the reset signal, so that all of the functional blocks can be reset simultaneously.
- the invention can reduce the power consumption and operate the image sensor at a high speed, so that various problems relating to the design and manufacture of the image sensor can be suitably solved.
- the invention according to claim 15 is an interlocking device comprising interlocking counters arranged in a square lattice and a hexagonal lattice, wherein the interlocking counters are a synchronous counter and an end value judging device.
- the interlocking device is characterized in that by stopping the counting, the counts of all the interlocking counters are made uniform.
- the interlocking counter generates the interlocking signal by inverting a ripple carry-out signal of the synchronous counter using the end value determination NOT gate.
- the counting NOR gate generates an enable signal from the interlocking signal.
- the interlocking signal is at the High level, the synchronous counting is performed and the interlocking latch circuit is reset. .
- the synchronous counter temporarily stops.
- the interlocking NOR gate integrates the external interlocking signal
- the interlocking latch circuit stores the integration result.
- the interlocking latch circuit controls the power supply N / R gate according to the integration result to start the operation of the synchronous power supply.
- the synchronous counter starts counting in synchronization with the externally input interlocking signal.
- the count numbers of all the interlocked counters can be matched.
- the higher the frequency of the clock signal the more the propagation delay time of the clock signal becomes a problem.
- the interlocking counters are dispersedly arranged in the LSI, the count numbers of all the interlocking counters can be made to match. Therefore, various problems related to the synchronization of the LSI It is preferably solved.
- the invention according to claim 16 is the interlocking device according to claim 15, wherein the interlocking counter includes an end value determining NOR gate that determines an end value of the count number, and the synchronizing counter operates in synchronization.
- An interlocking device comprising: a clearing unit; and a synchronous loading unit. If the synchronous counting is an up counting, an output of the up counting is connected to the end value judging NOR gate for judging a maximum value of the count number, and an output of the end value judging NOR gate is output. Is connected to the NOT gate for the end value determination.
- the output of the down counting is connected to the NOR gate for determining the end value to determine the minimum value of the count number, and the output of the NOR gate for determining the closing value is output.
- the output of the NOT gate for end value determination is connected to a clear terminal for controlling the clear means and a load terminal for controlling the load means, in addition to the linkage latch circuit and the count NOR gate. .
- the loading means inputs the initial value of the count number.
- the synchronous power counter can set an arbitrary initial value for the number of power points, so that the synchronous power counter can count from the initial value to the end value.
- the cycle of the interlocking counter can be freely designed, so that the timing of the electronic circuit can be easily adjusted. Therefore, the problems related to the synchronization of the LSI are suitably solved.
- the invention according to claim 17 is the interlocking device according to claim 15 or 16, wherein at least one of the interlocking counters includes a signal distribution decoder, a signal distribution latch circuit, and a signal distribution NOR.
- An interlocking device comprising: a gate; and a signal distribution flip-flop circuit, wherein the input signal is formed to be a positive multiple of the period of the clock signal.
- the signal distribution decoder can simultaneously output the same evening timing signal. Further, the time between the clock start time and the clock end time designated by two different ones of the evening timing signals is a time that is a positive multiple of the clock signal period.
- the input signal of the clock start time is determined by the signal distribution NOR gate.
- the signal is stored in the signal distribution flip-flop circuit until the time ends.
- the signal distribution flip-flop circuit is reset by the signal distribution decoder. Accordingly, the present invention can distribute the input signal synchronized with the clock signal to an arbitrary place of the LSI by only considering the longest propagation delay time of the input signal. Therefore, the problems relating to the design of the LSI are suitably solved.
- the invention according to claim 18 is an interlocking device comprising synchronous oscillation circuits arranged in a square lattice and a hexagonal lattice, wherein the synchronous oscillation circuit comprises two oscillation NOR gates, And two oscillation capacitors, two oscillation capacitors, two synchronization latch circuits, two synchronization NOR gates, and an initialization NOR gate, and output from the oscillation capacitor.
- An interlocking device wherein the phases of the clock signals of all the synchronous oscillation circuits are aligned by shifting the phase of the clock signal of the synchronous oscillation circuit adjacent to the synchronization signal.
- the oscillation portion of the synchronous oscillation circuit In the oscillation portion of the synchronous oscillation circuit, two of the oscillation NOR gates and two of the oscillation capacitors are connected in a ring, and the output and the input of the oscillation NOR gate are connected using the oscillation resistor. Is connected. As a result, the oscillating portion self-oscillates, and the clock signal is output from the oscillating NOR gate.
- the oscillation part is classified into two parts, A side and B side.
- the synchronous part of the synchronous oscillation circuit is classified into two parts, the A side and the B side, corresponding to the oscillation part.
- the synchronization latch circuit On both the A side and the B side of the synchronization portion, after the synchronization NOR gate integrates the synchronization signal input from the outside, the synchronization latch circuit stores the integration result.
- the synchronization latch circuit controls the oscillation NOR gate in accordance with the integration result, and sets the phase and the cycle of the synchronization signal of the synchronous oscillation circuit to the phase and the cycle of the synchronization signal input from the outside. Adjust to When the synchronous signal of the synchronous oscillation circuit and the externally input synchronous signal are irregular, such as when power is turned on, the initialization NOR gate outputs the output of the oscillation NOR gate on the A side. The above-mentioned phase of the synchronizing signal is determined by fixing to the Low level. According to the present invention, by connecting a plurality of the synchronous oscillation circuits to each other, the synchronization signals of all the synchronous oscillation circuits can be synchronized.
- An invention according to claim 19 is an interlocking device characterized in that the synchronous oscillation circuit according to claim 18 is provided with two input resistors.
- the NOR gate is made of CMOS (Complementary Metal Oxide Semiconductor)
- CMOS Complementary Metal Oxide Semiconductor
- the present invention protects the CMOS of the NOR gate by adding the input resistance to each of the oscillation portions on the A side and the B side.
- the resistance value of the input resistor increases, the current flowing through the NOR gate decreases, so that the time constant determined by the oscillation resistor and the oscillation capacitor becomes more accurate, and all the synchronous oscillations
- the cycle of the quick signal of the circuit is easily aligned. Therefore, various problems relating to synchronization of the digital circuit using the CMOS are preferably solved.
- FIG. 1 is an explanatory diagram of an information collection system that transmits information generated by an artificial eyeball to a user terminal.
- FIG. 2 is an explanatory diagram of a mobile phone equipped with an artificial eyeball.
- FIG. 3 is an explanatory diagram when a mobile camera and a visual device are installed at the rear of a car.
- FIG. 4 is an explanatory view when a mobile camera and a visual device are installed at the rear of the wheelchair.
- FIG. 5 is an explanatory diagram of an information collection system in which a monitoring terminal constantly monitors information generated by an artificial eyeball and transmits the information to a user terminal as needed.
- FIG. 6 is an explanatory diagram of an information collection system in which information generated by an artificial eyeball is stored in a database, and a user terminal searches the database as appropriate.
- FIG. 7 is an explanatory diagram of environmental data including the type and position of an object collected from a visual device that controls a moving camera, the number of objects, and a record of a database that records digital images.
- FIG. 8 is an explanatory diagram of an information collection system in which information generated by an artificial eyeball is stored in a database, a monitoring terminal searches the database as appropriate, and transmits the information to a user terminal as needed.
- FIG. 9 is a block diagram of an information collection system composed of one artificial eye and one user terminal.
- FIG. 10 is an explanatory diagram of a moving camera, a visual device, and an artificial eyeball having a communication function.
- FIG. 11 is a block diagram of a communication device mounted on an artificial eyeball.
- FIG. 12 is a flowchart showing the algorithm of the image compression means of the present embodiment.
- FIG. 13 is an explanatory diagram of an artificial eye including a moving camera, a visual device, a stimulus generator, an electrode array terminal, a connector, and a battery.
- Fig. 14 is a block diagram of a visual device that controls a moving camera to search for an object and counts the number of objects.
- FIG. 15 is an explanatory diagram of image coordinates when the moving camera is rotated around the lens optical axis.
- FIG. 16 is a block diagram of an array operation unit arranged in a lattice.
- FIG. 17 is an explanatory diagram of numbers near 8 that are ordered in a counterclockwise direction.
- FIG. 18 is a flowchart showing the algorithm of the image storage means of the present embodiment.
- FIG. 19 is an explanatory diagram when color information is generated for a specific color in a digital image.
- FIG. 20 is a flowchart showing an algorithm of the color information generating means of the present embodiment.
- FIG. 21 is an explanatory diagram of a case where rough edge information is generated using a digital image.
- FIG. 22 is a flowchart showing an algorithm of the edge information generating means of the present embodiment.
- Figure 23 shows the digital
- FIG. 9 is an explanatory diagram in a case where rough edge information is formed into formed edge information using a virtual image.
- FIG. 24 is a flowchart showing an algorithm of the edge information forming means of the present embodiment.
- FIG. 25 is a diagram illustrating a case in which low-resolution coarse edge information generated from a low-resolution digital image is formed as formed edge information.
- FIG. 26 is an explanatory diagram of a case where a region of low-resolution coarse edge information generated from a low-resolution digital image is cut out and then formed as formed edge information.
- FIG. 27 is an explanatory diagram for detecting the position and size of an object in an edge information image.
- FIG. 28 is a flowchart showing an algorithm of the position Z size detecting means of the present embodiment.
- FIG. 29 is an explanatory diagram for detecting the position and size of an object in an object region image.
- FIG. 30 is an explanatory diagram for detecting the position, size, and inclination of an object in an edge information image.
- FIG. 31 is a flowchart showing an algorithm of the position / size / inclination detecting device of the present embodiment.
- FIG. 32 is an explanatory diagram for detecting the position, size, and inclination of an object in the object region image.
- FIG. 33 is an explanatory diagram of an example of the migration source duplication information.
- FIG. 34 is an explanatory diagram for detecting the position, size, and inclination of a texture in a color information image.
- FIG. 35 is an explanatory diagram in the case of normalizing the separated object region of the digital image.
- FIG. 36 is a flowchart showing an algorithm of the area normalizing means of the present embodiment.
- FIG. 37 is an explanatory diagram of a case where a mask image is generated from a digital image using a separated object region image.
- FIG. 38 is a flowchart showing the algorithm of the mask means of the present embodiment.
- FIG. 39 is a flowchart showing the algorithm of the image holding means of the present embodiment.
- FIG. 40 is an explanatory diagram of a case where pattern matching is performed from a template image to a normalized image.
- FIG. 41 is a flowchart showing the algorithm of the pattern matching means of the present embodiment.
- FIG. 42 is an explanatory diagram showing a state where edge information of a triangle is separated into an inner area and an outer area of the triangle.
- FIG. 43 is a flowchart showing an algorithm of the object / background separation means of the present embodiment.
- FIG. 44 is an explanatory diagram showing a state where edge information of a triangle in a broken line state is separated into an inner region and an outer region of the broken line triangle.
- FIG. 45 is an explanatory diagram showing a state where edge information obtained by overlapping two triangles is separated into two triangle regions and a background region.
- Figure 46 shows two pieces of edge information in the dashed state when two circular object areas are superimposed.
- FIG. 4 is an explanatory diagram showing a state where the image is separated into a circular area and a background area.
- FIG. 47 is a block diagram of the internal structure of the array operation unit.
- FIG. 48 is a block diagram of the controller.
- FIG. 49 is an explanatory diagram showing input / output signals of the flag decoder.
- FIG. 49 is an explanatory diagram showing input / output signals of the flag decoder.
- FIG. 50 is an explanatory diagram showing input / output signals of the flag chain encoder.
- FIG. 51 is a flowchart showing an algorithm by which a processor transmits data to an adjacent array operation unit via a controller.
- FIG. 52 is a flowchart showing an algorithm in which the controller receives data from an adjacent array operation unit.
- FIG. 53 is a flowchart showing an algorithm by which the processor receives data from the upper input register.
- FIG. 54 is an explanatory diagram of an array operation unit when five controllers are arranged on a plane.
- FIG. 55 is an explanatory diagram of an array operation unit when five controllers are stacked.
- FIG. 56 is an explanatory diagram in a case where nine array operation units are regarded as one virtual array operation unit.
- FIG. 51 is a flowchart showing an algorithm by which a processor transmits data to an adjacent array operation unit via a controller.
- FIG. 52 is a flowchart showing an algorithm in which the controller receives data from an adjacent array operation unit.
- FIG. 57 is an explanatory diagram in the case where the allocation numbers for the controllers are assigned to nine array operation units included in one virtual array operation unit.
- FIG. 58 is an explanatory diagram of a shift register that sequentially stores nine result data output from nine array operation units.
- FIG. 59 is a block diagram of a function column of an image sensor for an artificial eye in which each processing is executed by a separate array operation unit.
- FIG. 60 is a block diagram of an image sensor in which functional columns are arranged in a two-dimensional lattice.
- FIG. 61 is a block diagram of a function column of an image sensor for an artificial eye in which all processing is executed by one array operation unit.
- FIG. 62 is a block diagram of a function column of an image sensor for an artificial eyeball in which all processes are executed by one virtual array operation unit.
- FIG. 63 is a block diagram of the image sensor when the function columns are mounted perpendicular to the mounting surface of the large-scale integrated circuit.
- Fig. 64 is a circuit diagram of a basic interlocking counter.
- Figure 65 is a block diagram of a network consisting of three interlocking counters.
- FIG. 66 is a timing chart when three interlocking counters are synchronized.
- Fig. 67 is a timing chart when one of the three interlocked counts is advanced.
- FIG. 68 is an evening timing chart when one of three interlocked counters is delayed in phase.
- Fig. 69 shows the timing when three interlocked counters have different phases.
- FIG. 70 is a block diagram of a network composed of interlocking counters arranged in a square lattice.
- FIG. 71 is a block diagram of a network including interlocked counters arranged in a hexagonal lattice.
- FIG. 72 is a circuit diagram of an interlocking signal distribution circuit that generates an output signal by using the third and fifth outputs of the signal distribution decoder.
- FIG. 73 is a timing chart of an interlocking signal distribution circuit that generates an output signal using the third and fifth outputs of the signal distribution decoder.
- FIG. 74 is a circuit diagram of a basic synchronous oscillation circuit.
- FIG. 75 is a circuit diagram of a synchronous oscillation circuit using an input resistor.
- FIG. 70 is a block diagram of a network composed of interlocking counters arranged in a square lattice.
- FIG. 71 is a block diagram of a network including interlocked counters arranged in a hexagonal lattice.
- FIG. 76 is a circuit diagram of a synchronous oscillation circuit using a crystal resonator.
- FIG. 77 is a block diagram of a network composed of three synchronous oscillation circuits.
- FIG. 78 is a timing chart when three synchronous oscillation circuits are synchronized.
- FIG. 79 is a timing chart in the case where one of the three synchronous oscillation circuits has advanced in phase.
- FIG. 80 is a timing chart when one of the three synchronous oscillation circuits is delayed in phase.
- FIG. 81 is a timing chart in the case where the phases of the three synchronous oscillation circuits are different.
- FIG. 82 is a timing chart when the power is turned on to the three synchronous oscillation circuits.
- FIG. 82 is a timing chart when the power is turned on to the three synchronous oscillation circuits.
- FIG. 83 is a block diagram of a network composed of synchronous oscillation circuits arranged in a square lattice.
- FIG. 84 is a block diagram of a network composed of synchronous oscillation circuits arranged in a hexagonal lattice.
- FIG. 85 is a block diagram of a central control device that communicates with a mobile camera, a visual device, a display, a voice synthesizer, and a voice recognizer.
- the invention according to claim 1 includes a digital image 111, one or more artificial eyeballs 1 that output the type, position, and number of objects, a digital image 111, an object
- a digital image 111 an object
- the artificial eye 1 is a moving turtle controlled by the visual device 2.
- the visual device 2 includes a camera 10 and identifies the type of the object in the digital image 11 1, specifies the position, and counts the number.
- the visual device 2 controls the pan (horizontal movement), tilt (vertical movement), roll (twisting), and zoom (telephoto) of the moving camera 10 according to a control command input from the user terminal 4.
- the individual user terminal 4 can receive the digital image 111 of the selected artificial eyeball 1, the type, the position, and the number of the object by selecting one of the plurality of artificial eyeballs 1. .
- the user terminal 4 includes a central control device 221, which has a communication function with the communication network 8, a digital image 1 11, a display 2 22 for displaying the type, position and number of objects, and a cursor key and a jog dial. And a control button 2 23. Since the artificial eye 1 and the user terminal 4 are connected by a communication network 8 using a wired technology and a wireless technology, the artificial eye 1 does not necessarily need to be installed near the user. Further, since the visual device 2 can identify the type of the object, the artificial eye 1 does not need to constantly transmit the digital image 111. Therefore, the user need not always monitor the user terminal 4.
- the simplest combination among the inventions described in claim 1 is that one artificial eyeball 1 and one user terminal 4 are connected by a wired communication network 8, and a digital image 111, an object type and a position
- This is an information collection system that can constantly communicate the number and control commands.
- a mobile phone 211 having an artificial eyeball 1 as shown in FIG.
- the mobile phone 211 can display a moving image captured by the mobile camera 10 on the display 222.
- the user of the mobile phone 2 1 1 can not only display his / her own form on the display 2 2 2 without worrying about the orientation and magnification of the moving camera 10, but also can use the display 2 2 2 2 You can see the surroundings taken by the moving camera 10.
- each mobile phone 211 communicates the digital image 111, the type, the position, the number of the object, and the control command with each other, and as shown in FIG.
- the information collection system as shown is constructed.
- the driver can confirm the blind spot.
- the visual device 2 automatically turns, tilts, rolls and zooms the moving camera 10 to change the shooting range 3 26, so that the driver can walk the pedestrian 3 2 behind the car 3 2 1.
- Bicycles, motorcycles 3 2 5 and another car 3 2 1 and children playing near the trunk of the car 3 2 1 can be found. Therefore, the driver can safely move backward in parking lots and garages with many blind spots.
- the driver should pay attention to places that are difficult to see with the pack mirror and side mirrors (particularly in distant places) and that they do not twist their bodies It is possible to check the places where it is not possible and the location of In other words, the visual device 2 automatically pans, tilts, rolls and zooms the moving camera 10 to change the shooting range 3 26 so that the driver can approach the pedestrian 3 coming from behind the car 3 2 1. 2, 4, bicycles, motorcycles 3 2 5 and another car 3 2 1 can be found. Therefore, the driver can safely open and drive the door.
- the case where the artificial eyeball 1 is installed at the rear part of the car 3 21 is described, but the same applies to the side part and the front part of the car 3 21.
- the driver can see the children playing near the door and the roadside zone. If the artificial eyeball 1 is installed in the center of the front bumper of the car 3 2 1 and in the front of the ceiling, the driver will be able to cross the front of the car 3 2 1 pedestrian 3 2 4, bicycle, motorcycle 3 2 5 and other It is possible to find a car 3221 and a child playing near the hood of the car 3221. Therefore, drivers can safely move forward at intersections of parking lots, garages and narrow alleys with many blind spots.
- the driver can grasp the situation of all the passengers. Especially when the mobile phone 2 1 1 is used as the user terminal 4, even after getting off the car 3 2 1 Can be. Therefore, even if the car 3 2 1 is in the devastation, the driver can rush to the car 3 2 1 immediately, and even if a child or baby is in the car, the driver can know the state of the child or baby. it can.
- the artificial eyeball 1 is installed behind the back of the wheelchair seat 331, and the user terminal 4 is installed on the handrail of the wheelchair seat 331.
- the user can check the object approaching from behind the wheelchair without twisting his body. In particular, it is difficult for the user of the wheelchair to see behind him because he cannot move his body freely, or the wheelchair moves forward and leans forward. However, by using the present invention, wheelchair users can move safely.
- the mobile camera 10 can photograph a traffic light, a road sign, and the like in a wider range.
- the information collection system has been described so that the user of the user terminal 4 can always manage the artificial eyeball 1.
- the artificial eyeball 1 transmits the digital image 11 1, the type, the position, and the number of the object to the user terminal 4 even during use / while sleeping.
- the digital image 111 transmitted by the artificial eye 1 and the type, position and number of objects are not always necessary for the user. Therefore, as shown in Fig. 5, some of the user terminals 4 are used as monitoring terminals 5, and a plurality of monitors operate the monitoring terminals 5 instead of the users, so that the users can only operate when necessary.
- the user terminal 4 can be used.
- the monitoring terminal 5 only has to transfer the digital image 111 received from the appropriate artificial eyeball 1 and the type, position and number of the object to the user terminal 4 of the user in accordance with the instruction of the monitoring person.
- the user terminal 4 and the monitoring terminal 5 are computer systems, using the existing technology, the observer can send a description of the situation to the user terminal 4 as necessary, Receiving instructions can also be facilitated.
- the invention described in claim 2 is an information provided with a database 6 such as a relational database having a database description language such as SQL (Structure Query Language) and an object-oriented database. It is a collection system.
- a database 6 such as a relational database having a database description language such as SQL (Structure Query Language) and an object-oriented database. It is a collection system.
- the database 6 records the digital image 111 transmitted by the artificial eyeball 1, the type, position and number of objects in a record as shown in FIG. 7, and calculates statistical data.
- the record records the record ID area 501 that records different record IDs, the time stamp area 502 that records the current time, and the number of the visual device 2 that the human eye 1 has.
- the data base 6 can calculate the number of objects of a specific type captured by the human eyeball 1 within a certain period of time, or estimate the movement of the objects.
- the database 6 searches for a record using the search key transmitted from the user terminal 4, and transmits the search result to the user terminal 4. By accessing the database 6 as necessary, the user can obtain only the target digital image 11 1, object type, position, number, and statistical data. The burden on the network 8 can be reduced.
- the intermediary of the plurality of monitoring terminals 5 between the plurality of user terminals 4 and the data base 6 allows the observer to provide the user with unique information and know-how. it can.
- franchise chains such as convenience stores and fast food restaurants, and management consultants, etc. provide information and know-how necessary for affiliated stores and customers. Can be provided. Intermediaries that anticipate traffic and crowds can also provide users with information.
- the user terminal 4 and the monitoring terminal 5 are computer systems, the monitor can use the existing technology to send a description of the situation to the user terminal 4 if necessary, You can easily receive the consultation.
- the user terminal 4 includes at least one of a speech synthesis device 226 and a speech recognition device 227.
- the speech synthesizer 226 converts the type, position, and number of objects received by the central controller 221 into a speech signal, and outputs the signal through the speaker 224.
- the voice recognition device 227 temporarily converts the user's voice collected by the microphone 225 into text and outputs the text to the central control device 221.
- the central control unit 222 converts this text into a control command and a search key, and then transmits the text to the artificial eye 1, the database 6, and the monitoring terminal 5.
- the mobile phone 211 Since the mobile phone 211 is already equipped with the speaker 222, the microphone phone 222, the processor 101 and the memory 102, the speech synthesis program and the speech recognition program are stored in the memory 102. By being implemented, the mobile phone 211 can easily be equipped with the speech synthesizer 222 and the speech recognizer 222. Regarding the napige system, a system equipped with a speech synthesizer 226 and a speech recognizer 227 has already been marketed. Therefore, the user of the user terminal 4 can control the direction, the inclination, and the magnification of the transfer lens 10 without being stared at the display 222 and without operating the troublesome control buttons 222. You can hear the type, position and number of objects.
- the information collection system including the artificial eyeball 1, the user terminal 4, and the database 6 connected by the communication network 8 has been described so far.
- the visual device 2 and the communication device 7 in the artificial eye 1 and the central control device 221 in the user terminal 4 are only briefly described. Therefore, in the following, the embodiment of the artificial eyeball 1 will be described focusing on the visual device 2 and the communication device 7, and then the embodiment of the central control device 221 will be described.
- the artificial eyeball 1 connected to the communication network 8 is represented by a configuration as shown in FIG. You.
- Many mobile cameras 10 are already on the market, and palm-sized mobile cameras 10 are also on the market.
- the video captured by the mobile camera 10 is transmitted to the visual device 2 as an analog video or a digital video without deteriorating the image quality of the frame image.
- the moving camera 10 also sends a signal indicating the current state of the moving force camera 10 to the visual device 2, while the visual device 2 controls the pan, tilt, roll, and zoom of the moving force camera 10.
- the manufacturer of the artificial eyeball 1 can use a commercially available product for the communication device 7, or can easily manufacture the communication device 7 using a communication platform.
- the communication device 7 has an image compression unit 701 and an image transmission unit 702.
- the image compression means 701 inputs a digital image 111 generated by the visual device 2 from a moving image captured by the moving camera 10 (solid line in FIG. 11), and compresses a moving image such as MPEG.
- the moving image and the frame image are compressed using a still image compression technology such as JPEG and a JPEG, and the MPEG image and the JPEG image are output to the image transmission unit 72.
- the image transmitting means 720 can transmit the MPEG image and the JPEG image to a remote computer system via a communication network 8 such as the Internet by implementing a communication protocol such as TCPZIP.
- a communication protocol such as TCPZIP.
- the communication device 7 can reduce the cost of moving images captured by the mobile camera 10, and save time and space. Can be transmitted without choosing.
- the artificial eyeball 1 can output a mask image 148 obtained by superimposing the digital image 111 and the object region image 142 instead of the digital image 111. Since the mask image 148 is obtained by painting the background of the digital image 111, the image compression means 701 can achieve a high compression ratio without adding a special function.
- the artificial eyeball 1 transmits a moving image of an object to a remote location by controlling the pan, tilt, roll, and zoom of the moving camera 10 using the visual device 2.
- simply using the artificial eyeball 1 described above does not necessarily mean that the digital image 111 and the mask image 148 show an object to be photographed. Especially when the object to be photographed is reflected in the digital image 1 1 1 but this object is not reflected in the mask image 1 4 8 Therefore, care must be taken when using the mask image 148.
- the image compression means 70 1 It is possible to compress only the digital image 111 and the mask image 148 on which the image is displayed, and output the compressed image to the image transmitting means 72.
- the algorithm of the image compression means 701 for the digital image 111 is as shown in FIG.
- step 71 it is determined whether or not the sequentially input recognition results have been exhausted. If there is no recognition result (step 711: Y E S), the algorithm ends. If there is a recognition result (step 711: NO), the process proceeds to step 712. However, if there is no need to judge the recognition result, an infinite loop may be used.
- step 7 1 and 2 enter the recognition result.
- step 7 13 it is determined whether or not the recognition result is a specific object type. If the recognition result is not a specific object (Step 7 13: N ⁇ ), the process returns to Step 7 12. If the recognition result is a specific object (step 7 13: Y E S), the process proceeds to step 7 14. In step 7 1 4, digital image 1 1 1 is input.
- a compressed digital image is generated using a compression technique such as MPEG and JPEG.
- step 7 16 the compressed digital image is output. Then, the process returns to step 7 1 1.
- the image compressing means 7 101 can compress only the digital image 11 1 showing a specific object. The same applies to the case where the mask image 148 is compressed.
- Moving cameras 10 using ultrasonic motors are currently being developed by several research institutions and companies, and moving cameras 10 cm in diameter, including the outer frame, have already been developed. .
- the ultrasonic motor does not require a mechanical mechanism such as a gear, the size of the transfer lens 10 using the ultrasonic motor can be further reduced.
- the artificial eyeball 1 is small and safe. Inexpensive communication device 7 and battery 212 can be used, and power consumption of battery 212 can be suppressed.
- a mobile camera 10 equipped with a small multi-degree-of-freedom camera such as an ultrasonic camera
- a user of the artificial eye 1 can wear the artificial eye 1.
- the artificial eyeball 1 was equipped with a moving camera 10, a visual device 2, a stimulus generator 2 13, an electrode array terminal 2 14, a connector 2 15 and a battery 2 12
- the user of the artificial eyeball 1 obtains a wide viewing angle, identifies an approaching object, and counts the number of the object by attaching the artificial eyeball 1 to eyeglasses or embedding it in the orbit. Will be able to do it.
- the algorithm of the vision device 2 for general use has already been developed, but needs some modification to be suitable for the human eye 1. Therefore, the visual device 2 will be mainly described below.
- a visual device 2 that searches for an object, recognizes the type of the object, and counts the number of objects (see Japanese Patent Application No. Hei 11-2005, Japanese Patent Application No. 200-64-64)
- the moving camera 10 has a mechanism for changing the direction in the horizontal direction and the vertical direction, and is capable of panning and tilting by inputting a command for controlling the moving angle from outside. I do.
- the mobile camera 10 has a mechanism for changing the magnification of the captured image, and can perform zooming by inputting a command for controlling the magnification from outside.
- the mobile camera 10 when the mobile camera 10 has a multi-degree-of-freedom motor, the mobile camera 10 has a mechanism that rotates around the lens optical axis, and rolls by inputting a command to control the tilt from outside. Can do it. Therefore, the moving force camera 10 can finely vibrate the camera itself by an external command. Accordingly, the frame image of the moving image captured by the moving camera 10 is blurred, and the object in the frame image is captured as if it were vibrating. Therefore, the stationary object is photographed as if it is moving in the frame image.
- the mobile camera 10 shall be able to output the current direction, tilt, and magnification moved by control commands such as pan, tilt, roll, and zoom, as necessary, and whether it is currently moving or stopped.
- the state of the mobile camera 10 such as whether or not it can be output can be output as needed.
- the visual device 2 can control the pan, tilt, roll, and zoom of the moving camera 10, the moving camera 10 can always take a moving object and a stationary object with an appropriate size.
- the visual device 2 can change the direction, the inclination, and the magnification of the moving camera 10.
- the visual device 2 uses three coordinate systems, that is, the camera coordinate system, the image coordinate system, and the environment coordinate system, depending on the function.
- the camera coordinate system is a three-dimensional spherical coordinate system inside the camera which is used by the moving camera 10 to control pan, tilt and zoom in each minimum control unit.
- the origin of the camera coordinate system is a position unique to the moving camera 10 called a home position.
- the camera coordinate system is the only coordinate system that can represent the physical position of moving and stationary objects. Nevertheless, since the camera coordinate system differs for each model of the moving camera 10, the force camera coordinate system has no versatility. Therefore, the visual device 2 needs to keep the camera coordinate system secret from other means.
- the image coordinate system is a two-dimensional coordinate system in units of pixels with the origin at the center of the frame image captured by the moving camera 10, that is, the lens optical axis. This is used to indicate at which pixel in the frame image the moving and stationary objects are located. Therefore, the image coordinate system is suitable for distinguishing the fine positions of multiple moving and stationary objects in a frame image, but the image coordinate system alone represents the physical positions of these moving and stationary objects. I can't.
- the environmental coordinate system is a three-dimensional spherical coordinate system that is used by the visual device 2 to logically unify the positions of a moving object and a stationary object.
- the environment coordinate system uses angles in radians in the horizontal and vertical directions, and uses real numbers in units of 1.0 in the distance direction to represent the product of the size of the object and the distance to the object. In general, since the size of an object does not change extremely in a short time, the distance to the object and the magnification of the moving camera 10 may be regarded as proportional.
- the origin of the environment coordinate system is arbitrary. In other words, environment coordinates The system is used to represent the relative coordinates of any two points on the environment coordinate system in principle.
- the visual device 2 can distinguish between a plurality of moving objects and stationary objects by projecting the moving objects and stationary objects in the environment that can be photographed by the moving camera 10 onto the environment coordinate system.
- the means playing the role are camera Z environment coordinate conversion means 20, image Z environment coordinate conversion means 21, and motion control means 23 (see FIG. 14). These means obtain the units of the camera coordinate system and the image coordinate system from the specifications of the moving camera 10 and the image acquisition means 11 (see Fig. 14), and calculate the matrix for conversion to the environmental coordinate system. . By calculating the inverse of the transformation matrix from the camera coordinate system to the environment coordinate system, the transformation matrix from the environment coordinate system to the camera coordinate system can also be obtained.
- the position of the environment coordinate system converted from the camera coordinate system is the relative position of the moving force camera 10 on the environment coordinate system from the home position.
- the position of the camera coordinate system converted from the environment coordinate system is a relative position from the current position of the moving camera 10 on the camera coordinate system.
- the image coordinate system is a two-dimensional coordinate system, it cannot be converted to the environment coordinate system using only the image coordinate system. Therefore, the image Z environment coordinate transformation means 21 calculates the transformation matrix as needed using the direction and magnification of the moving camera 10 expressed in the environment coordinate system and the size of the area of the moving object and the stationary object in the frame image. By doing so, it is possible to convert from the image coordinate system to the environment coordinate system.
- the position of the environment coordinate system converted from the image coordinate system is a relative position from the center of the image.
- the visual device 2 cannot obtain the environment coordinate system as described above.
- FIG. 15 it is assumed that there is formed edge information 114 of an object inclined at an angle 0 counterclockwise at a position ⁇ in a formed edge information image 115.
- the moving camera 10 is rotated so as to rotate counterclockwise by an angle 0 with respect to the center 0 of the formed edge information image 1 15 corresponding to the lens optical axis, the edge information forming means 15
- the formed edge information image 18 1 can be output.
- the object stands upright at the position ⁇ in the formed edge information image 18 1 after rotation, and the type of the object can be easily recognized.
- the position from the position ⁇ in the formed edge information image 18 1 after rotation) 3, that is, the position ⁇ in the formed edge information image 1 15 can be obtained.
- the image ⁇ environment coordinate conversion means 2 1 Can accurately determine the environment coordinates from the position ⁇ in the formed edge information image 18 1 after rotation.
- an embodiment of the visual device 2 will be described using the above-described camera coordinate system, image coordinate system, and environment coordinate system.
- an embodiment of the visual device 2 is an image acquisition unit that receives an image signal (frame image) of the moving camera 10 and converts the image signal into a digital image 111 of an appropriate format and size.
- image storage means 1 2 for storing digital image 1 1 1 for a certain period of time, image holding means 3 9 for holding digital image 1 1 1 until output to an output destination, and 2 digital images 1 1 1
- Edge information generating means 14 for generating a rough edge information image 113 of a moving object from the object, and edge information forming means 1 for forming the coarse edge information image 113 to a more accurate and clear formed edge information image 115 5 and an object that separates the area divided by the formed edge information 1 1 4 ⁇ Background separation means 16 and a mask hand that superimposes the separated area with the digital image 1 1 1 Step 40, image holding means 39 for holding the mask image 1 48 until output to the output destination, area normalizing means 27 for normalizing the separated area to the image size, and outputting to the output destination Image holding means 39 holding the normalized image 1 45 up to, geometric analysis means 37 analyzing the rough shape of the moving object using the formed edge information image 115, and the normalized moving object Pattern matching means 38 for performing pattern matching on the area of the image, image recognition means 29 for
- a position for detecting the position and size of each area Z size detection means 17, and a camera coordinate system representing the direction and magnification of the moving camera 10 are defined as an environment coordinate system which is a basic coordinate system of the visual device 2.
- Camera to convert to Z environment coordinate transformation The image Z environment in which the position and the position of the moving object detected by the position / Z magnitude detecting means 17 are converted into the environment coordinate system using the direction and the magnification of the movement force expressed by the step 20 and the environment coordinate system 10.
- Position selection means 22 to select one from the positions of a plurality of moving objects detected by 17, and timekeeping means 3 to measure time in millisecond units 2 and the orientation and magnification of the moving camera 10 in the environment coordinate system, the position of multiple moving objects in the environment coordinate system, the type of moving object, the geometric analysis result, and the time Environmental understanding means 31 to be created, environmental map holding means 33 to hold an environmental map until output to the output destination, and object position estimating means to estimate the position of the next moving object to be recognized using the environmental map 3 4 and control commands for externally controlling the direction and magnification of the moving camera 10
- a vibration command generating means 25 for generating a command to vibrate the moving camera 10, and a direction and a magnification of the moving camera 10 expressed in an environmental coordinate system
- the position of the mobile camera 10 represented by the camera coordinate system is referred to by referring to the four positions input from the position selection means 22, the control command input means 24, the vibration command generation
- the moving image is an analog signal when the image acquiring unit 11 inputs the frame image of the moving image from the moving camera 10, the frame image is converted into a digital signal by AZD conversion.
- the voltage of the image pickup device 207 such as a photodiode can be directly input, it is only necessary to convert the voltage into a digital signal having an appropriate number of bits by AZD conversion.
- the moving image is a digital signal, decompress it if it is compressed, or input it directly if it is not. With these, an arbitrary frame image can be cut out from the moving image.
- the image acquisition means 1 1 1 1 converts the image data into a format that can refer to the image data in pixel units, Cut out the required image size in and output it as digital image 1 1 1. If the image acquisition unit 11 can output all the pixels of the digital image 11 in parallel, the communication from the image acquisition unit 11 to the image storage unit 12 can be performed in parallel for each pixel. it can.
- the image storage means 12 When the image storage means 12 receives the digital image 111 from the image acquisition means 111, it stores the digital image 111 for a certain period of time in accordance with the time resolution of the visual device 2 or the calculation capability of each means. In other words, even if the digital image 1 1 1 is input during this fixed time, the image storage means 1 2 does not change the stored image, and the subsequent means input the same digital image 1 1 1 at different evenings. be able to. Moreover, since the image storage means 12 does not perform image processing on the digital image 111, all the pixels of the digital image 111 are stored while maintaining a two-dimensional positional correlation. If the image storage means 12 can output all the pixels of the digital image 11 in parallel, the image storage means 12 to the image holding means 39, the edge information generation means 14 and the color information generation means 4 1 Communication to the mask means 40 can be performed in parallel for each pixel.
- the image holding means 39 receives the digital image 111 from the image storage means 112, and if the output destination of the digital image 111 requests the digital image 111 of an appropriate format, the digital image 111 is obtained. 1. Convert the digital image 1 1 1 into the format required by the 1 output destination. Thereafter, the image holding means 39 stores the digital image 111 for a certain period of time so that the digital image 111 can be reliably transmitted to the output destination of the digital image 111. If the format to be converted is limited, the image holding means 39 can convert the digital image 111 only by neighborhood processing for each pixel. Is suitable for parallelization. If the image holding unit 39 can output all the pixels of the digital image 11 1 in parallel, the communication from the image holding unit 39 to the output destination of the digital image 11 1 is performed in parallel for each pixel. It can be carried out.
- the edge information generating means 14 When the edge information generating means 14 receives the digital image 1 1 1 from the image storing means 12, it generates a coarse edge information image 1 13 of the moving object by comparing with the digital image 1 1 1 inputted immediately before. I do. Since the edge information generating means 14 can generate the coarse edge information image 113 only by the neighborhood processing for each pixel, it is suitable for parallelization. If the edge information generating means 14 can output all the pixels of the coarse edge information image 113 in parallel, the edge information generating means 14 to the edge information forming means 15 and the position Z size detecting means 17 Can be performed in parallel for each pixel.
- the edge information forming means 15 When the edge information forming means 15 receives the coarse edge information image 113 from the edge information generating means 14, the coarse edge information image is referred to by referring to the digital image 111 stored in the image storage means 122. 1 13 Generates a more accurate and clear moving edge formed edge information image 1 15. Since the edge information forming means 15 can generate the formed edge information image 115 only by neighborhood processing for each pixel, it is suitable for parallelization. If the edge information forming means 15 can output all the pixels of the formed edge information image 115 in parallel, the edge information forming means 15 to the object / background separating means 16, the geometric analysis means 37 and the inclination Communication to the detection means 42 can be performed in parallel for each pixel.
- the object / background separating means 16 receives the formed edge information image 1 15 from the edge information forming means 15 and puts the pixels included in the object area 141 of the moving object and the pixels included in the background into different groups. Separate and output the separation result sequentially in group units. Note that if the object regions 14 1 are clearly distinguished by the formed edge information 1 14 even though they are adjacent, the object / background separation means 16 separates these object regions 1 4 1 into different groups. can do. Thus, the number of groups can be more than two.
- the object Z background separation means 16 is suitable for parallelization because the object area 141 and the background can be separated only by neighborhood processing for each pixel. If the object / background separation means 16 outputs all pixels of the object area image 14 2 in parallel, If possible, communication from the object / background separation means 16 to the mask means 40 and the area normalization means 27 can be performed in parallel for each pixel.
- the mask means 40 When the mask means 40 receives the object area image 14 2 from the object Z background separation means 16, the mask means 40 refers to the digital image 1 11 stored in the image storage means 12 and overlaps with the object area 14 1.
- the mask image 1 488 is generated by filling the pixels that do not need to be filled.
- the mask means 40 can generate a mask image 148 only by neighborhood processing for each pixel, and is suitable for parallelization. If the mask means 40 can output all the pixels of the mask image 1 48 in parallel, the communication from the mask means 40 to the image holding means 39 can be performed in parallel for each pixel.
- the image holding means 39 receives the mask image 1 48 from the mask means 40, and if the output destination of the mask image 1 48 requests a mask image 1 48 of an appropriate format, the mask image 1 4 Convert the mask image 14 8 to the format required by the 48 output destination. Thereafter, the image holding means 39 stores the mask image 148 for a certain period of time so that the mask image 148 can be reliably transmitted to the output destination of the mask image 148. If the format to be converted is limited, the image holding unit 39 can convert the mask image 148 only by the neighborhood processing for each pixel, and is suitable for parallelization. If the image holding means 39 can output all the pixels of the mask image 148 in parallel, the communication from the image holding means 39 to the output destination of the mask image 148 is performed in parallel for each pixel. Can be done.
- the area normalizing means 27 receives the object area image 14 2 and the digital image 11 1 from the object background separating means 16 and the image acquiring means 11 respectively, and the separated object area 14 3 of the moving object is digitally imaged.
- a normalized image 1 45 is generated by cutting out from 1 1 1 and complementing and enlarging as much as possible according to the image size of the digital image 1 1 1 while deforming the separated object region 1 4 3. Since the region normalizing means 27 can normalize the separated object region 144 only by neighborhood processing for each pixel, it is suitable for parallelization. If the area normalizing means 27 can output all the pixels of the normalized image 144 in parallel, communication from the area normalizing means 27 to the pattern matching means 38 and the image holding means 39 Can be performed in parallel for each pixel.
- the image holding unit 39 receives the normalized image 1 45 from the region normalizing unit 27, and If the output destination of the normalized image 145 requires the normalized image 145 in an appropriate format, convert the normalized image 145 to the format required by the output destination of the normalized image 145. . After that, the image holding means 39 stores the normalized image 145 for a certain period of time so that the normalized image 145 can be reliably transmitted to the output destination of the normalized image 145. If the format to be converted is limited, the image holding means 39 can convert the normalized image 145 only by neighborhood processing for each pixel, and is suitable for parallelization. If the image holding means 39 can output all the pixels of the normalized image 144 in parallel, the communication from the image holding means 39 to the output destination of the normalized image 144 is performed in parallel for each pixel. Can be done.
- the geometric analysis means 37 receives the formed edge information image 115 from the edge information forming means 15 and performs a geometric analysis such as a stroke extraction method, a Fourier transform, an affine transform and a Hough transform to obtain a frame image. Estimate the shape of the moving object inside and generate a geometric analysis result.
- a geometric analysis such as a stroke extraction method, a Fourier transform, an affine transform and a Hough transform.
- the object Z background separating means 16, the area normalizing means 27, the pattern matching means 38, and the image recognizing means 29 perform recognition processing on the pattern of the moving object in the frame image.
- the recognition processing performed by the object background separation means 16, the area normalization means 27, the pattern matching means 38, and the image recognition means 29 is not good at recognizing geometric figures such as contours of a moving object. Therefore, the geometric analysis means 37 can easily recognize geometric figures by roughly analyzing the formed edge information image 115 by a simple method, and also reduce the load on the geometric analysis means 37 itself. can do.
- the pattern matching means 38 does not need to recognize a geometric figure, and the pattern matching means 38 is simplified.
- the environment understanding means 31 can create an environment map accurately and at high speed. The geometric analysis results are converted from the geometric analysis means 37 to the image recognition means 29 and Output to environmental understanding means 31
- the pattern matching means 3 8 compares the normalized image 1 4 5 with a plurality of template images 1 4 6 on a pixel-by-pixel basis, and the matching result image 1 4 Generate 7
- the pattern matching means 38 can use a template image 144 prepared in advance according to the purpose of the visual device 2, or the input normalized image 144 as the template image 144. It can also be used. In general, each pixel of the normalized image 144 is similar to the corresponding pixel of the different template image 144, so that the matching result image 147 looks like a patch consisting of several clumps. If the pattern matching means 38 can output all the pixels of the matching result image 1 47 in parallel, the communication from the pattern matching means 38 to the image recognition means 29 is performed in parallel for each pixel be able to.
- the image recognition means 29 receives the matching result image 147 from the pattern matching means 380, and recognizes the matching result image 147 using an appropriate pattern recognition method, and determines the recognition result, that is, the type of the moving object. Output.
- the image recognizing means 29 can simply select the largest matching result among the matching result images 1 47 as in the case of majority voting, or learn by the error back propagation method (back propagation).
- a neural network such as a perceptor can be used.
- parallelization and speeding up are possible by using the neural network exclusive filer.
- the recognition result of the matching result image 147 that is, the type of the moving object, is output from the image recognition means 29 to the recognition result holding means 30 and the environment understanding means 31.
- the image recognition means 29 receives the result of the geometric analysis from the geometric analysis means 37, it first determines whether or not the image is recognized by the image recognition means 29. If the result of the geometric analysis is not a target figure, the image recognition means 29 does not operate. If the geometric analysis result is the target figure, the image recognition means 29 recognizes the matching result image 147. For example, in the case of a perceptron learning by the backpropagation method, the perceptron creates learning data using the matching result image 147 for each target figure, and then the image recognition means 29 learns from the geometric analysis result. By selecting the data, the perceptron can efficiently perform pattern recognition on the target figure.
- the result of the geometric analysis is circular, only the learning of a circular object such as a human face or a pole is used. If the result of the geometric analysis is a triangle, learning of a triangular object such as a road sign or mountain range is used. Use only data. In this case, not only can a large number of objects be recognized by a small-scale perceptron, but also the perceptron itself can learn efficiently.
- the recognition result holding means 30 inputs the recognition result of the matching result image 1 47 from the image recognition means 29, if the recognition result output destination requests a signal in an appropriate format, the recognition result output destination is Convert the recognition result to the required format. Thereafter, the recognition result holding means 30 stores the recognition result for a certain period so that the recognition result can be reliably transmitted to the output destination of the recognition result.
- the position and size detecting means 17 receives the coarse edge information image 113 from the edge information generating means 14, it detects the position and size of the area of the moving object indicated by the coarse edge information 112. .
- the position / size detection means 17 can generate the overlapping information image 13 2 representing the detection result of the position and size of the area of the moving object only by the neighborhood processing for each pixel, so that it is suitable for parallelization. I have. If the position Z size detection means 17 can output all the pixels of the overlapping information image 13 2 in parallel, the communication from the position Z size detection means 17 to the image Z environment coordinate conversion means 21 is , Can be performed in parallel for each pixel.
- the camera Z environment coordinate conversion means 20 and the image / environment coordinate conversion means 21 convert the position of the area of the moving object represented by the overlapping information image 13 2 generated by the position Z size detection means 17 into the environment coordinate system. To the position. At this time, if the total number of positions of the moving object area in the frame image is two or more, two or more positions exist on the environment coordinate system. Therefore, in order to control the pan, tilt, and zoom of the moving camera 10 and adjust the direction and the magnification of the moving camera 10 to one of the moving objects, one of a plurality of positions on the environment coordinate system is set. Must be selected.
- the position selecting means 22 selects one position on the environment coordinate system according to a certain judgment criterion. The criteria used here are mainly as follows.
- the position selection means 2 2 is a ring
- the closest object (or the largest object) to the moving camera 10 among the positions of the moving object on the boundary coordinate system is selected. This is because when the moving object is far (or small), the edge information generating means 14 may have generated noise, and the probability that the moving object exists from a plurality of positions on the environment coordinate system Is selected by the position selection means 22.
- the position selection means 22 selects one of them. There are two possibilities for this. One is the possibility that the edge information generating means 14 has generated coarse edge information 1 1 2 dispersed for one moving object, and the other is the possibility that there are actually multiple moving objects. is there.
- the position selection means 22 may select one position appropriately. Third, when a large number of positions on the environment coordinate system are approximately the same distance (or approximately the same size), the position selection means 22 is located closest to the origin, that is, closest to the direction of the moving camera 10. Select a position. By appropriately combining these in accordance with the application and the situation, the position selecting means 22 can select one from a plurality of positions on the environment coordinate system.
- the timer means 32 outputs the current time in 1 millisecond units by a timer circuit.
- the current time is constantly output from the timing means 32 to the environmental understanding means 31.
- the environment understanding means 31 inputs the recognition result of the object area 141 of the moving object from the image recognition means 29, and the environment understanding means 31 includes an environment consisting of the recognition result, the position of the moving camera 10 on the environment coordinate system, and the current time. Create environment data. Also, when the environment understanding means 31 inputs the positions of all the object regions 14 1 in the frame image on the environment coordinate system from the image Z environment coordinate conversion means 21, the recognition result consisting of null data, the moving camera 10 Creates as many environmental data as the number of object areas 1 4 1 including the position on the environment coordinate system and the position of the single object area 1 4 1 in the frame image on the environment coordinate system plus the current time. I do.
- the environment map is a set of environment data created a predetermined time before the current time, and the position in the environment data is represented by an environment coordinate system whose origin is the home position of the mobile camera 10.
- the environmental understanding means 31 adds and deletes environmental data as time elapses from the environmental map.
- the environmental understanding means 31 deletes the duplicated environmental data by replacing it with a change in the position, etc., and the recognition result is null data. If the position in one environment data is near the position in the other environment data where the recognition result is not null data, the environment data in which the recognition result is null data is deleted.
- the former Delete environmental data if the position in the environmental data whose recognition result is not null data is near the position in other environmental data whose recognition result is not null data, if the recognition results in these environmental data match, the former Delete environmental data.
- the accuracy of the environmental map is determined by the recording time of the environmental data and the range near the location of the environmental data.
- the environment map is converted from the environment understanding means 31 to the environment map holding means 33, the object counting means 35, and the object position estimating means. Output to 3-4.
- the environment understanding means 31 receives the result of the geometric analysis from the geometric analysis means 37, it first determines whether the image is recognized by the image recognition means 29.
- the environment understanding means 31 waits for a recognition result from the image recognition means 29 for a certain period of time.
- the environmental map holding means 33 When the environmental map holding means 33 inputs the environmental map from the environmental understanding means 31 1, if the output destination of the environmental map requires a signal in an appropriate format, the environmental map is stored in the format required by the output destination of the environmental map. Convert. Thereafter, the environmental map holding means 33 stores the environmental map for a certain period of time so as to reliably transmit the environmental map to the output destination of the environmental map.
- the object counting means 35 When inputting an environmental map from the environmental understanding means 31, the object counting means 35 counts the number of environmental data having recognition results indicating a specific object in the environmental map, and generates an arbitrary number of objects. Any number of recognition results in the environment data can be selected from the types of objects identified by the image recognition means 29. The number of arbitrary objects is output from the object counting means 35 to the object number holding means 36. If necessary according to the application, by changing the object counting means 35, the type of the object to be counted can be easily specified from the outside.
- the object position estimating means 34 selects one environment data whose recognition result is null data, and extracts the position during the environment data. By subtracting the current position of the moving force lens 10 on the environment coordinate system calculated by the camera / environmental coordinate conversion means 20 from this position, the object position estimating means 34 obtains the environment of the moving camera 10.
- this position is the estimated position of the moving object where the moving object may exist.
- an appropriate position on the environmental coordinate system is generated within a movable range of the mobile camera 10. At this time, the following can be considered as criteria for generating an appropriate position. First, an arbitrary position is generated by a pseudo random number. Second, calculate the locations where the density of locations in the environmental data in the environmental map is low.
- positions within the range in which the mobile camera 10 can move are sequentially generated in an appropriate order. For example, within the range in which the mobile camera 10 can move, the position is generated in order from the upper left end to the right, and when it reaches the right end, it descends one step and then generates the position sequentially to the left. After reaching the left end, go down one step and repeat to generate positions sequentially to the right.
- the object estimation position is output from the object position estimation means 34 to the motion control means 23.
- the color information generation means 41 When the color information generation means 41 inputs the digital image 111 from the image storage means 122, one or more colors and color areas in which the color of each pixel of the digital image 111 is designated in advance. Then, a color information image 17 2 is generated by selecting the one that matches with. Since the color information generation means 41 can generate the color information 17 1 in pixel units, it is suitable for parallelization. If the color information generating means 41 can output all the pixels of the color information image 17 2 in parallel, the communication from the color information generating means 41 to the position / size detecting means 17 is performed for each pixel. Can be done in parallel.
- the position Z size detecting means 17 When the position Z size detecting means 17 inputs the color information image 17 2 from the color information generating means 41, The position and size of the area of the moving object pointed by the color information 17 1 are detected.
- the position / Z magnitude detection means 17 can generate a duplicate information image 13 2 representing the detection result of the position and size of the area of the moving object by only neighborhood processing for each pixel. I have. If the position / size detection means 17 can output all the pixels of the overlapping information image 13 2 in parallel, the communication from the position / size detection means 17 to the image environment coordinate conversion means 21 It can be performed in parallel for each pixel. After that, one position of the moving object in the overlapping information image 13 2 is selected by the image Z environment coordinate conversion means 21 and the position selection means 22 and is output to the motion control means 23.
- the position, size, and inclination detecting means 44 determine the position, size, and inclination of the area of the moving object indicated by the formed edge information 1 14. To detect.
- the position / size Z tilt detection means 44 performs only neighborhood processing for each pixel, and calculates the overlapping information image 13 2 that indicates the detection result of the position and size of the moving object area and the tilt angle of the moving object area. It is suitable for parallelization because it can generate a tilt angle image to represent. If the position size and inclination detecting means 44 can output all the pixels of the overlapping information image 1 32 and all the pixels of the inclination angle image in parallel, the position Z size and the inclination detecting means 44 detect the inclination. Communication to the means 42 can be performed in parallel for each pixel.
- the tilt detecting means 42 When the tilt detecting means 42 receives the tilt angle image from the position Z size and the Z tilt detecting means 44, the tilt detecting means 42 detects the tilt angle located at the center. This allows the mobile camera 10 to roll in accordance with the object currently being photographed in the center. The tilt angle is output from the tilt detection means 42 to the motion control means 23.
- the visual device 2 can control the moving camera 10, identify the type of the object, and specify the position of the object without necessarily using all the means shown in FIG. 14. Is also possible. Therefore, when actually manufacturing the visual device 2, the manufacturer and the programmer of the visual device 2 need to take necessary measures according to the object to be photographed, the required performance, the provided computer resources, and the available LSI technology. You just have to choose.
- the visual device 2 is moved to the following position in addition to the position selected by the position selecting means 22. It is required to pan, tilt, roll and zoom the power camera 10.
- the visual device 2 must vibrate the moving camera 10 so that the moving camera 10 captures an image as if the stationary object is moving. Therefore, the vibration command generation means 25 specifies a position to which the mobile camera 10 should move as a position on the environment coordinate system. The position specified by the vibration command generation means 25 is determined by a pseudo random number or the like within a range where the moving camera 10 does not vibrate extremely.
- the visual device 2 is required to pan, tilt, roll, and zoom the mobile camera 10 by inputting a control command from the outside.
- panning, tilting, rolling and zooming are performed based on the current orientation, tilt, and magnification of the moving camera 10.
- the motion control means 23 selects one of the five positions on the environment coordinate system described above.
- the motion control means 23 is derived from the control command input means 24, the object position estimating means 34, the position selecting means 22 derived from the edge information generating means 14, and the color information generating means 41.
- the input position is selected from the means that can provide the input position.
- the selected position is converted from the environment coordinate system to the camera coordinate system by the motion control means 23.
- the camera command generation means 26 replaces the command with a command recognizable by the mobile camera 10 and transmits the command to the mobile camera 10. This allows the visual device 2 to control the pan, tilt, roll, and zoom of the moving camera 10.
- the motion control means 23 selects one of the five positions described above and outputs it to the camera command generation means 26, and then issues a command for inquiring information such as whether the mobile camera 10 is moving.
- the motion control means 2 3 is transmitted to the moving camera 10 by the camera command generation means 2. Instruct 6 to wait until this information is received from the moving camera 10 via the camera / environmental coordinate conversion means 20.
- the motion control means 23 If the received information indicates that the moving camera 10 is moving, the motion control means 23 generates a camera command again so as to send a command to the moving camera 10 as to whether or not the moving camera 10 is moving. Instruct means 26. If the received information indicates that the camera is stopped, the motion control means 23 sends a command to the camera command generation means 26 so that the mobile camera 10 sends a command for inquiring the current orientation, tilt and magnification to the mobile camera 10. To instruct. During this period, the operation control means 23 does not select five positions. The camera command generation means 26 transmits a corresponding command to the mobile camera 10 according to the instruction from the motion control means 23.
- the camera Z environment coordinate conversion means 20 transmits the information such as whether or not the moving force lens 10 is moving to the movement control means 23 as it is, and furthermore, the camera's current direction, inclination and magnification are transferred to the camera. Convert from the coordinate system to the position of the environment coordinate system. This allows the visual device 2 to control the pan, tilt, roll, and zoom of the mobile camera 10 while sequentially examining the state of the mobile camera 10.
- the edge information generation means 14 and the position Z size detection means 17 and the edge information formation means 15 and the position and size inclination detection means 44, the object Z background separation means 16 and the area It is not always necessary to make the resolution or image size of the image input by each of the normalizing means 27, the image holding means 39, and the pattern matching means 38 identical.
- the edge information generating means 14 is input with a low-resolution digital image 1 16 obtained by lowering the resolution of the digital image 111, while the edge information forming means 15 is input to the edge information forming means 15.
- a coarse edge information image 1 13 which is obtained by enlarging the image size of the low resolution coarse edge information image 1 17 generated by the edge information generating means 14 to the image size of the digital image 111 by an appropriate method is input,
- the load on the edge information generating means 14 can be reduced (see FIG. 25).
- the pan, tilt, roll, and the like of the moving camera 10 after the position and Z size detection means 17 are almost unchanged without substantially changing the quality of the normalized image 144 generated by the edge information forming means 15 or later. Zoom control can be made faster.
- the edge information forming means 15 includes the low-resolution coarse edge generated by the edge information generating means 14. From the information image 1 17, a cut-out coarse edge information image 1 19 obtained by cutting out the area where the coarse edge information 1 1 2 exists is input, and the object Z background separation means 16 and the area normalization means 27 are By inputting a cut-out digital image 120 cut out from the digital image 1 1 1 at the same position as the cut-out coarse edge information image 1 1 9, a normalized image 1 4 5 after the edge information forming means 15 This reduces the load on the generation of (see Fig. 26).
- the visual device 2 allows the moving camera 10 to shoot the moving object at an appropriate size in the center of the frame image, the cut-out coarse edge information image 1 19 for the digital image 111
- the cutout area of the cutout digital image 120 can be determined in advance.
- the visual device 2 can achieve the same performance as the object search device using the wide-angle camera and the high-resolution camera by using one moving camera 10.
- the inclination detecting means 44, the object / background separating means 16, the area normalizing means 27, the masking means 40, the image holding means 39, and the pattern matching means 38 are arranged in an array operation unit 100 (ARRAY OPERATION UNIT).
- An array operation unit 100 ARRAY OPERATION UNIT.
- the array operation unit 100 generates one pixel of the output image by using one pixel of the input image and its neighboring pixels. Therefore, as shown in FIG. 16, by using the data processing device 110 in which the array operation units 100 are arranged in a lattice in accordance with the size of the input image, the data processing device 110 An output image can be generated from the image.
- the array operation unit 100 is abbreviated as AOU.
- the array operation unit 100 may be implemented by dedicated hardware, or may be implemented by software on a general-purpose computer. In other words, as long as the output image can be generated from the input image, the mounting means is not limited. Therefore, by indicating the algorithm of the array operation unit 100, it is possible to indicate the image processing of the data processing device 110.
- the image storage means 12 the color information generation means 41, the edge information generation means 14, the edge information formation means 15 and the position size shown in FIG.
- the mathematical formulas used in the size detecting means 17, the position / size inclination detecting means 44, the area normalizing means 27, the masking means 40, the image holding means 39, and the pattern matching means 38 will be described.
- x is value at p (i, j, k), l ⁇ i ⁇ w, l ⁇ j ⁇ h, l ⁇ k ⁇ b ⁇ (2)
- y (yijk ⁇ Vijk is value at p (i, j, k) , l ⁇ i ⁇ w, l ⁇ j ⁇ h, l ⁇ k ⁇ b ⁇ (3)
- w is value at p ⁇ i,, k), l ⁇ i ⁇ w, l ⁇ j ⁇ h, l ⁇ k ⁇ b ⁇ (4)
- the maximum value is selected from the values of each band of the pixel in the i-th row and the j-th column in accordance with Equation (6). Since the band maximum value image is a single band image, it is handled as the image having one band for convenience. Therefore, the third subscript of the function B u 1 (A) is 1.
- a set P i jk (Q) of the positions near q at the position p (i, j, k) of the image is represented by Expression 9.
- q is a sequence of 4, 8, 24, 48, 80, 120, (2 r + l) 2 — 1 and r is a natural number.
- the position P (i, j, k) shall be used unless otherwise specified.
- a fictitious position whose pixel value is equal to 0 and which is not included in the image is substituted according to the specification. Thereby, the edge processing is automatically performed. Therefore, the number N ij k of elements of the set P i jk (q) is always Q.
- Equation 10 The smoothing at the position P (i, j, k) of the image X is performed according to Equation 10 below.
- the zero point found by Expression 12 is not a place where there is an edge, but a place where there is noise, that is, a place where there is no edge.
- real values are binarized using Equations 12 and 13.
- the calculation is performed according to Expression 14.
- the diagonal line cannot be detected due to its property, so that it is better to set the vicinity to 8 as much as possible.
- the two images are arbitrary binary images, and the image is an image in which a pixel with a line width of 1 is detected in the image, the line width of the pixel with a line width of 1 in the image is expanded. Therefore, calculation is performed according to Equation 16 using four neighboring pixels.
- each band pixel value of the image L can be masked using the band pixel value of the image according to Equation 21.
- the image y can be shaped based on the image X according to Equation 22.
- the band pixel value of the image ⁇ that is not specified by the image and the image that is specified by the image in the vicinity of the band pixel value of the image according to Equation 23 Is interpolated by the average value of the band pixel values of.
- int (v) means truncation of the real number v below the decimal point.
- the denominator is a positive number because image y is a binary image
- the processing is simplified by treating the position and the movement amount of the pixel like image data. This is called position imaging.
- position imaging In the following, some functions related to position imaging and operations are described.
- the operator that converts each value of m and o into band pixel values as image data is #, and the converted band pixel values are #p (1, m, o). I do.
- the band pixel value moves from position P (i, j, k) to position p (i + 1, j + m, k + o).
- the shift amount of the band pixel value is represented as a position p (1, m, o). That is, the movement amount can be regarded as a vector from a certain position.
- the movement amount p (i, j, k) can be turned in the opposite direction by 180 degrees in a plane represented by the width direction and the height direction.
- T (p (i, j, k)) p (-i, -j, k) (24) If there is an image L and the image is a single-band binary image, the position p (i, j, The amount of movement to the position of the center of gravity in 1) is calculated according to Equation 25. Note that division must be performed when calculating the center of gravity, but division is canceled out when calculating the amount of movement into the neighborhood 182, so division is omitted in Equation 25.
- Giji (x) p ((l-i) ximi, ⁇ (m-j) xi m i, 0) (25)
- Equation 27 is used only when Equation 26 cannot be used due to image discretization.
- Equations 25, 26 and 27 the band pixel value of the moving amount image of the single-band binary image ⁇ in the direction of the center of gravity can be simply described according to Equations 28 and 29.
- the number of bands of the moving amount image is one.
- Equation 24 the position opposite to the center of gravity can be obtained. Therefore, according to Equation 30, the band pixel value of the moving amount image in the direction opposite to the center of gravity of the single-band binary image can be simply described. be able to.
- the number of bands of the moving amount image is one.
- the band pixel value of the image is moved to the same band pixel after moving the band pixel value of the image to the movement position indicated by the image according to Equation 31.
- i jk (x, y) ⁇ , x lmk for p (l, m, 1)
- G ⁇ ⁇ ( ⁇ ) and l yimi p (im, 0).
- Equations 3 and 1 simply describe the sum of the band pixel values that have moved to the same band pixel after moving the single-band grayscale image in the direction of the center of gravity in accordance with Equation 32 or Equation 33. be able to.
- the movement amount image ⁇ can specify the movement source from among the eight neighbors 18 2 for each pixel. Therefore, the image is a duplicate information image 13 2, and as shown in Fig. 17, numbers 1 to 8 are assigned counterclockwise to the 8 neighbors 18 2 of each pixel of L. In this case, it is possible to determine whether or not the eight pieces of formulas from Formula 34 to Formula 41 have movable information 13 1 that can be moved for each of the eight sources. if
- Equations 42 and 43 instead of Equations 32 and 33, an eight-band grayscale image composed of eight source duplicate information 1 83 and a single-band grayscale composed of duplicate information 13 1 From the image y, source duplicate information 183 for each of the eight sources is generated.
- the band pixel value of the movable image that indicates that the moving destination of each band pixel value of the image does not overlap and that there is a moving band pixel value is generated according to Equation 44. Note that the number of bands of movable images is one.
- Equation 30 Equation 30, Equation 44 and Equation 45, the band pixel value of the image obtained by moving the band pixel of image 21 in the direction opposite to the position of the center of gravity calculated from the binary image can be easily calculated according to Equation 46. Can be described.
- Image storage means 12 color information generation means 41, edge information generation means 14, edge information formation means 15, position / size detection means 17, position / size Z inclination detection means 44, area normalization means 27, mask means 40, and all the array operation units of the data processing device 110 in which the image holding means 39 are mounted.
- G can describe 100 algorithms. In the following, an image storage unit 12, a color information generation unit 41, an edge information generation unit 14, an edge information formation unit 1 are described using an algorithm of an arbitrary array operation unit 100 in the data processing device 110. 5, position Z size detecting means 17, position Z size Z inclination detecting means 44, area normalizing means 27, masking means 40, and image holding means 39 will be described.
- the array operation units 100 arranged in a lattice form operate synchronously and in parallel. Assuming that the array operation unit 100 arranged at the i-th row and the j-th column on the lattice is AO U i, the algorithm of AO U i j is as shown in FIG.
- step 1 201 AO U u is arranged at the i-th row and the j-th column on the grid. This is necessary to determine the neighborhood of AO U ij, whether logical or physical.
- step 122 the neighborhood of AO U i j and initial values of variables are set.
- step 123 it is determined whether or not the digital images 111 to be sequentially input have been lost. If there is no digital image 1 1 1 (step 1203: Y E S), the algorithm ends. If there is a digital image 1 1 1 (step 1 203: N ⁇ ), the process proceeds to step 1 204. However, when the array operation unit 100 is implemented only for a specific image size, an infinite loop may be used.
- step 124 input is waited for until digital image 111 is prepared.
- step 1205 the pixels of the i-th row of the digital image 1 1 1); AO Because of this U u is that require memory 1 0 2 for storing image de Isseki at least the number of bands.
- the pixel at the i-th row and the j-th column of the digital image 111 is stored so that the digital image can be output while waiting for input.
- step 127 the band pixel value of the digital image 111 is output. Thereafter, the flow returns to step 1203.
- the image storage means 12 can store the digital image 111.
- the array operation units 100 arranged in a lattice are synchronized. And work in parallel. If the array operation unit 100 arranged at the i-th row and the j-th column on the grid is AOUi j, the algorithm of AOUi j is as shown in FIG.
- step 4101 AOUi j is arranged at the i-th row and the j-th column on the grid. This is necessary to determine the neighborhood of AOUij, whether logical or physical.
- step 4102 the neighborhood of AOUij and the initial values of variables are set.
- step 4103 it is determined whether or not the digital images 111 sequentially input have been exhausted. If there is no digital image 111 (step 4103: YES), the algorithm ends. If there is a digital image 111 (step 4103: NO), the flow shifts to step 4104. However, when the array operation unit 100 is implemented only for a specific image size, an infinite loop may be used.
- step 4104 the pixels in the i-th row and the j-th column of the digital image 111 are input for the number of bands. For this reason, AOU ij requires a memory 102 for storing at least image data for the number of bands.
- step 4105 a specific color is extracted from the pixel at the i-th row and the j-th column of the digital image 111 to generate color information 171.
- the generated color information 171 is handled as a band pixel value of the color information image 172.
- step 4106 the band pixel value of the color information image 172 is output. Thereafter, the flow returns to step 41 03.
- the color information generating means 41 can generate the color information image 172 from the digital image 111 using the data processing device 110 including the array operation unit 100.
- the edge information generating means realized by the data processing device 110 In order for 14 to generate the coarse edge information image 113 from the digital image 111, the array operation units 100 arranged in a lattice form operate synchronously and in parallel. Assuming that the array operation unit 100 arranged at the i-th row and the j-th column on the grid is AOUi j, the algorithm of AOU; j for the edge information generating means 14 is as shown in FIG.
- step 1401 AOU ij is arranged at row i and column j on the grid. This is necessary to determine the neighborhood of AOUij, whether logical or physical.
- step 1402 the neighborhood of AOUij and the initial values of variables are set.
- the neighborhood size Q used in each of the above functions may be individually determined to be 4 or 8, or the whole may be unified to 4 or 8.
- the edge information generating means 14 can cope with the situation by appropriately changing the neighborhood size as necessary, due to restrictions on the calculation time for generating the coarse edge information 112, the number of bands of the digital image 111, and the like.
- step 1403 it is determined whether the digital image 111 has been completed. If there is no digital image 111 (step 1403: YES), the algorithm ends. If there is the digital image 111 (step 1403: NO), the process proceeds to step 1404. However, when the array operation unit 100 is implemented for a specific number of bands and an image size, an infinite loop may be used.
- step 1404 the pixels of the i-th row and the j-th column of the digital image 111 are input for the number of bands. This is because AOU ij collectively processes the pixels in the i row and j column of the digital image 111. Therefore, AOUi "requires a memory 102 for storing at least the image data of the number of bands.
- step 1405 AOUi communicates with the nearby array operation unit 100 to perform smoothing on each band pixel value of the input digital image 111 in accordance with the function Sij k (2L).
- the smoothed band pixel values are treated as band pixel values of the smoothed image.
- the function S i jk (x) may be repeated several times as needed. For general multi-band images, Two is enough.
- step 1406 logarithmic transformation is performed on each band pixel value of the smoothed image according to the function Lij k (2L).
- the logarithmically converted band pixel values are treated as band pixel values of the logarithmically converted image.
- step 1407 A ⁇ Uij communicates with the neighboring array operation unit 100 to sharpen each band pixel value of the logarithmically transformed image according to the function Ei jki ⁇ L).
- the sharpened band pixel values are treated as band pixel values of the sharpened image. If the digital image 111 contains a large amount of noise, this step can be omitted.
- each band pixel value of the one-input pre-sharpened image is subtracted from each band pixel value of the sharpened image according to the function Di j k (2L, according to the function. It is treated as a band pixel value.
- each band pixel value of the one-input pre-sharpened image is replaced with a corresponding band pixel value of the sharpened image.
- step 1410 AOUij communicates with the nearby array operation unit 100 to calculate Laplacian for each band pixel value of the time difference image according to the operator V? Jk .
- the band pixel value for which the Laplacian has been calculated is treated as the band pixel value of the time difference Laplacian image.
- step 1411 the AOU u communicates with the neighboring array operation unit 100 to extract a zero point for each band pixel value of the time difference Laplacian image according to the function Z i ⁇ k ().
- the band pixel value from which the zero point has been extracted is treated as the band pixel value of the time difference zero point image.
- step 1412 the maximum value of each band pixel value is detected according to the function Bi j x_) for each band pixel value of the time difference Laplacian image.
- the detected maximum band pixel value is treated as the band pixel value of the maximum value time difference zero point image.
- the number of bands is 1 for convenience.
- AOUi communicates with the nearby array operation unit 100 to calculate Laplacian for each band pixel value of the sharpened image according to the operator V ⁇ jkL .
- the band pixel value for which the Laplacian is calculated is treated as the band pixel value of the Laplacian image.
- AOUij communicates with the neighboring array operation unit 100 to extract zero points according to the function Zi "for each band pixel value of the Laplacian image.
- the band pixel value from which the zero point is extracted is treated as the band pixel value of the zero point image.
- step 1415 the maximum value among the band pixel values of each band pixel of the Laplacian image is detected according to the function j i ().
- the detected maximum band pixel value is treated as the band pixel value of the maximum zero point image.
- the number of bands is 1 for convenience.
- step 1416 the maximum value of the band pixel values at the same position of each image according to the function M ij k (n) is calculated for each band pixel value of the Laplacian image and each band pixel value of the time difference Laplacian image. Is detected.
- the detected maximum band pixel value is treated as the band pixel value of the hybrid zero point image.
- the number of bands is 1 for convenience.
- AOU ij communicates with the neighboring array operation unit 100 to remove holes according to the function F ijk (2L) for band pixel values of the hybrid zero-point image.
- the band pixel values from which holes have been removed are treated as band pixel values of the hole removal hybrid zero-point image.
- the number of bands is one for convenience.
- the function F ijk () may be repeated several times as necessary. For general multi-band images, one time is sufficient.
- step 1418 AOU ij communicates with the neighboring array operation unit 100, so that the isolated pixel and the isolated pixel according to the function A ij ⁇ ) for the band pixel value of the hole-removed hybrid zero-point image Remove the holes.
- the band pixel value from which the isolated points and the isolated holes have been removed is treated as the band pixel value of the noise removal hybrid zero point image.
- the number of bands is 1 for convenience.
- step 14 19 0 and 1 are inverted according to the function I i jk () with respect to the band pixel value of the noise removal hybrid zero point image.
- the inverted band pixel value is treated as a band pixel value of the coarse edge information image 113.
- step 144 the band pixel value of the rough edge information image 113 is output. After that, return to step 1403.
- the edge information generating means 14 can generate a coarse edge information image 113 from the digital image 111.
- the edge information forming means 15 realized by the data processing device 110 has a coarse edge information image 113 composed of coarse edge information 112, and a digital image 111.
- the array operation units 100 arranged in a lattice form operate synchronously and in parallel. Assuming that the array operation unit 100 arranged at the i-th row and the j-th column on the lattice is AO U i j, the algorithm of AO U i j is as shown in FIG.
- AO U i j is arranged at the i-th row and the j-th column on the grid. This is necessary to determine the neighborhood of AO U! J, whether logical or physical.
- step 1502 the neighborhood of AO U i j and initial values of variables are set.
- the neighborhood size Q used in each of the above functions may be individually determined to be 4 or 8, or the whole may be unified to 4 or 8.
- the edge information forming means 15 can change the neighborhood size as needed, if necessary, depending on the calculation time for forming the coarse edge information 112 and the number of bands of the input digital image 111. I can deal with it.
- step 1503 it is determined whether or not the digital image 111 or the coarse edge information image 113 input sequentially is lost. If there is no digital image 111 or coarse edge information image 113 (step 1503: Y E S), the algorithm ends. If there is either the digital image 111 or the coarse edge information image 113 (step 1503: N O), the process proceeds to step 1504. However, when the array operation unit 100 is implemented for a specific number of bands and an image size, an infinite loop may be used.
- step 1504 the pixels of the i-th row and the j-th column of the digital image 111 and the coarse edge information image 113 are input for the number of bands. This is because AOUij processes the pixels on the i-th row and the j-th column of the digital image 111 and the rough edge information image 113 all at once. For this reason, AO U ij also requires a memory 102 for storing image data for the number of bands.
- step 1505 the pixel at the i-th row and the j-th column of the digital image 111 and the pixel at the i-th row and the j-th column of the rough edge information image 113 are separated. This is because AOU ij processes the pixel at the i-th row and the j-th column of the digital image 111 and the pixel at the i-th row and the j-th column of the coarse edge information image 113 as independent image pixels. If the pixel at the i-th row and the j-th column of the digital image 111 and the pixel at the i-th row and the j-th column of the coarse edge information image 113 are separated and input from the beginning, nothing is performed.
- step 1506 AOUij communicates with the nearby array operation unit 100 to perform smoothing on each band pixel value of the input digital image 111 according to the function S ijk ().
- Each smoothed band pixel value is treated as a band pixel value of the smoothed image.
- the function S ij k () may be repeated several times as needed. For a general multiband image, two times is sufficient.
- step 1507 logarithmic transformation is performed on each band pixel of the smoothed image according to the function L uk ().
- the logarithmically converted band pixel values are treated as band pixel values of the logarithmically converted image.
- step 1508 AOUij communicates with the neighboring array operation unit 100 to sharpen each band pixel value of the logarithmically transformed image according to the function E ijk (2L).
- the sharpened band pixel values are treated as band pixel values of the sharpened image.
- step 1509 Ri by that AOUi "to communicate with the array operation unit 100 in the vicinity, the calculation of Laplacian according Opere Isseki V jk L for each band-pixel value of the enhanced image.
- the band pixel value for which the Laplacian has been calculated is treated as the band pixel value of the Laplacian image.
- step 1510 Ri by that AOUi j to communicate with the array operation Yunitto 100 near, extracts zero-point according to the function Z ij k (2L) for each band-pixel value of the Laplacian image.
- the band pixel value from which the zero point has been extracted is treated as the band pixel value of the zero point image.
- step 1511 the maximum value of each band pixel value is detected according to the function Bu for each band pixel value of the zero point image.
- the detected maximum band pixel value is treated as the band pixel value of the maximum zero point image.
- the number of bands is 1 for convenience.
- step 1512 0 and 1 are inverted with respect to the band pixel value of the maximum value zero point image according to the function I ijk ().
- the inverted band pixel value is treated as a band pixel value of the basic edge information image.
- the input band pixel value of the coarse edge information image 113 is first treated as a band pixel value of the shaped coarse edge information image, and AOUij communicates with the neighboring array operation unit 100.
- the band pixel value of the shaped rough edge information image is shaped according to the function Qij k (x, ⁇ ) using the band pixel value of the basic edge information image.
- the shaped band pixel value is treated again as the band pixel value of the shaped coarse edge information image.
- the function Q uk (,) is repeated until the band pixel value of the shaped coarse edge information image no longer changes.
- the shaping process should be terminated after an appropriate number of repetitions. Good.
- step 1514 the AOU ij communicates with the neighboring array operation unit 100 to complement the line width of the band pixel value of the shaped coarse edge information image according to the function C i ” k (L). I do.
- the complemented band pixel values are treated as band pixel values of the formed edge information image 115.
- step 1515 the band pixel value of the formed edge information image 115 is output. Then, return to step 1503.
- the edge information forming means 15 forms the rough edge information image 113 into the formed edge information image 115 using the data processing device 110 composed of the array operation unit 100. Can be.
- the formation of the rough edge information image 1 13 into the formed edge information image 1 15 means that the same scene was shot from the edge information generated from the low resolution digital image 1 11 It can be regarded as estimating edge information to be generated from the high-resolution digital image 111. Therefore, for the natural number n, as shown in FIG. 25, from the low-resolution digital image 1 16 in which the resolution of the digital image 11 When the coarse edge information image 1 1 7 is generated, The coarse edge information image 113 can be generated by enlarging the edge information image 117 by n times. However, in order to set the resolution of the digital image 111 to lZn, it is sufficient to simply extract one of n consecutive digital images 111 in the horizontal and vertical directions.
- the band pixel value is simply set to 0 between successive pixels in the horizontal and vertical directions. It is sufficient to fill n-1 pixels.
- the data processing device 110 that realizes the edge information forming means 15 forms the coarse edge information image 1 13 by enlarging the low-resolution coarse edge information image 1 17
- the edge information forming means 15 uses the coarse edge information in order to refer to which edge information of the internally generated edge information is used by the edge information forming means 15 using the digital image 11. This is because only the information image 1 1 3 is used. Therefore, when the coarse edge information image 113 obtained by enlarging the low resolution coarse edge information image 117 is input to the edge information forming means 15, the low resolution coarse edge information image 117 from the low resolution digital image 116 is obtained.
- the data processing device 110 that realizes the generated edge information generating means 14 can reduce the amount of calculation or the amount of hardware.
- the position / size detection means 17 realized by the data processing device 110 is composed of the coarse edge information image 1 13 having the coarse edge information 1 12 as a pixel and the overlapping information 1
- the array operation units 100 arranged in a lattice form operate synchronously and in parallel in order to generate a duplicate information image 13 2 having 31 as pixels. If the array operation unit 100 arranged at the i-th row and the j-th column on the grid is AO U i j, the algorithm of AO U i j is as shown in FIG.
- step 1701 AOU; j is arranged at the i-th row and the j-th column on the grid. This is necessary to determine the neighborhood of AO U i j, whether logical or physical.
- step 1702 the neighborhood of AO Uij and the initial values of variables are set.
- the neighborhood size q used in each of the above functions may be determined individually, or all may be unified.
- the position / size detection means 17 may be used as necessary, depending on the calculation time for calculating the center of gravity of the coarse edge information 1 12 of the object, the size of the input coarse edge information image 113, etc. This can be dealt with by appropriately changing the neighborhood size.
- step 1703 it is determined whether or not the sequentially input coarse edge information images 113 have disappeared. If there is no coarse edge information image 113 (step 1703: Y E S), the algorithm ends. If there is a coarse edge information image 113 (step 1703: NO), the flow shifts to step 1704. However, when the array operation unit 100 is implemented only for a specific image size, an infinite loop may be used.
- step 1704 pixels in the i-th row :) 'column of the coarse edge information image 113 are input for one band. Therefore, AOUij requires a memory 102 for storing image data of at least one band.
- step 1705 the rough edge information 1 1 2 of the rough edge information image 1 1 3 is converted into the overlap information 1 3 1 of the overlapping information image 1 32.
- Duplicate information 1 3 1 is a band pixel value corresponding to 1 or 0.
- AOU i communicates with the neighboring array operation unit 100 to calculate the amount of movement in accordance with the function A ij) for each band pixel value of the overlapping information image 132. You.
- the band pixel value obtained by imaging the movement amount is treated as a band pixel value of the movement amount image.
- step 1707 AOU ij communicates with the neighboring array operation unit 100, so that the amount of movement for each band pixel value of the overlapping information image 132 is determined by the function A iji (L). Move to where you point.
- the shifted band pixel value is newly treated as a band pixel value of the overlapping information image 132.
- step 1708 it is determined whether or not the number of movements, which indicates the number of repetitions from step 1706 to step 1707, has reached the specified number. If the number of times of movement has not reached the specified number of times (step 1708: N ⁇ ), the flow returns to step 1706. If the number of times of movement has reached the specified number of times (Step 1708: Y E S), the flow shifts to Step 1709.
- the specified number of times is determined by the size of the formed edge information image 115, the size of the object represented by the formed edge information 114, and the nearby size q. If appropriate parameters are set according to the purpose of use, it does not matter if the number of specified times is overestimated, but if the number of specified times is too large, the time required to detect the position and size will increase. Become.
- step 1709 AOU u communicates with the nearby array operation unit 100 to calculate the amount of movement according to the function ⁇ ⁇ () for each band pixel value of the overlapping information image 132 I do.
- the band pixel value obtained by imaging the movement amount is treated as a band pixel value of the movement amount image.
- step 1710 the AOU ij communicates with the neighboring array operation unit 100 to obtain the amount of movement according to the function ⁇ ' U 1 () for each band pixel value of the overlapping information image 1 32. Move to the point indicated by.
- the shifted band pixel value is newly treated as a band pixel value of the overlapping information image 132.
- step 1711 the band pixel value of the overlapping information image 1332 is output. Thereafter, the flow returns to step 1703.
- each piece of overlapping information 1 3 1 of the overlapping information image 1 3 2 represents the total number of coarse edge information 1 1 2 around the position, and consequently the size of the object around the position. Means Will be.
- the position / Z magnitude detecting means 17 can generate the duplicated information image 132 from the coarse edge information image 113. Can be.
- the position / size detection means 17 realized by the data processing device 110 is also used for overlapping information representing overlapping information 1 3 1 from the object region image 1 42 representing the object region 1 4 1.
- Image 1 32 can be generated.
- each piece of overlap information 13 1 of the overlap information image 13 2 represents the total number of pixels of the object area 14 1 centered on that position. Means the area of the object about. Therefore, when calculating the size of the object from the overlapping information image 132, care must be taken, such as taking the square root of the overlapping information 131.
- the position / size realized by the data processing device 110, the inclination detecting means 44 overlaps the coarse edge information image 1 13 with the coarse edge information 1 12 as a pixel.
- the array operation units 100 arranged in a lattice form operate synchronously and in parallel in order to generate the overlapping information image 1332 having the information 1311 as pixels.
- the algorithm of AO U i j is as shown in FIG. In step 4401, AO U i j is arranged at the i-th row and j-th column on the grid. This is necessary to determine the neighborhood of AO Ui ”, whether logical or physical.
- the neighborhood of AO U ij and initial values of variables are set.
- the neighborhood size q used in each of the above functions may be determined individually, or all may be unified.
- the position, size, and Z-tilt detection means 44 need to be changed depending on the calculation time for calculating the center of gravity of the rough edge information 112 of the object, the size of the input coarse edge information image 113, etc. Size as needed Can be dealt with by changing.
- step 4403 it is determined whether or not the sequentially input coarse edge information image 113 has been lost. If there is no rough edge information image 113 (step 4403: YES), the algorithm ends. If there is a coarse edge information image 113 (step 4403: NO), the flow shifts to step 4404. However, when the array operation unit 100 is implemented only for a specific image size, an infinite loop may be used.
- step 4404 the pixel in the i-th row and the j-th column of the rough edge information image 113 is inputted for one band.
- AO Uij requires a memory 102 for storing at least one band of image data.
- step 4405 the rough edge information 112 of the rough edge information image 113 is converted into the overlap information 131 of the overlap information image 132.
- the duplication information 131 is a band pixel value corresponding to 1 or 0.
- the eight pieces of source overlap information 183 are cleared with a band pixel value equivalent to zero.
- AOUij communicates with the neighboring array operation unit 100 to calculate the movement amount according to the function ⁇ i ”i () for each band pixel value of the overlapping information image 132.
- the band pixel value obtained by imaging the movement amount is treated as a band pixel value of the movement amount image.
- the AOUij communicates with the neighboring array operation unit 100 to determine the destination indicated by the movement amount according to the function A ⁇ jk (2L, L) for each band pixel value of the source overlap information image 184. Move to The shifted band pixel value is newly treated as a band pixel value of the source overlapping information image 184.
- step 4408 AOUij communicates with the neighboring array operation unit 100 to move each band pixel value of the overlapping information image 132 to the point indicated by the movement amount according to the function Aij).
- the shifted band pixel value is newly treated as a band pixel value of the overlapping information image 132.
- step 4409 it is determined whether or not the number of movements representing the number of repetitions from step 4406 to step 4408 has reached the designated number. If the number of movements has not reached the specified number of times (step 4409: NO), the flow returns to step 4406. If the number of moves is the specified number If the number has reached (step 4409: YES), the flow shifts to step 4410.
- the number of times specified is determined by the size of the rough edge information image 113, the size of the object represented by the rough edge information 112, and the size Q of the neighborhood. There is no problem if the number of specified times is overestimated if appropriate parameters are set according to the purpose of use. Becomes longer.
- AOUij communicates with the neighboring array operation unit 100 to calculate the amount of movement in accordance with the function ⁇ ′′) for each band pixel value of the overlapping information image 132.
- the band pixel value obtained by imaging the movement amount is treated as a band pixel value of the movement amount image.
- step 4411 AOUij communicates with the neighboring array operation unit 100 to move each band pixel value of the source overlapping information image 184 to the point indicated by the moving amount according to the function ⁇ .
- the shifted band pixel value is newly treated as a band pixel value of the overlapping information image 132.
- step 4412 the AOU u communicates with the neighboring array operation unit 100 to move each band pixel value of the overlap information image 132 to the point indicated by the movement amount according to the function ⁇ .
- the shifted band pixel value is newly treated as a band pixel value of the overlapping information image 132.
- step 4413 it is determined whether or not the number of times of degeneration indicating the number of repetitions from step 4406 to step 4412 has reached the designated number. If the number of times of degeneration has not reached the designated number (step 4413: NO), the process returns to step 4406. If the number of times of degeneration has reached the designated number (step 4413: YES), the flow shifts to step 4414.
- the number of times specified is determined by the size of the coarse edge information image 113, the size of the object represented by the coarse edge information 112, and the size Q of the neighborhood. There is no problem in determining the number of specified times if appropriate parameters are set according to the purpose of use, but if the number of specified times is too large, the time required to detect the position, size and inclination becomes longer.
- step 4414 the band pixel value of the overlap information image 132 and each band pixel value of the source overlap information image 184 are output. Thereafter, the flow returns to step 440-3.
- each piece of overlap information 13 1 of the overlap information image 13 2 represents the total number of coarse edge information 1 12 around the position, and as a result, the size of the object around the position It means that. Also, since only the pixel having the overlapping information 13 1 has the moving source overlapping information 18 3, it means the inclination of the object around that position.
- the duplicate information image 18 4 can be generated.
- the formed edge information image 1 15 composed of the formed edge information 114 is used instead of the coarse edge information image 113 composed of the coarse edge information 112. Can also.
- an object area image 142 representing the object area 141 is used instead of the coarse edge information image 113 composed of the coarse edge information 112, as shown in FIG.
- the size Z tilt detecting means 44 can also generate the overlap information image 13 2 and the source overlap information image 18 4 from the object area image 14 2 representing the object area 14 1.
- each piece of overlapping information 13 1 of the overlapping information image 13 2 represents the total number of pixels of the object area 14 1 centered on that position. Means the area of the object around.
- the inclination angle 0 expressed in the counterclockwise direction is For example, it is obtained according to Equation 47. Note that the tilt angle 0 is expressed in radians, and that the object is horizontal and vertical. In this case, the tilt angle 0 is 0 radians. arctan (3 ⁇ 4 ⁇ 3 ⁇ 4)-f if Nl + N5> N3 + N7, ()
- Equation 47 is a simple equation for calculating the tilt angle ⁇ , but of course, by performing more detailed conditions than Equation 47, the user of the present invention can obtain the tilt angle 0 more accurately.
- the array operation unit 100 calculates the expression 47, the array operation unit 100 outputs eight source overlap information 18 3, A small amount of data can be output.
- the tilt angle 0 can be represented by a fixed step size, the memory 102 only needs to store a lookup table in which the required number of tilt angles are prepared instead of having a logarithmic table.
- Image 1 4 2 is input to position ⁇ size ⁇ inclination detection means 44, but of course, even if there are multiple objects, position ⁇ size ⁇ inclination detection means 44 is the position, size and inclination of each individual object Can be detected.
- the position / size ⁇ the inclination detecting means 44 detects the position, the size and the inclination from the mass of the individual color information 17 1. Can be detected. Therefore, the position ⁇ size ⁇ inclination detection means 44 can also be used for texture analysis.
- the area normalizing means 27 realized by the data processing apparatus 110 outputs the object area image 142 including the object area 141, and the separated object overlapping the object area 141.
- the array operation units 100 arranged in a grid are synchronized in parallel. Operate. Assuming that A ⁇ ⁇ U ij is an array operation unit 100 arranged at i-th row and j-th column on the grid, the algorithm of A ⁇ U i] is as shown in FIG. In step 2.701, AO U ij is arranged at the i-th row and the j-th column on the grid. This is necessary to determine the neighborhood of AOU ij, whether logical or physical.
- step 2702 the neighborhood of AO Uij and the initial values of variables are set.
- the neighborhood size Q used in each of the above functions may be determined individually, or all may be unified.
- the area normalizing means 27 appropriately changes the neighborhood size as necessary depending on the calculation time constraints for normalizing the separated object area 1 4 3 and the size of the input digital image 1 1 1 This can be dealt with.
- step 2703 it is determined whether or not the sequentially input object region image 144 or digital image 111 has disappeared. If there is no object area image 142 or digital image 111 (step 270: Y E S), the algorithm ends. If there is an object area image 142 or a digital image 111 (step 270: NO), the flow proceeds to step 274. However, when the array operation unit 100 is implemented only for a specific number of bands and image size, an infinite loop may be used.
- step 2704 the pixel of the i-th row and the j-th column of the object area image 142 is input for one band, and the pixel of the i-th row and the j-th column of the digital image 111 is input for the number of bands.
- AOU JJ collectively processes the pixel on the i-th row and the j-th column of the object area image 142 and the pixel on the i-th row and the j-th column of the digital image 111.
- AOUij requires a memory 102 for storing image data of at least the total number of bands.
- step 2705 the pixel at the i-th row and the j-th column of the object area image 142 is separated from the pixel at the i-th row and the j-th column of the digital image 111.
- AOU ij processes the pixel at the i-th row and j-th column of the object area image 142 and the pixel at the i-th row and the j-th column of the digital image 111 as independent image pixels. If the pixel at the i-th row and the j-th column of the object area image 142 and the pixel at the i-th row and the j-th column of the digital image 111 are separated and input from the beginning, nothing is performed.
- the object area image 142 and the digital image 111 are copied to the updated object area image and the updated digital image, respectively.
- AOUij communicates with the neighboring array operation unit 100 to calculate the movement amount according to the function R inix) for each band pixel value of the updated object region image.
- the band pixel value obtained by imaging the movement amount is treated as a band pixel value of the movement amount image.
- the AOUij communicates with the neighboring array operation unit 100 to move the destination band pixel according to the function Hij k (x, y_) for each band pixel value of the updated object region image. You can find the value. The value indicating whether it is a movable destination is treated as the band pixel value of the movable image.
- step 2708 Ri by that AOUi j to communicate with the array operation Yuni' Bok 100 near, to move a movable destination for each band-pixel value of the updated object-area image function U ij k (2L,) in accordance with .
- the shifted band pixel value is newly treated as a band pixel value of the updated object region image.
- AOUij communicates with the nearby array operation unit 100 to move the pixel value of each band of the updated digital image to the movable destination according to the function Ui " k (2L, _). .
- the shifted band pixel value is newly treated as a band pixel value of the updated digital image.
- step 2710 it is determined whether or not the number of movements representing the number of repetitions from step 2706 to step 2709 has reached the specified number. If the number of times of movement has not reached the specified number of times (step 2710: NO), the flow returns to step 2706. If the number of times of movement has reached the specified number of times (step 2710: YES), the flow shifts to step 2711.
- the designated number is determined by the size of the digital image 111, the size of the separated object area 143 of the digital image 111, and the size Q of the neighborhood. If appropriate parameters are set according to the purpose of use, it does not matter if the number of specified times is overestimated, but if the number of specified times is too large, the time required for normalization becomes longer.
- step 2711 Ri by that AOUi j to communicate with the array operation Yunitto 100 near the function Vi j k for each band-pixel value of the updated object-area image was complete the move (2L,) the average value of neighboring accordance To interpolate.
- the band pixel value filled with the average value is treated as a band pixel value of the normalized updated object region image.
- step 2712 AOU u communicates with the nearby array operation unit 100 by Then, each band pixel value of the updated digital image after the movement is filled with the average value in the vicinity according to the function Vij k (L,). Is the updated digital image, and is the updated object area image. The band pixel value filled with the average value is treated as a band pixel value of the normalized updated digital image.
- step 2711 it is determined whether or not the number of interpolations representing the number of repetitions from step 2711 to step 2712 has reached a specified number. If the number of interpolations has not reached the designated number (step 2713: NO), the flow returns to step 2711. If the number of times of interpolation has reached the specified number of times (step 2713: Y E S), the process proceeds to step 2714. Generally, the number of interpolations is about half of the neighborhood size Q.
- step 2714 it is determined whether or not the number of repetitions representing the number of repetitions from step 2706 to step 2713 has reached the designated number. If the number of continuations has not reached the specified number (step 2714: N ⁇ ), the process returns to step 2706. If the number of continuations has reached the specified number of times (step 2714: Y E S), the process proceeds to step 2715.
- the designated number is determined by the size of the digital image 111, the size of the separated object region 144 of the digital image 111, and the size Q of the neighborhood. If appropriate parameters are set according to the purpose of use, it does not matter if the number of specified times is overestimated, but if the number of specified times is too large, the time required for normalization becomes longer.
- step 2715 the band pixel value of the updated digital image is output as the band pixel value of the normalized image 144. Then, the process returns to step 2703.
- the region normalizing means 27 converts the normalized image 1 45 from the object region image 144 and the digital image 111 into Can be generated.
- the mask means 40 realized by the data processing device 110 generates a mask image 144 from the digital image 111 and the object region image 142 by using a grid.
- the array operation units 100 arranged in a matrix operate synchronously and in parallel. If the array operation unit 100 arranged at i-th row and j-th column on the grid is AOU i, then the algorithm of AOU u is 3 It becomes like Figure 8.
- step 4001 A ⁇ U i j is arranged at the i-th row and j-th column on the grid. This is necessary to determine the neighborhood of AO U i j, whether logical or physical.
- step 4002 the neighborhood of AOUu and initial values of variables are set.
- step 4003 it is determined whether or not the digital image 111 or the object area image 142 input sequentially is gone. If there is no digital image 111 or object region image 142 (step 4003: Y E S), the algorithm is terminated. If there is either the digital image 111 or the object area image 142 (step 4003: N ⁇ ), the process proceeds to step 400. However, when implementing the array operation unit 100 for a specific number of bands and image size, an infinite loop may be used.
- step 4004 the pixels in the i-th row and the j-th column of the digital image 111 and the object area image 142 are input for the number of bands.
- AOU ij collectively processes the pixels in the i-th row and the j-th column of the digital image 111 and the object area image 142.
- AOUij requires a memory 102 for storing image data of at least the number of bands.
- step 4005 mask processing is performed on each band pixel of the digital image 111 according to the function O ij " ⁇ , ⁇ ).
- the masked band pixel value is the band pixel value of the mask image 1 48 Is treated as
- step 4006 the band pixel value of the mask image 148 is output. Thereafter, the flow returns to step 4003.
- the masking means 40 can mask portions other than the separated object region 144 in the digital image 111.
- the array operation units 100 arranged in a lattice form operate synchronously and in parallel. If the array operation unit 100 arranged at i-th row and j-th column on the grid is AOU ij, the algorithm of AO U ij is as shown in FIG. In step 3801, AO U ij is arranged at the i-th row and the j-th column on the grid. This is necessary to determine the neighborhood of AO Ui ”, whether logical or physical.
- step 3802 the neighborhood of AO U u and initial values of variables are set.
- step 3803 it is determined whether or not the sequentially input normalized images 145 have disappeared. If there is no normalized image 1 45 (Step 380 3: Y E S), the algorithm ends. If there is a normalized image 145 (Step 380 3: N O), the flow shifts to Step 380 4. However, when the array operation unit 100 is implemented only for a specific image size, an infinite loop may be used.
- step 3804 the pixels in the i-th row and the j-th column of the normalized image 144 are inputted for the number of bands. Because AO U u this reason needs a memory 1 0 2 for storing image data at least the number of bands.
- step 3805 the format of the normalized image 144 is converted if necessary by the output destination device. In particular, if the number of bands of the normalized image 1 4 5 is set to 1, or if the number of bands of the digital image 1 1 1 is 4 or more, the number of bands of the normalized image 1 4 5 is set to 3, making it easier to generate analog signals. Convenient in case. Otherwise do nothing.
- step 3806 the pixels in the i-th row :) 'column of the normalized image 1450 are stored so that the image data can be reliably transmitted to the output destination device having a different processing speed.
- step 807 the band pixel value of the normalized image 145 is output. Thereafter, the flow returns to step 3.803.
- the image holding means 39 can output the normalized image 145 using the data processing device 110 composed of the array operation unit 100. Note that the image holding means 39 holding the digital image 111 and the mask image 148 operates in the same manner.
- image storage means 12, color information generation means 41, and edge information generation means in which data processing apparatus 110 inputs one or two images and performs image processing consisting only of neighborhood processing 14, edge information forming means 15, position Z size detecting means 17, position size Z inclination detecting means 44, area normalizing means 27, masking means 40, and image holding means 39 have been described.
- the pattern matching means 38 must use a very large number of images. No. Therefore, a method for realizing the pattern matching means 38 by the data processing device 110 will be described below while showing the neighborhood processing used by the pattern matching means 38.
- the normalized image 1 4 5 and L First, n pieces of template image 1 4 6 1, 2,,, and h ,,, y_ n.
- the matching result ⁇ i> of the i-th row and the j-th column of the matching result image 1 4 7 is obtained by dividing the normalized image 1 4 5 and the i-th row and the j-th column of the template image 1 4 6 according to Equation 48.
- the number of the template image 144 having the pixel most similar to the pixel of the normalized image 144 is indicated. Since the matching result image 147 is a single band image, it is handled as an image having one band for convenience. Therefore, the third subscript of the matching result ⁇ 5 i ”is 1.
- the matching result ⁇ i j 1 generated according to Expression 48 is not necessarily unified in the entire matching result image 1 47. If there are many template images 146, the matching result image 147 is likely to be rather mosaic. Therefore, a method in which the data processing apparatus 110 calculates a histogram for the matching result ⁇ ij 1 and the matching result in the vicinity of Q and converges the matching result ⁇ ij 1 will be described below.
- the matching result image 147 is updated according to Equations 49 and 50. Since the matching result image 147 is a single band image, it is handled as an image having one band for convenience. Therefore, the third subscript of the function] ⁇ )) is 1. ( 49 )
- the matching result image 147 By repeatedly calculating Equations 49 and 50 by the data processing unit 110 until the matching result image 147 no longer changes, the entire matching result of the matching result image 147 can be converged. At this time, the matching result converges as follows due to the combination of the normalized image 144 and the template image 144. If about half of the pixels in the normalized image 1 4 5 are most similar to the pixels in the particular template image 1 4 6, then most of the matching results in the matching result image 1 4 7 Converge to the number 6. However, if some clusters of pixels in the normalized image 144 are similar to clusters of pixels in several different template images 144, the matching result image 144 will be surrounded by zeros. There are a number of template images 1 4 6 numbered blocks.
- the matching result of the matching result image 144 becomes almost zero. Therefore, in the pattern matching realized by the data processing device 110, it is difficult to specify the template image 144 that is most similar to the normalized image 144, but several It is considered possible to select a template image 14.6 that is similar. Therefore, in the process of generating the recognition result from the pattern matching result, one of the most prominent candidates is selected from the similar candidates of the template images 144 enumerated by the matching result images 147 generated by the pattern matching. Just do it. As shown in FIG. 40, the pattern matching realized by the data processing device 110 indicates the number of the image most similar to the normalized image 144 of the template images 144.
- the array operation units 100 arranged in a lattice form operate synchronously and in parallel. Assuming that the array operation unit 100 arranged at the i-th row and the j-th column on the lattice is AOUi j, the algorithm of A ⁇ Ui j is as shown in FIG.
- step 2901 AOUi j is arranged on the i-th row and the j-th column on the grid. This is necessary to determine the neighborhood of AOUi j, whether logical or physical.
- neighbors of AOUi ”and initial values of variables are set.
- the neighborhood size Q used in each of the above functions may be determined individually, or all may be unified.
- pattern matching can be dealt with by appropriately changing the neighborhood size as needed.
- step 2903 it is determined whether or not the sequentially input template images 146 have been exhausted. If there is no template image 146 (step 2903: YES), the procedure moves to step 2905. If there is a template image 146 (step 2903: NO), the procedure moves to step 2904.
- step 2904 the pixels of the i-th row and the j-th column of the template image 146 are inputted for the number of bands. For this reason, AOUij requires a memory 102 for storing at least image data obtained by multiplying the number of bands by the number of template images 146. Thereafter, the flow returns to step 2903.
- step 2905 it is determined whether or not the sequentially input normalized images 145 have been exhausted. If there is no normalized image 145 (step 2905: YES), the algorithm ends. If there is a normalized image 145 (step 2905: NO), the flow shifts to step 2906. However, when the array operation unit 100 is implemented only for a specific image size, an infinite loop may be used.
- step 2906 the pixels in the i-th row and the j-th column of the normalized image 145 are input for the number of bands. others Therefore, AOUij requires a memory 102 for storing at least the image data for the number of bands.
- step 2907 a matching result ⁇ 5u of the matching result image 144 is calculated from the normalized image 144 and the template image 144.
- the matching result is a band pixel value indicating the number of the template image 144 closest to the normalized image 144.
- AO Uij communicates with the neighboring array operation unit 100 to obtain a matching result according to the function 3 ⁇ 4 ⁇ j) for each band pixel value of the matching result image 147.
- the updated band pixel value is again treated as the band pixel value of the matching result image 147.
- the function j) is repeated until the band pixel value of the matching result image 147 no longer changes.
- step 209 the band pixel value of the matching result image 147 is output. Then, return to step 2905.
- the pattern matching in the image recognizing means 29 uses the data processing device 110 composed of the array operation unit 100 to generate the matching result image 144 from the normalized image 144. Can be.
- a nonlinear oscillator generally causes a pull-in phenomenon.
- This pull-in phenomenon is a phenomenon in which nonlinear oscillators having different periods interact with each other and oscillate at a period of a simple constant ratio in a periodic behavior such as a limit cycle attractor.
- the vibration of one nonlinear oscillator is changed, the vibration of the other nonlinear oscillator is also changed, so these nonlinear oscillators are synchronized.
- the phase difference between the vibrations can be made as small or large as possible. So this mutual By manipulating the action, a group of nonlinear oscillators can be divided into multiple groups with different phases.
- the object Z background separation means 16 uses such a nonlinear oscillator pull-in phenomenon to separate the object and the background so that the edge information in the edge information image is a boundary, and the object area representing the object area 141 Generate image 142.
- a case where a van der Pol is used as the nonlinear oscillator will be described as an example.
- Equation 51 a sequence of 4, 8, 24, 48, 80, 120, (2 r +1) 2 — 1 and r is a natural number. If the nonlinear oscillator that exceeds the network size is included in the neighborhood set ⁇ u (Q), the nonlinear oscillator ou is substituted. Thereby, the edge processing is automatically performed.
- the nonlinear oscillator network is treated the same as a single-band image.
- the width direction the height direction.
- the nonlinear oscillator is coupled by i jkl Te bond value calculated in accordance with Equation 52 with the nonlinear oscillators in a neighbor set ⁇ ij (Q a) contained in the vicinity of Q a. If a logarithmic table is not used, approximation using Equation 53 is also possible. ⁇ And Les are appropriate positive constants.
- the nonlinear oscillator c ij will continue to operate in the same phase forever as long as the calculation is performed by the processor 101. Therefore, if a disturbance P ij is given, such a state can be avoided. Although a pseudo-random number can be used as the disturbance, a simple equation such as Equation 54 is sufficient. Note that ⁇ ij represents the presence / absence of edge information on the ith row and jth column of the edge information image. Set to 1 if there is edge information, 0 if not. Also, / is a suitable positive constant.
- the nonlinear oscillator ⁇ i ⁇ is the nonlinear oscillator ⁇ k of the neighborhood set ⁇ ⁇ j (q a ).
- the neighbor input sum ij is calculated according to Equation 55.
- phase shift is calculated based on whether the parameter jj is greater than or less than the threshold value 0.
- An output ⁇ ij that outputs a result obtained by separating the object region 141 and the background region is obtained by Expression 58. Note that 0 is an appropriate positive constant.
- edge information If the edge information is not enough to separate the object from the background, the edge information must be interpolated. For this purpose, it is necessary to find out how many non-linear oscillators are out of phase in the set of non-linear oscillators ⁇ ij (q b ) near Q b of the non-linear oscillator ⁇ ij. Then, the contour parameter 7 ⁇ ij is calculated by Expression 59. Based on this result, a boundary parameter ij indicating the interpolation ratio of the edge information is calculated by Expression 60.
- the ⁇ , ⁇ , rj mi n, 7? MA X is an appropriate positive constant.
- the object / background separation means realized by the data processing device 110 16 is separated into the inner region of the triangle 1 52 and the outer region of the triangle 1 53 using the triangle edge information 1 51, so that the array operation units 100 arranged in a grid form Operate.
- the algorithm of AOUi j is as shown in FIG.
- step 1601 AOU j j is arranged at the i-th row and the j-th column on the grid.
- step 1602 the neighbors co u and o kl are connected by i jkl based on the formulas 52 and 53.
- step 1603 appropriate initial values are set for the parameters ⁇ u and ij of the nonlinear oscillator.
- step 1604 it is determined whether or not the sequentially input formed edge information images 115 have disappeared. If there is no formed edge information image 1 15 (step 1604: YES), the algorithm ends. If there is a formed edge information image 1 15 (Step 1 604: N 0), the flow proceeds to Step 1 605. However, when the array operation unit 100 is implemented only for a specific number of bands and image sizes, an infinite loop may be used.
- Step 1 60 and inputs the zeta u of formed edge information 1 1 4.
- step 1606 a disturbance P ij is calculated from ⁇ ij of the formed edge information 114 input immediately before in accordance with Expression 54.
- Step 1 60 7 and enter the neighbor set j ⁇ k from AOU k i of array operation Interview knit 1 00 there is a nonlinear oscillator co kl in (Q a) have k have boiled kl, the total value shed ij Calculate according to Equation 55.
- step 1608 the parameters ⁇ u , ij> ij of the nonlinear oscillator are calculated according to equations 56 and 57. That is, the differential equations shown in these equations are solved by the Runge-Kuy ⁇ method.
- step 1609 the output ⁇ ij of the nonlinear oscillator is calculated according to Equation 58.
- step 1610 input A kl from AOU k i of array operation unit 100 with nonlinear oscillator c kl in neighborhood set Q u (Q b ), and calculate contour parameters ⁇ j Obey 59 To calculate.
- step 1 611 the boundary parameter ⁇ i j is calculated according to equation 60. That is, the differential equation shown in this equation is solved by the difference method or the Runge-Kuy ⁇ method.
- step 1612 it is determined whether or not the number of separations, which represents the number of repetitions from step 1606 to step 1611, has reached the specified number. If the number of separations has not reached the specified number (step 1612: N ⁇ ), the process returns to step 1606. If the number of separations has reached the specified number of times (step 1612: Y E S), the flow shifts to step 1613.
- step 1613 the output ⁇ ij of the nonlinear oscillator that becomes the band pixel value of the object area image 142 is output. Thereafter, the flow returns to step 1604.
- the following method can be used to determine the number of separations in step 1612.
- the separation ends in a certain time in almost all the formed edge information 114 regardless of the initial state of the nonlinear oscillator. It is sufficient to calculate the number of repetitions from step 1606 to step 1611. This is because if the initial state of the nonlinear oscillator is within a certain range, there is not much difference in the time until the nonlinear oscillator is synchronized by the pull-in phenomenon. Only by calculating the nonlinear oscillator in this way, the inner region 15 2 of the triangle and the outer region 15 3 of the triangle can be separated using the edge information 15 1 of the triangle. This is because the phenomenon of entrainment is used.
- nonlinear oscillators when two nonlinear oscillators are coupled with a positive coupling value, they tend to be in phase, and when coupled with a negative coupling value, the phase difference tends to be as large as possible.
- non-linear oscillators that are not directly coupled have the same phase by coupling non-linear oscillators arranged in a lattice with neighboring elements with a positive coupling value.
- both sides of the edge information will be out of phase with each other as much as possible. In this way, different sets of phases can be formed inside and outside the triangle edge information 151 without connecting all the nonlinear oscillators.
- the object Z background separation means 16 separates the inside area 15 2 of the triangle and the outside area 15 3 of the triangle as shown in FIG. Then the inside area of the triangle 1
- the phase difference between 52 and the outer region 153 of the triangle exceeds 90 degrees and approaches 180 degrees as much as possible, so that the triangle and the background region can be separated.
- the joint value is changed in a pseudo manner by the following method.
- the coupling value for coupling the nonlinear oscillator o k i to the nonlinear oscillator ⁇ ij is given by ij k ! (See step 1602).
- ⁇ ⁇ ⁇ ij and ⁇ ⁇ ⁇ kl are both 1 when there is an edge and 0 when there is no edge.
- the formed edge information 114 ⁇ k i is transferred from AOU kl of the array operation unit 100 to A OUj j, and A ⁇ ⁇ U; Ij k i (1—k l ) is calculated as a substitute for the combined value r ij kl (see step 1607).
- the substitution binding value each ijkl (1-zeta kappa iota) the boundary parameters Isseki xi] ij acts as the magnification between 0 and 1 (see step 1607).
- each A ⁇ Uij calculates the output ⁇ ij of the nonlinear oscillator (see step 1609).
- Ij representing the number of approximately the same phase der shall of nonlinear oscillators in the neighborhood is calculated according to equation 59 (see step 161 0).
- the boundary parameter which is a magnification of the joint value, is reduced according to Equation 60; Increase according to equation 60 (see step 1611). For example, in the case of 8 neighborhoods 18 2, if it is between 3 and 5, the boundary parameter may be reduced according to Equation 60. If this process is repeated, the edge area 15 5 of the dashed triangle and the outer area 15 5 of the dashed triangle shown in FIG. Separated into six. As shown in FIG.
- edge information 157 of the front triangle and edge information 158 of the rear triangle are obtained.
- the phases of the nonlinear oscillators in the three regions of the inner region 15 9 of the front triangle, the inner region 16 0 of the rear triangle, and the background region 16 1 of the double triangle are shifted from each other. Separated into regions. Also, as shown in Fig. 46, even if the two overlapping circular edge information 16 2 are broken lines, the front circular inner area 16 3 and the rear circular inner area 16 4 and the double circle background It is divided into three of the area 1 65.
- the object Z background separating means 16 uses the formed edge information 1 1 4 of the formed edge information image 1
- the area 14 1 and the background area can be separated.
- the plurality of separated areas are sequentially extracted with the passage of time by the output ⁇ ij, but the array operation unit 100 calculates the phase of the nonlinear vibrator to calculate the plurality of separated areas. They can be output at once as a phase group.
- the visual device 2 has been described. Naturally, these visual devices 2 can be implemented by a general-purpose computer, but when a moving object is to be searched, each of the means is executed at high speed depending on the moving speed of the moving object. There is a need.
- the image storage unit 12, the color information generation unit 41, the edge information generation unit 14, the edge information formation unit 15, and the position are processed for the image itself.
- the Z size detecting means 17, the object Z background separating means 16, the area normalizing means 27, the masking means 40, the image holding means 39, and the pattern matching means 38 are arranged in the width direction and the height direction, respectively.
- the calculation amount increases in proportion to the image size or the resolution. Therefore, the visual device 2 may not be able to achieve the desired performance in some applications.
- the array operation unit 100 is arranged in a grid pattern as shown in FIG. 16 in the data processing device 110, and the array operation unit 100 is further arranged in the data processing device 110. It is wired so that it can communicate with only the adjacent array operation unit 100. In other words, the four neighbors are directly wired. This makes it possible to operate at the same high speed with a small number of electronic components and wiring amount as compared to the case of wiring between 8 neighborhoods 1 8 2 and easily expandable even when the neighborhood size is expanded in the future. Can be.
- the array operation unit 100 stores a processor (PROCESSOR) 101 for calculating mathematical formulas in image processing and all parameters, constants, functions and operators used in the mathematical formulas.
- PROCESSOR PROCESSOR
- MEMORY 102
- CONTROLER CONTROLER
- the processor 101 is specified by the address bus 51. An arbitrary memory element and a register of the memory 102 and the controller 103 can be selected by the address (ADDRESS).
- the processor 101 is bidirectionally connected to the memory 102 and the controller 103 via the data bus 52 so as to be communicable with each other. Data (DATA) can be accessed.
- DATA Data
- the controller 103 When the array operation unit 100 inputs a front input data group (FRONT INPUT DATA SET) composed of one or more input pixels, the controller 103 stores the previous input data group in the memory 102. . Further, the controller 103 sends the calculation data in the memory 102 created by the function to the adjacent array operation unit 100 and also stores the calculation data received from the adjacent array operation unit 100 in the memory. It is stored in 102, and if necessary, transferred to the array operation unit 100 other than the input. Finally, the controller 103 outputs the image data of the output image as a result data (RESULT DATA).
- FRONT INPUT DATA SET front input data group
- the controller 103 stores the previous input data group in the memory 102. . Further, the controller 103 sends the calculation data in the memory 102 created by the function to the adjacent array operation unit 100 and also stores the calculation data received from the adjacent array operation unit 100 in the memory. It is stored in 102, and if necessary, transferred to the array operation unit 100 other than
- the reason why the controller 103 is mounted in each array operation unit 100 is that the processor 101 can operate while the array operation units 100 are communicating with each other. Since the data can be calculated even during the communication waiting time and high-speed processing can be realized, there is no need to change the hardware even if the number of array operation units 100 in the vicinity is changed. Since the controller 103 can automatically perform the edge processing of the image, that is, the exception processing for the edge pixels in the image, the program of the processor 101 does not need to perform the edge processing, which is extremely simple. Because it becomes.
- the address buffer (ADDRESS BUFFER) 53 receives an address (ADDRESS) from the processor 101 via an address bus (ADDRESS BUS) 51, and receives each register and other data through an address decoder (ADDRESS DECODER) 54. Select a function block.
- the data buffer (DATA BUFFER) 55 receives data (DATA) from the processor 101 via the data bus (DATA BUS) 52, and connects the register selected by the address decoder 54 to the internal data bus 56. Communicate exclusively via The communication direction is specified by READ.
- the data is stored in the flag register 57, decoded by the flag decoder (FLAG DECODER) 58, and arranged as a multiple signal (SIGNALS). Sent to 100.
- the multiple signals are received by the flag encoder (FLAG ENCODER) 59, analyzed, stored in the status register (STATUS REGISTER) 60, and returned to the source array operation unit 100 0 as reception (RECEIVE). You.
- the reception is received by the flag encoder 59 of the transmission source of the plural signals, and as a result, the completion of transmission of the plural signals is confirmed.
- the status register 60 When the status register 60 is selected by the address, the contents of the status register 60 are transmitted to the processor 101 as data via the data bus 52.
- the flag encoder 59 When one or more FRONT INPUT SENDs corresponding to one or more input images (INPUT IMAGE) are received by the flag encoder 59, the previous input data group (FRONT INPUT DATA) consisting of one or more input images is received. SET) is read into the front input data register (FRONT INPUT DATA REGISTER) 61 provided for the required storage capacity. If you select the previous input register 6 The contents of the input data register 61 are transmitted to the processor 101 as data. When the processor 101 completes the calculation, the result data register (RESULT DATA REGISTER) 62 is selected according to the address. ). At the same time, the flag encoder 59 sends a RESULT SEND.
- the output data register (OUTPUT DATA REGISTER) 63 is selected as the address and transmitted to the nearby array operation unit 100.
- the data to be read is read into the output data register as calculation data (CALCURATION DATA). Then, it is transmitted as calculation data to all adjacent array operation units 100.
- the calculation data is read into the upper input data register (UPPER INPUT DATA REGISTER) 64. Thereafter, when the upper input data register 64 is selected by the address, the contents of the upper input data register 64 are transmitted as calculation data.
- the lower input data register 65, left input data register 66, and right input data register 67 operate in the same way. I do.
- Each block of various buffers, various registers, and the address decoder 54 is a general-purpose electronic circuit. Specifically, the flag decoder 58 and the flag encoder 59 have input / output signals as shown in FIGS. 49 and 50.
- the type (TYPE) indicates the type of the content read into the OUTPUT DATA REGISTER 63 by 6 bits. This number of bits is a value sufficient to distinguish all the calculation data to be transmitted and received by the array operation unit 100.
- Each of the count X (COUNT-X) and the count Y (COUNT-Y) represents an unsigned 4-bit integer, and indicates the number of transfers between the array operation units 100. When the array operation unit 100 transmits the calculation data, each count becomes 0.
- the count of the flag encoder 59 is performed.
- X is added to 1 and the calculation data transmitted from the upper and lower array operation units 100 and 0 are transmitted again, the count of the flag encoder 59-Y is added to 1.
- the output data register 63 is specified after the transmission flag (SEND FLAG) of the flag register 57 specifies in which direction the data 101 should transmit the contents of the output data register 63, up, down, left, and right.
- the flag decoder 58 outputs a delivery (SEND) in accordance with the direction specified by the delivery flag.
- the transmission flag is represented by 4 bits, and when the calculation data of array operation unit 100 is transmitted to array operation units 100 on all sides, processor 101 sets 1 1 1 1 1 and the right side To transfer the calculation data sent from the array operation unit 100 to the upper and lower left, set the processor 101 to 1 1 1 0, and to transfer the calculation data from the left to the upper and lower right, set to 1 110 However, if the transfer is from the lower side to the upper side, set to 10000, and if the transfer is from the upper side to the lower side, set to 0100. This not only eliminates duplication of transfer and allows efficient transfer, but also clarifies the rules for determining the transfer direction.
- the flag encoder 59 Which type of calculation data has been transmitted from which array operation unit 100 can be determined. At the same time, the result data (RESULT DECODING) is received by the flag decoder 58, and the result delivery (RESULT SEND) is transmitted.
- the flag encoder 59 When the flag encoder 59 receives the transmission in any one of the four directions, the flag encoder 59 receives the type of the receiving direction, the count—X, and the count—Y, and updates the contents of the status register 60 of that part. At the same time as this update, set the reception to 1 in the receiving direction and send.
- the flag encoder 59 of the array operation unit 100 of the transmission source receives the moment the reception becomes 1, and updates the reception status (RECEIVE STATUS) of the status register 60.
- the processor 101 only checks the reception status of the status register 60 to determine which input data register stores valid calculation data. Can be.
- the processor 101 can read the data from the upper input data register 64 by specifying the address, but at the same time, the address decoder 54 Decoding on (UPPER DECODING) is sent to the flag encoder 59, the upper part of the reception status is returned to 0, and the reception toward the upper side is transmitted as 0. The same applies to the lower left and right sides.
- the status register 60 sets the front input delivery status (FRONT INPUT SEND STATUS) for the input image corresponding to the received previous input delivery. Set to 1.
- the address decoder 54 transmits the pre-decoding (FRONT DECODING) to the flag encoder 59 and receives the received pre-input data. Set the previous input delivery status corresponding to force delivery to 0. By reading the contents of the status register 60, the processor 101 can determine whether the latest input image is stored in the previous input data register 61.
- FIG. 51 shows an algorithm in the case where the processor 101 transmits the calculation data to the array operation units 100 on the four sides via the controller 103.
- FIG. 51 shows processing by program control by the processor 101 and mixing with hardware logic by the flag decoder 58 and the flag encoder 59.
- the processor 101 reads the contents of the status register 60.
- the processor 101 determines the type of data to be transmitted to the adjacent array operation unit 100, the count and the transmission direction, and writes the contents to the flag register 57.
- step 74 the data to be transmitted to the array operation unit 100, which is in contact with the processor 101, is written to the output data register 63.
- step 75 the contents of the output data register 63 are transmitted as calculation data to the adjacent array operation unit 100.
- step 76 the transmission is set to 1 only in the direction specified by the transmission flag of the flag register 57 and transmitted.
- the processor 101 starts this transmission algorithm each time the data to be transmitted is updated in the memory 102.
- FIG. 52 shows the algorithm.
- FIG. 52 shows processing by hardware logic by the flag decoder 58 and the flag encoder 59.
- the flag encoder 59 inputs the delivery.
- the flag encoder 59 determines whether the delivery is 1 or not. If NO, the process ends. If YES, proceed to step 83.
- the upper input data register 64 reads the calculated data transmitted from the upper side.
- the flag encoder 59 sets the reception status for the upper side of the status register 60 to 1 and simultaneously sets the reception to 1 and transmits it to the upper array operation unit 100. The same applies to the lower left and right sides.
- the one-time receiving algorithm of the controller 103 ends.
- the controller 103 constantly monitors the delivery from the upper, lower, left and right array processing units 100, and starts this receiving algorithm each time the delivery is received.
- FIG. 53 shows processing by program control by the processor 101 and mixing with hardware logic by the flag decoder 58 and the flag encoder 59.
- the processor 101 reads the contents of the status register 60.
- step 92 it is determined whether the reception status for the upper part of the read contents is 1 or not. If NO, the process ends. If yes, go to step 93.
- step 93 the processor 101 reads data from the upper input data register 64.
- the flag encoder 59 sets the reception status for the upper side of the status register 60 to 0, and simultaneously sets the reception to 0 and sends it to the upper array operation unit 100.
- the one-time receiving algorithm of the processor 101 ends.
- the processor 101 monitors the contents of the status register 60 at regular intervals, and starts this reception algorithm whenever the reception status of one of the upper, lower, left, and right is "1". Even if the processor 101 does not monitor the contents of the status register 60 at regular intervals, it can be implemented by interrupt processing.
- the array operation unit 100 mainly generates one output image from one or more input images. Although the description has been made on the assumption that the calculation is performed, it is necessary to change the circuit so that the calculation data in the middle of the calculation can be output depending on the application. At this time, the program is set so that the result delivery of the flag decoder 58 is increased by the number of calculation data to be output, and only the result delivery corresponding to the calculation data read into the result data register 62 is set to 1. You just need to change
- the array operation unit 100 when transferring an image with a large number of bands, such as the source overlap information image 18 4, if the array operation unit 100 has only one controller 103, the transfer efficiency may decrease. There is. Therefore, an array operation unit 100 including a plurality of controllers 103 will be described last.
- the array operation unit 100 according to claim 9 can include a plurality of controllers 103.
- the array operation unit 100 has five controllers 103 arranged diagonally, and each controller 103 is connected to the address bus 51 and the data bus 52. It is connected. A different address is assigned to the address decoder 54 of each controller 103 so that the processor 101 can identify the five controllers 103.
- each controller 103 Since each controller 103 has the previous input data register 61 and the result data register 62, all the controllers 103 independently input the previous input data group and output the result data. Can output overnight. Furthermore, each controller 103 is connected to a different one of the five controllers 103 of the adjacent array operation unit 100, so that the array operation unit 100 can increase the communication bandwidth up to five times. Can be extended to As a result, even when an image having a large number of bands, such as the source duplicate information image 184, is transferred, the transfer efficiency does not decrease if the number of controllers 103 is increased.
- the processor 101 follows the algorithm shown in FIGS. 51 and 53, and the status registers of all the controllers 103 are arranged. You should monitor the status in 0.
- the round robin method is mainly used for monitoring scheduling.
- the controllers 103 are logically arranged in a ring, and the processor 101 accesses the controllers 103 sequentially along the ring.
- Processor 101 is one The algorithm shown in FIG. 53 is executed for the controller 103. If there is more data to be transmitted, the processor 101 executes the algorithm shown in FIG. 51 for the controller 103. Thereafter, the processor 101 moves to the next controller 103 in the next order. Therefore, all controllers 103 are treated equally. This method is effective when the communication bandwidth of the controller 103 is narrower than the amount of data transmitted by the array operation unit 100 in a short time.
- a different priority number may be assigned to the controllers 103, and a priority method may be used in which the processors 103 access the controllers 103 in order from the highest priority every time. This method is effective when the communication bandwidth of the controller 103 is wider than the amount of data transmitted by the array operation unit 100 in a short time.
- the array operation unit 100 has only a small number of controllers 103.
- the round robin method is more effective than the priority method. It is advantageous.
- the array operation unit 100 has a sufficient number of controllers 103 so that the visual device 2 can execute local parallel image processing including functions that must transfer a large amount of data at high speed, the remaining If the function of ⁇ refers only to the pixel values in the 4 neighborhoods and the 8 neighborhoods 18 2, the communication efficiency of the round robin method will decrease. In such a case, the priority method is more advantageous.
- the controller 103 uses three-dimensional LSI technology. That is, they are stacked in the array operation unit 100. In the case of FIG. 55, five controllers 103 are stacked on the processor 101 and the memory 102, and each controller 103 has a vertically wired address bus 51 and data bus. It is connected to the processor 101 and the memory 102 by an overnight path 52. As is clear from FIG. 55, even if the number of controllers 103 is increased, the mounting area of each layer does not increase, and the controller 103 is connected to the adjacent controller 103 in the same layer. Good.
- the LSI designer wants to increase the number of controllers 103, he only has to increase the necessary number of layers on which the controller 103 is mounted, and if even the address bus 51 is properly designed, the LSI designer Does not require any modification to the underlying layer.
- the controller 103 of each layer is constituted by the same circuit except for the address decoder 54, the design of the controller 103 of the newly stacked layer becomes extremely simple.
- the array operation unit 100 includes a plurality of controllers 103 so that the adjacent array operation unit 100 Not only can the communication band between 0 and 0 be expanded, but also which calculation data has been received from which controller 103 can be identified. Therefore, in the following, a plurality of continuous array operation units 100 are grouped into one group by using a plurality of controllers 103, and furthermore, the array operation units 100 in this group are processed by the processors 101 and The data processing device 110 sharing the memory 102 will be described.
- the array operation units 100 arranged in a two-dimensional lattice are grouped into rectangular groups of an appropriate size. This group is called a virtual array operation unit 105.
- the virtual array operation unit 105 is a group of 3 ⁇ 3 array operation units 100 in one group.
- the virtual array operation unit 105 is composed of the processor 101, the memory 102, and the array operation unit 100.
- a controller 103 that is equal to or more than the maximum value of the number of horizontal lines.
- the virtual array operation unit 105 requires three or more controllers 103. Note that different numbers are assigned to these controllers 103 for convenience.
- the same channel numbers as those of the controller 103 are also assigned to the communication paths of the array operation unit 100. However, make sure that the facing numbers are the same.
- a program that allows the processor 101 to execute the nine array operation units 100 is stored in the memory 102. However, when the array operation unit 100 communicates with the array operation unit 100 of the adjacent virtual array operation unit 105, the array operation unit 100 always goes through the controller 103 that matches the number assigned to the communication path. Program is implemented as follows. As a result, the virtual array operation unit 105 can identify which array operation unit 100 to which array operation unit 100 the calculation data has been transmitted.
- the virtual array operation unit 105 cannot completely implement a plurality of array operation units 100 only by increasing the number of controllers 103. This is because the virtual array operation unit 105 must input the previous input data group from another virtual array operation unit 105 and output the result data to another virtual array operation unit 105. Because. If the controller 103 is mounted on a two-dimensional plane as shown in Fig. 54, if the number of controllers 103 is sufficient, number the controllers 103 in the same way as the calculation data This problem is solved by the program. However, even in the case of Fig. 57, nine controllers 103 are required. Therefore, when the number of array operation units 100 included in the virtual array operation unit 105 increases, this method becomes very inefficient.
- controllers 103 are stacked as shown in FIG. 55, it is very difficult to secure a communication path for the previous input data group and the result data. Therefore, in the following, a method for solving this problem by changing only the previous input data register 61 of the controller 103 will be described.
- the pre-input data register 61 is a pre-input data register composed of the same number of registers as the number of array operation units 100 in the virtual array operation unit 105.
- a shift register 68 and a previous input counter 69 are provided.
- Before input count 6 9 This is a down-counting event whose initial value is the number of characters, and when the count value of the previous input counting device 69 becomes 0, the previous input counter 69 outputs the previous input delivery. Thereafter, when the result data is input, the previous input counter 69 is initialized to the initial value.
- the front input shift register 68 is composed of nine registers. When nine result data are input to the front input shift register 68, the front input delivery is output. Next, each of the previous input shift register 68 and the previous input counter 69 inputs the result delivery as a trigger signal. Further, the address decoder 54 decodes the address and selects one of the registers of the previous input shift register 68. As a result, when the processor 101 designates an appropriate register address of the previous input shift register 68, each register can output the result data to the data bus 52. In FIG. 58, other signal lines of the address decoder 54 are omitted.
- the virtual array operation unit 105 outputs the result data of the nine array operation units 100 in a fixed order, these result data are stored in the register of the previous input shift register 68 in the output order. Therefore, if the processor 101 accesses each register at the time when the previous input delivery output by the previous input counter 69 is stored in the status register 60, the processor 101 can use a specific array operation unit. The result data of 100 can be obtained.
- the visual device 2 has been described. If the image size of the digital image 1 1 1 is limited and the calculation time is tolerated to some extent, the visual device 2 can be realized with a commercially available single-chip computer, but it is necessary to increase the image size and achieve real-time performance. Need to implement a dedicated large-scale integrated circuit. In particular, by mounting some means of the visual device 2 on a large-scale integrated circuit, the visual device 2 can realize space saving, low power consumption, and real-time performance.
- an image sensor using the visual device 2 will be described.
- the image sensor used in the invention according to claim 7 is, for example, one in which function columns 201 shown in FIG. 59 are arranged and wired in a two-dimensional lattice as shown in FIG.
- the array operation unit 100 is abbreviated as AOU
- the function column 201 is abbreviated as FC.
- duplicate information 1 3 1 and match from image sensor In order to make it easier to output the result of the marking, three image holding means 39 are added to FIG.
- a plurality of array operation units 100 realize each means, but as shown in FIG. 61, a single array operation unit 100 can realize a plurality of means by a program. .
- a single virtual array operation unit 105 can realize a plurality of array operation units 100 by a program.
- an imaging device such as a photodiode and a CCD. 207 and an A / D conversion circuit 204 are incorporated.
- an individual oscillation circuit instead of inputting a clock signal from the outside of the function column 201 in the function column 201, an individual oscillation circuit may be used.
- the photodiode can be mounted on the same large-scale integrated circuit (LSI) as CMOS (complementary metal oxide semiconductor), the design and implementation of the image sensor becomes extremely simple.
- each array operation unit 100 in FIG. 59 is located in the vicinity of 4 and communicates with the array operation unit 100 executing the same means via a signal line.
- each array operation unit 100 can configure the data processing device 110 according to the processing.
- the image sensor is implemented by three-dimensional large-scale integrated circuit (three-dimensional VLSI) technology, as shown in Fig. 63, all the array operation units in the function column 201 are used by the three-dimensional VLSI. If implemented vertically, the signal lines between the function columns 201 will not cross even between the array operation units 100. Therefore, the design and implementation of the image sensor becomes extremely easy.
- the interlocking county 4 16 consists of a county part and an interlocking part. Also, in Fig. 64, the interlocking counter 4 16 has four interlocking counters 4 16 Input the interlocking signals BLKl, BLK2 ', BLK3, and BLK4.
- a synchronous type counter 411 Input the interlocking signals BLKl, BLK2 ', BLK3, and BLK4.
- a NOT gate 412 for end value determination In the counter section, a synchronous type counter 411, a NOT gate 412 for end value determination, and a N ⁇ R gate 415 for force control are connected in a ring.
- FIG. 64 shows a binary 3-digit synchronous counter 411 composed of three flip-flops synchronized with the clock signal (CLK). By using N flip-flops, It is easy to change to a binary N-digit synchronous power pin. Also, by changing the input of the NOR gate from the Q 'terminal to the Q terminal of the flip-flop in the synchronous counter 411,
- the synchronous counter 411 outputs the count number in synchronization with the clock signal.
- the NOT gate 412 for the end value determination outputs the interlock signal BLK0 '.
- the counting NOR gate 415 outputs the enable signal (G ') until the interlocking signal BLK0' is input. Therefore, the synchronous counter 411 counts until the count reaches the end value, and stops counting when the count reaches the end value.
- the interlocking NOR gate 414 and the interlocking latch circuit 413 control the counting NOR gate 415 according to the interlocking signals BLK 1 ′, BLK 2 ′, BLK 3 ′, and BLK 4 ′ input from the outside.
- the interlocking part can adjust the phase and period of the interlocking signal BLK0 'to the phase and period of the interlocking signals BLK1', BLK2 ', BLK3' and BLK4 '.
- the interlocking counter 416 operates in synchronization with the clock signal, the clock signal itself is not always supplied to all the interlocking counters 416 at the same time.
- FIG. 64 shows the case where the interlocking counter 416 inputs the interlocking signal from the four interlocking counters 416.
- the interlocking counter 416 can be set to an arbitrary initial value by using a synchronous counter 411 having a synchronous clear means and a synchronous loading means such as a commercially available synchronous counter 411. End value can be set. In this case, the end value is determined by adding a NOR gate before the NOT gate 412 for the end value determination. Of course, the combination of the NOT gate 412 and the NOR gate for the end value determination is replaced by a NAND gate.
- the interlocking counter 416 is abbreviated as ICU.
- the interlocking signal B L K 0 ′ of each interlocking counter 4 16 is input to the interlocking NOR gate 4 14 of the remaining interlocking counter 4 16. Therefore, the interlocking NOR gate 4 1 4 should just be a 2-input NOR gate.
- the timing chart of one of the interlocked counters 4 16 is shown in FIG. Since all the interlocking counters 4 16 are symmetrical, the timing chart of the remaining interlocking counters 4 16 is the same.
- the interlocking counter 416 operates regardless of the interlocking signal BLK 1 '. Therefore, the interlocking signal BLK1 'has no effect on the count.
- the interlocking counter 416 that generates the interlocking signal BLK1 ' operates so that the interlocking signal BLK1' matches the phase of the interlocking signals BLK0 'and BLK2'.
- interlocking counter 416 operates to match the phase of interlocking signal BLK0' to the phase of interlocking signal BLK2 '. I do. Therefore, until the interlocking signal BLK 2 ′ becomes L level, the interlocking power supply 416 continues to output the end value as the count number.
- the interlocking counter 416 interlocks the phase of the interlocking signal BLK0 '. It operates to match the phase of the signal BLK 2 '. Therefore, the interlocking counter 416 continues to output the end value as the count number until the interlocking signal BLK2 'becomes L level.
- the three interlocking counters 416 adjust the count number to the one with the slowest count. This is true even if interlocking counters 416 with different end values are connected. Therefore, when the power is turned on, even if the counts of the three interlocked counts 416 are different, the counts of all interlocked counts 416 match within the time obtained by multiplying the clock signal cycle by the maximum number of end values. I do.
- the interlocking counter 416 does not necessarily need to be connected to all other interlocking counters 416 as shown in FIG. Therefore, a case where the interlocking type counters 416 are regularly arranged will be described below.
- the interlocking device composed of the interlocking counter 416 is a network in which interlocking counters 416 arranged in a square lattice are connected to each other.
- the number of inputs of the interlocking NOR gate 414 is four.
- the input of the interlocking NOR gate 414 having no connection destination is pulled down.
- the interlocking counters 416 are arranged in a hexagonal lattice as shown in Fig. 71. They can be adjacent to each other.
- the interlocking counters 416 By arranging the interlocking counters 416 in this manner, the lengths of all interlocking signal lines are substantially equal, so that the interlocking counters 416 can easily cooperate with each other. Therefore, for large and regular digital circuits such as pipeline processors, DSPs (Digital Signal Processors), systolic arrays, data flow machines, and parallel image processors, these two-dimensional The counter network can easily supply the count number of the interlocking counter 4 16, that is, the frequency-divided signal of the clock signal.
- DSPs Digital Signal Processors
- the three-dimensional power network is a network in which a plurality of interlocking counters 4 16 arranged in a square lattice and a hexagonal lattice as described above are superimposed using the three-dimensional LSI technology. Therefore, when arranged in a square lattice, the number of inputs of the interlocking NOR gate 4 14 is 6, and when arranged in a hexagonal lattice, the number of inputs of the interlocking NOR gate 4 14 becomes 8. In addition, by adjusting the position of the interlocking counters 4 16 in each layer, the length of the signal line can be minimized, so that the propagation delay time of the interlocking signal of each interlocking counter 4 16 is almost equal.
- the interlocking council 4 16 can be easily interlocked with each other.
- These three-dimensional counting networks are based on systolic arrays and parallel image processing, in which data processed simultaneously by processors 101 arranged in a square grid and hexagonal grid are pipelined in the vertical direction. It is possible to efficiently supply the device with the count of the interlocking counters, that is, the frequency-divided signal of the clock signal.
- a network of interlocked counters 4 16 arranged in a square lattice and hexagonal grid and a digital circuit such as a processor 101 and an arithmetic circuit are used. Can be implemented in different layers of a three-dimensional LSI. Therefore, due to the arrangement of the digital circuit, the arrangement of the interlocking counters 4 16 does not shift, and the signal line of the interlocking signal BLK 0 ′ does not bypass. In addition, since the interlocking counter 416 is not affected by the noise of the digital circuit, the interlocking counter 416 operates stably.
- the digital circuit can input the count number from any of the interlocking counters 4 16 of the interlocking counter 4 16 network, the design of the digital circuit becomes easier.
- all the linked counters 4 16 can supply the same count number over the entire LSI.
- an interlocking signal distribution circuit can be designed such that appropriate signals are simultaneously distributed to the entire LSI.
- the interlocking device that forms the input signal at a positive multiple of the period of the clock signal encodes the count number of the interlocking counter 416 by the signal distribution decoder 421, and outputs the clock signal.
- CLK clock start time
- the signal distribution decoder 421 can measure a time that is a positive multiple of the cycle of the clock signal.
- the timing signals No. 3 and No. 5 are used to represent the clock start time and the clock end time, respectively. Of course, other timing signals may be used.
- the signal distribution latch circuit 422 stores this signal once, and then the signal distribution NOR gate 423 stores the signal distribution flip-flop at the clock start time. Input to circuit 4 2 4.
- the signal distributing flip-flop circuit 424 stores the input signal in synchronization with the clock signal, and is reset at the clock end time.
- the interlocking signal distribution circuit can output the input signal that arrived before the clock start time from the clock start time to the clock end time. If the logic of the input signal is inverted, the interlocked signal distribution circuit can operate normally by adding a NOT gate before the signal distribution latch circuit 422.
- FIG. 72 a timing chart of the interlocking signal distribution circuit shown in FIG. 72 is shown in FIG.
- the input signal (SIGIN) is input
- the timing signal No. 5 is output while the timing signal No. 5 is output while the clock signal (CLK) falls while the timing signal No. 3 is output.
- the output signal (SIGOUT) is output until the time when (CLK) falls. Therefore, no matter where the digital circuit is placed in the LSI, the output signal (SIG OUT) can be reliably input at the rise of the timing signal No. 5.
- Such functions include reset signals, interrupt signals, input / output signals, This is indispensable for incorporating one system LSI without changing the circuit.
- the synchronous oscillation circuit 410 is composed of an oscillation section, a synchronization section, and an initialization NOR gate 402.
- the oscillation section and the synchronization section are divided into two sides, A side and B side, respectively. Be classified. Also, in FIG.
- the synchronous oscillation circuit 410 outputs the synchronization signals SyncAl, Sync A2, Sync A3, SyncA4, Sync B 1, Syn B 1, and Syn B 2 ′ from the four synchronous oscillation circuits 410.
- nc B 3 'and Sync B 4' shall be input.
- the A side oscillation NOR gate 401a, the A side oscillation capacitor 404a, the B side oscillation NOR gate 401b and the B side oscillation capacitor 404b are connected in a ring, and the A side oscillation
- the output and input of NOR gate 401a and B side oscillation NOR gate 401b are connected using A side oscillation resistor 403a and B side oscillation resistor 403b, respectively.
- the resistance values of the A-side oscillation resistor 403a and the B-side oscillation resistor 403b are both R ohms
- the capacitance of the A-side oscillation capacitor 404a and the B-side oscillation capacitor 404b are both C farads.
- the oscillating part generates two clock signals C 1 oc kA and C 1 oc kB and two synchronization signals Sync A 0 ′ and Syn ncBO by performing self-oscillation according to the time constant RC. Can be.
- the A side synchronization latch circuit 405 a and the A side synchronization NOR circuit follow the synchronization signals SyncAl ', Sync A2', Sync A 3 Sync A4 SyncBl ', Sync B 2 SyncB 3' and Sync B4 '.
- the gate 406a, the B-side synchronization latch circuit 405b, and the B-side synchronization NOR gate 406b control the A-side oscillation NOR gate 401a and the B-side oscillation NOR gate 401b, respectively. .
- the synchronization part sets the phase and period of the synchronization signals Sync AO 'and Sync B0' to the synchronization signals SyncAl ', SyncA2', SyncA3 SyncA4 ', SyncB 1, SyncB 2', Sync B 2 ', Sync B Match the phase and period of 3 'and SyncB4 Can be.
- the initialization NOR gate 402 determines the phase of the synchronization signal Sync B0 ′ by controlling the A-side oscillation NOR gate 401 a when the power is turned on.
- NOR gate 406 b for 406 a and B side synchronization
- NOR gate 406 b for B side otherwise.
- the synchronous oscillation circuit 410 shown in FIG. 74 can be implemented by using many semiconductor technologies such as TTL (Transistor-Transistor Logic) and ECL (Emitter Coupled Logic Circuit). However, when an FET (field effect transistor) such as a CMOS (complementary metal oxide semiconductor) is used, the charge stored in the A-side oscillation capacitor 404a and the B-side oscillation capacitor 404b is transferred to the NOR gate. If the current flows to the input all at once, the NOR gate may be destroyed. As shown in FIG. 75, the synchronous oscillation circuit 410 uses an A-side input resistor 407a and a B-side input resistor 407b to avoid this problem.
- TTL Transistor-Transistor Logic
- ECL emitter Coupled Logic Circuit
- the resistance of the A-side input resistance 407a and the B-side input resistance 407b is R0 ohm.
- the resistance value R0 is determined by referring to the power supply voltage, the input characteristics of the NOR gate, the capacitance C, and the like.
- the synchronous oscillation circuit 410 uses a crystal instead of the A-side oscillation capacitor 404a.
- the capacitance C of the B-side oscillation capacitor 404 b needs to be set to a value such that the synchronous oscillation circuit 410 oscillates at about the frequency of the crystal oscillator 408.
- the synchronous oscillation circuit 410 is abbreviated as SOU.
- the synchronization signals Syn c AO ′ and Syn c B0 ′ of each synchronous oscillation circuit 410 are input to the A-side synchronization NOR gate 406a and the B-side synchronization NOR gate 406b of the remaining synchronous oscillation circuits 410. Therefore, the A-side synchronization NOR gate 406a and the B-side synchronization NOR gate 406b may be two-input NOR gates.
- the synchronous oscillation circuit 410 self-oscillates starting from the point when the input voltage of the NOR gate 401a for side oscillation and the NOR gate 401b for B side oscillation reaches the threshold value. As shown in FIG. 79, when the waveforms of the synchronization signals Sync A1 and Sync B 1 are shortened for some reason, the synchronous oscillation circuit 410 operates regardless of the synchronization signals Sync A1 ′ and Sync B 1 ′. Operate. Therefore, there is no influence on the clock signals C 1 ok kA and C 1 ok kB.
- the synchronous oscillation circuit 410 that generates the synchronization signals Sync A1 ′ and SyncB l, converts the synchronization signals Sync A1 ′ and Sync B l ′ into the phases of the synchronization signals Sync AO Syn A2 ′, Sync BO ′ and SyncB2 ′. It works to match. As shown in Fig. 80, if the waveforms of the sync signals Sync A2 'and Sync B2' become longer for some reason, the synchronous oscillation circuit 410 synchronizes the phase of the sync signal Sync b0 '(or SyncAO'). Operates to match the phase of signal Sync B 2 ′ (or Sync A2 ′). Therefore, the period of the clock signals C 1 OkA and C 1 OckB becomes longer in accordance with the period of the synchronization signal Sync B2 ′ (or Sync A2 ′).
- the synchronous oscillator 410 operates so as to match the phase of the synchronization signal Syn c B0 (or Sync AO ') to the phase of the synchronization signal Sync B 2' (or Sync A2 '). Therefore, the periods of the clock signals C1OcKA and C1OcKB become longer in accordance with the period of the synchronization signal SyncB2 '(or SyncA2').
- the timing chart for the case where three synchronous oscillators 410 are connected has been described, but at least one of the three synchronous oscillators 410 includes a crystal oscillator 408.
- the same operation is performed when the oscillation circuit 410 is used.
- the phase of the synchronous oscillation circuit 410 that does not include the crystal oscillator 408 is the same as that of the synchronous oscillation circuit that includes the crystal oscillator 408.
- the length of the waveform of the synchronous oscillation circuit 410 that does not include the crystal oscillator 408 changes preferentially so as to match the phase of 0. Therefore, in the network of the synchronous oscillator circuit 410, if there is one synchronous oscillator circuit 410 including the crystal oscillator 408, the clock frequency of the entire network can be kept constant.
- the synchronous oscillation circuit 410 need not necessarily be connected to all other synchronous oscillation circuits 410 as shown in FIG. 77. Therefore, a case where the synchronous oscillation circuits 410 are regularly arranged will be described below.
- the two-dimensional oscillator circuit network is a network in which synchronous oscillator circuits 410 arranged in a square lattice are connected to each other.
- the number of inputs to the A-side synchronization NOR gate 406a and the B-side synchronization NOR gate 406b is four.
- the inputs of the A-side synchronous NOR gate 406a and the B-side synchronous N ⁇ R gate 406b, which are not connected are pulled down.
- the synchronous oscillating circuits 410 in a square lattice, they can be arranged in a hexagonal lattice as shown in FIG.
- the two-dimensional network can easily supply clock signals with the same phase as compared to the case of distributing clock signals input from the outside. Can be.
- the three-dimensional oscillating circuit network is a network in which a plurality of synchronous oscillating circuits 410 arranged in a square lattice and a hexagonal lattice as described above are superimposed using a three-dimensional LSI technique. Therefore, when arranged in a square lattice, the number of inputs to the A-side synchronization NOR gate 406a and the B-side synchronization NOR gate 406b is 6, and they are arranged in a hexagonal lattice. In this case, the number of inputs to the A-side synchronization NOR gate 406a and the B-side synchronization NOR gate 406b is eight.
- the synchronous oscillation circuit 410 is easily synchronized with each other.
- These three-dimensional oscillating circuit networks consist of systolic arrays and parallel image processing in which data processed by processors 101 arranged in a square lattice and hexagonal lattice at the same time is vertically pipelined. A clock signal having the same phase can be efficiently supplied to the device.
- a network of synchronous oscillators 410 arranged in a square lattice and a hexagonal lattice and digital circuits such as a processor 101 and an arithmetic circuit are arranged in different layers of a three-dimensional LSI.
- the arrangement of the synchronous oscillator circuit 410 does not shift due to the arrangement of the digital circuit, and the signal lines of the synchronization signals Sync AO 'and Sync B 0' do not bypass.
- the synchronous oscillation circuit 410 since the synchronous oscillation circuit 410 is not affected by the noise of the digital circuit, the synchronous oscillation circuit 410 operates stably.
- the digital circuit can input the clock signals C 1 ock A and C 1 ock B from any synchronous oscillation circuit 410 of the synchronous oscillation circuit 410, the digital circuit can be easily designed. Become. So far, the artificial eyeball 1, the visual device 2, the communication device 7, and the image sensor have been described. Therefore, the central control device 221 will be described last.
- the central controller 221 shown in Fig. 85 is divided into three groups, namely, the map text and the number of objects that can be input by the speech synthesizer 226 to the environment map and the arbitrary number of objects output by the visual device 2.
- the group that inserts map text, object count text, and instruction text into image 1 1 1 will be described separately.
- the first group includes environment map input means 301, map Z text conversion means 300, object number input means 303, object number Z text conversion means 304, and map Z object number text storage means 310. 5 is included.
- the environmental map input means 301 inputs this environmental map and sequentially stores it in the buffer. I do.
- the map text conversion means 302 becomes inputtable, the environment map input means 301 outputs environmental data in accordance with the input order.
- the map Z text conversion means 302 inputs the environmental data, it converts the type and position of the object area 141 into a character string (map text).
- the map Z text conversion means 302 converts the position of the object and the change in the position into coordinates that are based on the user's constant viewing direction, that is, the front, with the user's position as the origin.
- the words are changed to words that are easy for the user to understand, such as front, back, right, left, up, down, near, far, approaching, retreating, crossing, passing, and passing.
- the map Z text conversion means 302 converts the position based on the direction and magnification of the moving camera 10 included in the environmental map into coordinates having the origin as described above. And then add it to the map text. Thereby, the map text can represent the position of the same object area 141 with different coordinates.
- the map / object number text holding means 300 sequentially stores the map text converted by the map Z text conversion means 302 in a buffer, and outputs the map text to the audio synthesizing device 226.
- the number-of-objects input means 303 inputs the number of arbitrary objects and sequentially stores the number in the buffer. I do.
- the number of objects Z text conversion means 304 becomes inputtable, the number of objects input means 303 outputs an arbitrary number of objects according to the input order.
- the number of objects Z text conversion means 304 inputs an arbitrary number of objects, it converts the number of object regions 144 into a character string (object number text).
- Map object number text holding means 3 05 is the number of objects Z number of objects converted by text conversion means 3 04 The text is sequentially stored in the buffer and output to the speech synthesizer 226.
- the second group includes a command text input means 306, a text command conversion means 307, and a control command holding means 308.
- a character string instruction text
- the instruction text input means 306 inputs this instruction text and sequentially stores it in a buffer.
- the instruction text input means 306 outputs each instruction text according to the input order.
- the instruction text input means 303 also sequentially stores the control codes assigned to the control buttons 223 in the buffer.
- the text command conversion means 307 receives the command text and converts the command text into a control command that can be input to the visual device 2. At this time, the text / command conversion means 307 converts the command text into coordinates based on the orientation of the moving camera 10, that is, the center of the display 2 22 and the position based on the magnification of the moving camera 10. Then, the relative movement amount of the moving camera 10 from the current position is calculated. Thus, the user can easily control the moving camera 10 while watching the display 222.
- the control command holding means 308 sequentially stores the control commands converted by the text Z command conversion means 307 in a buffer and outputs the control commands to the visual device 2.
- the third group includes digital image receiving means 309, text inserting means 310, and video signal output means 311.
- the digital image receiving means 309 inputs the digital image 111 transmitted by the communication device 7 and outputs it to the text inserting means 310.
- the text insertion means 310 inputs the map text from the map Z text conversion means 310, the object number text from the object number text conversion means 304, or the instruction text input means 310. Each time the instruction text is entered or is, the map text, the number of objects text, and the instruction text are inserted into the digital image 111 using software that implements the super pause function.
- the text inserting means 310 outputs the digital image 111 to the video signal output means 311 irrespective of the presence or absence of the map text, the object number text and the instruction text.
- the video signal output means 3 1 1 inputs the digital image 1 1 1 from the text insertion means 3 1 0 Then, it converts it to a video signal of NTSC, PAL, HDTV, VGA, MPEG format, etc., and outputs it to the display 222.
- the artificial eyeball 1 can transmit the type of the object photographed by the mobile camera 10 to the user terminal 4, so that the user can It is possible to know that a specific type of object exists around the human eyeball 1. Therefore, a plurality of human eyeballs 1 are arranged around and inside vehicles, ships, houses, buildings, space stations, and the like, and one or more user terminals 4 are installed in a specific room, or By wearing it, the user can detect an object approaching the artificial eyeball 1 or monitor a person moving around the artificial eyeball 1. In particular, by using the mobile phone 211 and the portable information terminal as the user terminal 4, the user can grasp the situation of the vehicle, the ship, the house, the building, the space station, etc. even when going out and in the bedroom.
- the human eyeball 1 by arranging the human eyeball 1 around the vehicle, the driver can know the existence of an object located in a blind spot, and can simultaneously grasp the positions of a plurality of objects.
- the artificial eyeball 1 by arranging the artificial eyeball 1 so as to capture the inside of the vehicle, the user can prevent an accident such as leaving the baby in the vehicle and causing death.
- the artificial eyeball 1 can transmit an image of the suspicious person to the user terminal 4, so that the user can respond quickly regardless of where they are. By recording this image, you can provide evidence of vehicle damage at a later date.
- the observer operating the monitoring terminal 5 can always use the artificial eye 1 By monitoring the surroundings, the user of the user terminal 4 is relieved of the trouble of monitoring the surroundings of the artificial eyeball 1.
- the artificial eyeball 1 can notify the monitoring terminal 5 of the approaching object, so that one observer can manage many artificial eyes 1. Therefore, security companies and corporate security departments can monitor many locations with a small number of guards.
- the artificial eyeball 1 can transmit the type of the object photographed by the moving force lens 10 to the data base 6, so that the user Through this, it is possible to know when, where and how many objects of a particular type existed. Therefore, by arranging a plurality of artificial eyeballs 1 around and inside roads, shopping streets, stores and stations, users can check the traffic volume of people and vehicles, and furthermore, And the traffic volume of vehicles can be predicted. That is, the store manager can adjust the production and stock of the product by checking the traffic volume in front of the store and the number of customers visiting the store from the database 6. This effect is especially noticeable when tf ⁇ overnight is combined with the weather forecast.
- the vehicle driver can make an operation plan to avoid traffic congestion by examining the degree of congestion on the road from Day 6.
- the observer operating the monitoring terminal 5 constantly monitors the database 6, so that the user of the user terminal 4 monitors the database 6. You will be released from the hassle of doing it.
- the observer since the observer accumulates know-how to analyze the statistical data in the database 6, the user does not need any special knowledge to analyze the statistical data. Therefore, organizations such as franchise chains can exhibit stores quickly and efficiently.
- the visual device 2 controls the moving camera 10 so that the moving camera 10 captures an object at a high resolution, and the visual device 2 identifies the type of the object, and Locate your body. Therefore, by mounting the present invention, the pet mouth pot can move toward the searched person. Further, if the pet robot is equipped with a device and a program for determining the face of a person, the pet mouth pot can easily determine whether or not this person is the owner. According to the invention described in claim 5, since the visual device 2 controls the moving camera 10 so that the moving camera 10 automatically tracks the object, the remote user and the computer constantly control the moving camera 10. do not have to. Moreover, if the object is a human, this human does not have to constantly check that he or she is properly reflected.
- the user and the computer at the remote place can do other work in the spare time, so that the user and the computer can work efficiently and the communication device 7 does not need to constantly transmit the control command.
- the report and interview can be taken alone and transmitted directly to a broadcasting station. That is, when an individual distributes a moving image on the Internet, the present invention is a useful photographing device.
- the communication device 7 transmits only the digital image 111 and the mask image 148 in which the visual device 2 determines whether or not an object is present in the moving image captured by the moving camera 10. Therefore, even the communication device 7 having a low communication speed can efficiently transmit a moving image in which an object is reflected.
- the present invention can effectively use a medium having a limited communication line capacity such as a mobile phone 211.
- the visual device 2 can easily extract a face as the object area 14 1, the user can use the present invention as a personal authentication device by connecting the mobile phone 2 1 1 to a computer system at a remote location.
- the present invention protects the user's privacy and allows the user to enjoy the present invention for taking a photo sticker.
- the communication device 7 transmits the digital image 111 and the mask image 148 only when the visual device 2 recognizes an object, by installing the present invention in offices, houses, factories, roads, vehicles, and the like, Users and security companies can also use the present invention as an inexpensive monitoring device.
- the movable camera 10 performs pan, tilt, roll, and zoom, thereby providing the artificial eyeball 1.
- the visual device 2 automatically searches for objects, it is possible to detect objects approaching the user in advance, such as pedestrians 3 2 4, bicycles and vehicles 3 2 1, so that users can safely You can walk. Since the visual device 2 provides the user with information useful for the user to recognize the object, the present invention can reduce the burden on the user as compared with a case where a CCD camera is simply used.
- the visual device 2 controls the moving camera 10 using the electrical stimulation. be able to.
- the moving camera 10 uses multi-freedom motion as a mechanism for controlling pan, tilt, and roll
- the multi-degree of freedom motion has a spherical shape close to the shape of an actual eyeball
- the artificial eyeball 1 Can be easily embedded in the orbit.
- the multi-degree-of-freedom mode is an ultrasonic mode and a piezoelectric actuator type using a piezoelectric element
- the multi-degree-of-freedom mode can achieve space saving, low weight, and low power consumption.
- the present invention can reduce the burden on the user.
- the present invention can realize high-speed image processing while saving space and consuming low power.
- the present invention can facilitate the maintenance of the artificial eyeball 1 while reducing the burden on the user.
- the position Z size Z tilt detecting means 44 performs processing in parallel on a pixel-by-pixel basis, so that the formation edge information image 115, the object area image 144, and the color information
- the position, size, and inclination of the object can be detected from the image 172.
- the position / Z magnitude / inclination detection means 44 can obtain the inclination of the object.
- Position size / tilt detection means 4 4 detects multiple objects from a frame image of a moving image captured by a video camera or a still image captured by a digital camera or captured by a scanner.
- the position Z size Z inclination detecting means 44 can be used also as preprocessing for identifying the texture. Since the array operation unit 100 does not output eight source overlap information 18 3, the system that inputs the overlap information image 1 3 2 and the tilt angle from the position / size Z tilt detection means 4 4 The mechanism can be simplified.
- the array operation unit 100 can transfer a large amount of data to the nearby array operation unit 100 in a short time. Therefore, when a large number of data must be transferred to the vicinity as in the source duplicate information image 184, the present invention realizes the invention of claim 8 without deteriorating the performance due to the communication bottleneck. Can be.
- the present invention can also efficiently execute these local parallel image processing even when several local parallel image processings are executed simultaneously.
- each controller 103 has the previous input data register 61 and the result data register 62, the array operation unit 100 can input and output a plurality of data simultaneously.
- the present invention is effective for an edge information forming apparatus and an area normalizing apparatus that execute local parallel image processing using a plurality of images.
- the LSI designer can implement a three-dimensional LSI. Bumps can be used effectively.
- the array operation unit 100 can transfer a large amount of data to the nearby array operation unit 100 in a short time. Therefore, when a large number of data must be transferred to the vicinity as in the source duplicate information image 18 4, the present invention can realize the invention according to claim 8 without deteriorating the performance due to the bottleneck of communication. it can.
- the present invention can also efficiently execute these local parallel image processing even when several local parallel image processings are performed simultaneously.
- the present invention implements the controller 103 in an independent layer, and since the controller 103 communicates with the processor 101 and the memory 102 via the address bus 51 and the data bus 52, an LSI Designer can use controller 10 if necessary Only by increasing the number of layers on which 3 is mounted can the communication performance between the array operation units 100 be improved. By combining the inventions of claims 9 and 10, the LSI can integrate the controller 103 with high density.
- the visual device 2 when the visual device 2 processes a moving image having a fixed number of pixels, the visual device 2 performs a smaller number of virtual array operations than in the case where the array operation unit 100 is used. Only unit 105 is required. In other words, if the number of array operation units 100 that can be implemented by conventional LSI (large-scale integrated circuit) technology is less than the number of pixels in a moving image, the LSI designer must use the virtual array operation unit 105. Thereby, the visual device 2 can be mounted on an LSI.
- LSI large-scale integrated circuit
- the virtual array operation unit 105 Since it is sufficient to refer to these calculation data in the memory 102, the LSI designer can determine the number of array operation units 100 to be allocated to the virtual array operation unit 105 by visual determination. The device 2 can be operated at high speed. Moreover, since the array operation unit 100 can share the program and the look-up table in the virtual array operation unit 105, the data processor 110 can reduce the consumption of the memory 102 as a whole. Can be.
- the manufacturers of the CCD (charge coupled device) and CMOS (complementary metal oxide semiconductor) image sensor are LSIs, CCDs owned by the manufacturer. And a high-performance image sensor can be manufactured using the CMOS manufacturing technology as it is.
- a manufacturer can manufacture a high-performance image sensor in a short period of time and at a low cost, so that users of the present invention can use a house, a convenience store, a supermarket, an office, a factory, a general road, a parking lot, a station,
- users of the present invention can use a house, a convenience store, a supermarket, an office, a factory, a general road, a parking lot, a station,
- High-performance image sensors can be used.
- the designer of the CCD and CMOS image sensors only needs to design one functional column 201 and arrange it in a two-dimensional grid pattern. It is possible to easily design a high-performance image sensor capable of capturing an image of an appropriate size. Therefore, these designers can design high-performance image sensors in a short period of time and at low cost.
- the interlocking counters 4 16 are distributed and arranged throughout the LSI so that the propagation delay time falls within the cycle of the clock signal.
- the number of counts of all interlocking counts 4 16 can be matched.
- the clock signal is synchronized throughout the LSI even if the frequency of the clock signal is high, so that all the interlocking counters 4 16 are synchronized. be able to.
- the interlocking counter 4 16 and the synchronous oscillation circuit 4 10 are easily separated from the other digital circuits.
- the present invention relates to a parallel processing system, such as a pipeline processing device, a DSP (Digital Signal Processor 1), a systolic array, a data flow machine, and a parallel image processing device.
- a parallel processing system such as a pipeline processing device, a DSP (Digital Signal Processor 1), a systolic array, a data flow machine, and a parallel image processing device.
- the digital circuit arranged in the entire LSI can simultaneously receive signals transmitted from an arbitrary place of the LSI.
- the higher the frequency of the clock signal the more the individual functions are adjusted so that the timing of the reset signal, interrupt signal, and input / output signal match.
- Changing the design of a block becomes difficult.
- the timing of the reset signal, the interrupt signal, and the input / output signal can be controlled only by considering the maximum propagation delay time regardless of the arrangement of the individual functional blocks. Will replace these functional blocks in one without changing the design of these functional blocks in any way. It can be implemented in LSI.
- processors 101 input the same instruction, such as an SIMD (Single Instruction on Stream Mu1ti DataStream) type multi-processor
- SIMD Single Instruction on Stream Mu1ti DataStream
- All the processors 101 must operate at the same timing, even though the propagation delay time of the signal from the processor 101 to each processor 101 is different.
- an instruction can be simultaneously supplied to all the processors 101 regardless of the clock frequency, so that the LSI designer can easily design the processor 101.
- bipolar semiconductors such as TTL (Tran sister-Tran sister Logic) and ECL (Emitsu coupling logic circuit), and CMOS (Complementary metal oxide semiconductor) Regardless of semiconductor manufacturing technology such as)
- the digital circuits of the entire LSI Large Scale Integrated Circuit
- the clock frequency has been increased due to the speeding up of digital circuits, and it has become difficult to maintain the synchronization of the entire LSI. Therefore, by using the present invention, a semiconductor maker can easily design the processor 101 and the measurement device that require high-speed operation.
- some of the synchronous oscillators 410 of the interlocking device input a synchronizing signal from a stable oscillation circuit such as the crystal oscillator 408, thereby adjusting the clock frequency of the entire interlocking device to the frequency of the crystal oscillator 408.
- a stable oscillation circuit such as the crystal oscillator 408
- the present invention relates to an overall parallel system, such as a pipeline processing device, a DSP (Digital Signal Processor), a systolic array, a data flow machine, and a parallel image processing device. Since a synchronized clock signal can be supplied, LSI designers can design large-scale parallel systems while avoiding the problem of delay time.
- DSP Digital Signal Processor
Description
Claims
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
AU15521/01A AU1552101A (en) | 1999-11-30 | 2000-11-24 | Data acquisition system, artificial eye, vision device, image sensor and associated device |
EP00977947A EP1248465A4 (en) | 1999-11-30 | 2000-11-24 | DATA COLLECTION SYSTEM, ARTIFICIAL EYE, VISIBILITY, IMAGE SENSOR AND ASSOCIATED DEVICE |
Applications Claiming Priority (20)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP33936099A JP2001157196A (ja) | 1999-11-30 | 1999-11-30 | 警戒装置及び警戒方法 |
JP11/339360 | 1999-11-30 | ||
JP11/369220 | 1999-12-27 | ||
JP36922099A JP2001184498A (ja) | 1999-12-27 | 1999-12-27 | イメージセンサー |
JP2000021315A JP2001204757A (ja) | 2000-01-31 | 2000-01-31 | 人工眼球 |
JP2000/21315 | 2000-01-31 | ||
JP2000/64544 | 2000-03-09 | ||
JP2000064544A JP2001256471A (ja) | 2000-03-09 | 2000-03-09 | 視覚装置 |
JP2000/68155 | 2000-03-13 | ||
JP2000068155A JP2001256477A (ja) | 2000-03-13 | 2000-03-13 | 情報収集装置 |
JP2000/111675 | 2000-04-13 | ||
JP2000111675 | 2000-04-13 | ||
JP2000/138125 | 2000-05-11 | ||
JP2000138125 | 2000-05-11 | ||
JP2000/174616 | 2000-06-12 | ||
JP2000174616A JP2001359082A (ja) | 2000-06-12 | 2000-06-12 | 人工眼球 |
JP2000/236461 | 2000-08-04 | ||
JP2000236461 | 2000-08-04 | ||
JP2000/322422 | 2000-10-23 | ||
JP2000322422 | 2000-10-23 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2001041448A1 true WO2001041448A1 (fr) | 2001-06-07 |
Family
ID=27580511
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2000/008334 WO2001041448A1 (fr) | 1999-11-30 | 2000-11-24 | Systeme d'acquisition de donnees, oeil artificiel, dispositif de vision, capteur d'images, et appareil associe |
Country Status (4)
Country | Link |
---|---|
EP (1) | EP1248465A4 (ja) |
CN (1) | CN100355284C (ja) |
AU (1) | AU1552101A (ja) |
WO (1) | WO2001041448A1 (ja) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
GB2379309A (en) * | 2001-07-31 | 2003-03-05 | Hewlett Packard Co | Recognition and identification apparatus |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DK2202609T3 (en) * | 2004-06-18 | 2016-04-25 | Tobii Ab | Eye control of computer equipment |
SE529156C2 (sv) * | 2005-10-28 | 2007-05-15 | Tobii Technology Ab | Ögonföljare med visuell återkoppling |
CN106954026B (zh) * | 2017-04-27 | 2019-12-31 | 盯盯拍(深圳)技术股份有限公司 | 拍摄方法以及触发监控装置 |
Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPS63174356A (ja) * | 1987-01-14 | 1988-07-18 | Agency Of Ind Science & Technol | 画像処理用半導体装置 |
JPH0235425A (ja) * | 1988-07-26 | 1990-02-06 | Matsushita Electric Ind Co Ltd | 半導体積層集積回路素子 |
JPH052399A (ja) * | 1991-06-25 | 1993-01-08 | Toshiba Corp | 音声認識装置 |
JPH0553595A (ja) * | 1991-08-29 | 1993-03-05 | Canon Inc | 音声合成装置 |
JPH05324954A (ja) * | 1991-11-28 | 1993-12-10 | Nec Corp | 駐車台数計数装置 |
JPH07135293A (ja) * | 1990-04-25 | 1995-05-23 | Mitsubishi Electric Corp | 積層型三次元半導体集積回路 |
JPH0997337A (ja) * | 1995-09-29 | 1997-04-08 | Fuji Heavy Ind Ltd | 侵入物監視装置 |
JPH10136345A (ja) * | 1996-11-01 | 1998-05-22 | Matsushita Electric Ind Co Ltd | 情報提供システム |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4908751A (en) * | 1987-10-15 | 1990-03-13 | Smith Harry F | Parallel data processor |
JPH07234447A (ja) * | 1993-12-28 | 1995-09-05 | Canon Inc | 撮影ユニットおよびカメラ |
JP3658893B2 (ja) * | 1996-11-15 | 2005-06-08 | カシオ計算機株式会社 | 宅配荷物管理装置 |
AU2565699A (en) * | 1998-01-29 | 1999-08-16 | Internet Monitoring, Inc. | Networked security system for network-based monitoring and control of an environment |
JP4532610B2 (ja) * | 1998-01-30 | 2010-08-25 | キヤノン株式会社 | カメラ制御システムおよびその方法およびその動作処理を実行するプログラムを記憶した記憶媒体 |
-
2000
- 2000-11-24 CN CNB008147329A patent/CN100355284C/zh not_active Expired - Fee Related
- 2000-11-24 AU AU15521/01A patent/AU1552101A/en not_active Abandoned
- 2000-11-24 EP EP00977947A patent/EP1248465A4/en not_active Withdrawn
- 2000-11-24 WO PCT/JP2000/008334 patent/WO2001041448A1/ja not_active Application Discontinuation
Patent Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPS63174356A (ja) * | 1987-01-14 | 1988-07-18 | Agency Of Ind Science & Technol | 画像処理用半導体装置 |
JPH0235425A (ja) * | 1988-07-26 | 1990-02-06 | Matsushita Electric Ind Co Ltd | 半導体積層集積回路素子 |
JPH07135293A (ja) * | 1990-04-25 | 1995-05-23 | Mitsubishi Electric Corp | 積層型三次元半導体集積回路 |
JPH052399A (ja) * | 1991-06-25 | 1993-01-08 | Toshiba Corp | 音声認識装置 |
JPH0553595A (ja) * | 1991-08-29 | 1993-03-05 | Canon Inc | 音声合成装置 |
JPH05324954A (ja) * | 1991-11-28 | 1993-12-10 | Nec Corp | 駐車台数計数装置 |
JPH0997337A (ja) * | 1995-09-29 | 1997-04-08 | Fuji Heavy Ind Ltd | 侵入物監視装置 |
JPH10136345A (ja) * | 1996-11-01 | 1998-05-22 | Matsushita Electric Ind Co Ltd | 情報提供システム |
Non-Patent Citations (1)
Title |
---|
See also references of EP1248465A4 * |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
GB2379309A (en) * | 2001-07-31 | 2003-03-05 | Hewlett Packard Co | Recognition and identification apparatus |
GB2379309B (en) * | 2001-07-31 | 2003-10-08 | Hewlett Packard Co | Recognition and identification apparatus |
Also Published As
Publication number | Publication date |
---|---|
CN1382346A (zh) | 2002-11-27 |
EP1248465A4 (en) | 2005-06-22 |
CN100355284C (zh) | 2007-12-12 |
EP1248465A1 (en) | 2002-10-09 |
AU1552101A (en) | 2001-06-12 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10547812B2 (en) | Video capture device and method | |
US7664220B2 (en) | Visual device, interlocking counter, and image sensor | |
JP4324327B2 (ja) | 視覚装置 | |
US20140300758A1 (en) | Video processing systems and methods | |
AU2022246412A1 (en) | A method and apparatus for conducting surveillance | |
JP2000512790A (ja) | データ処理システムおよび方法 | |
CN106210547A (zh) | 一种全景拍摄的方法、装置及系统 | |
CN102833525A (zh) | 一种360度全景视频的浏览操作方法 | |
WO2018129794A1 (zh) | 一种实时大规模场景三维扫描建模方法及系统 | |
Yong et al. | Panoramic background image generation for PTZ cameras | |
Fawzi et al. | Embedded real-time video surveillance system based on multi-sensor and visual tracking | |
CN106060658B (zh) | 一种图像处理方法及装置 | |
CN110674918A (zh) | 信息处理方法、设备、系统及存储介质 | |
WO2001041448A1 (fr) | Systeme d'acquisition de donnees, oeil artificiel, dispositif de vision, capteur d'images, et appareil associe | |
JP3963424B2 (ja) | 視覚装置 | |
CN108184062A (zh) | 基于多层次异构并行处理的高速追踪系统及方法 | |
JP2001204757A (ja) | 人工眼球 | |
CN100442819C (zh) | 互锁装置 | |
TW498290B (en) | Information collection system, mechanical eyeball, vision device, image sensor, and interlocking device | |
Choi et al. | Real-time panorama video system using networked multiple cameras | |
Gentile et al. | Real-time image processing on a local plane SIMD array | |
Wolf et al. | Architectures for distributed smart cameras | |
JP2000341568A (ja) | 撮像装置、撮像システム及び撮像方法 | |
JP2001256477A (ja) | 情報収集装置 | |
CN116580169B (zh) | 一种数字人驱动方法及装置、电子设备和存储介质 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AK | Designated states |
Kind code of ref document: A1 Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BY BZ CA CH CN CR CU CZ DE DK DM DZ EE ES FI GB GD GE GH GM HR HU ID IL IN IS KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NO NZ PL PT RO RU SD SE SG SI SK SL TJ TM TR TT TZ UA UG US UZ VN YU ZA ZW |
|
AL | Designated countries for regional patents |
Kind code of ref document: A1 Designated state(s): GH GM KE LS MW MZ SD SL SZ TZ UG ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE TR BF BJ CF CG CI CM GA GN GW ML MR NE SN TD TG |
|
DFPE | Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101) | ||
121 | Ep: the epo has been informed by wipo that ep was designated in this application | ||
WWE | Wipo information: entry into national phase |
Ref document number: 008147329 Country of ref document: CN |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2000977947 Country of ref document: EP |
|
WWE | Wipo information: entry into national phase |
Ref document number: 10148399 Country of ref document: US |
|
WWP | Wipo information: published in national office |
Ref document number: 2000977947 Country of ref document: EP |
|
REG | Reference to national code |
Ref country code: DE Ref legal event code: 8642 |
|
WWW | Wipo information: withdrawn in national office |
Ref document number: 2000977947 Country of ref document: EP |