CN114143513A - Multi-dimensional feature acquisition device and image acquisition method - Google Patents
Multi-dimensional feature acquisition device and image acquisition method Download PDFInfo
- Publication number
- CN114143513A CN114143513A CN202111632461.7A CN202111632461A CN114143513A CN 114143513 A CN114143513 A CN 114143513A CN 202111632461 A CN202111632461 A CN 202111632461A CN 114143513 A CN114143513 A CN 114143513A
- Authority
- CN
- China
- Prior art keywords
- module
- image
- deserializing
- signals
- signal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/04—Synchronising
- H04N5/06—Generation of synchronising signals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/2624—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects for obtaining an image which is composed of whole input images, e.g. splitscreen
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/268—Signal distribution or switching
Abstract
The invention provides a multidimensional characteristic acquisition device and an image acquisition method, wherein the multidimensional characteristic acquisition device comprises a main control module, a deserializing module and at least two image acquisition modules, each image acquisition module comprises an image sensor and a serialization unit connected with the image sensor, the main control module is connected with the deserializing module, the deserializing module is connected with each serialization unit, each serialization unit is used for serially coding a plurality of paths of image frame signals into a single path of high-frequency signals and transmitting the single path of high-frequency signals to the deserializing module, and the multidimensional characteristic acquisition device controls a plurality of image acquisition modules to synchronously expose through synchronous control signals while realizing distributed deployment.
Description
Technical Field
The invention relates to the technical field of multi-dimensional feature acquisition, in particular to a multi-dimensional feature acquisition device and an image acquisition method.
Background
With the development of the video monitoring industry, scenes faced by video monitoring are more and more diversified and complicated. Many scenes need to monitor multiple directions or multiple viewing angles, and even some scenes need to monitor multiple targets, such as motor vehicles, non-motor vehicles, pedestrians, and the like, for example, some emerging biometric identification technologies in the security industry have strong dependency on multi-dimensional information of the targets, typically gait identification which is identified as a future development trend by a public security system. Therefore, the human body characteristic collection of single dimension can not meet the requirements of users, and the collection side can provide multi-azimuth and multi-angle target materials according to the current urgent needs so as to meet the requirements of complex functions and improve the stability and reliability of artificial intelligence.
In the conventional multi-dimensional feature acquisition scheme, when multi-dimensional feature acquisition is realized by deploying a plurality of cameras in different scenes, because camera lens expansion is performed by means of a network, and the network has time delay, clock synchronization of data acquired among the lenses is difficult to realize, acquisition of multi-dimensional features by an integrated multi-view camera does not have the capability of expandability and flexible deployment, and when the lenses are required to be deployed in a 'surrounding manner' from the periphery of a target, the integrally formed multi-view camera cannot realize distributed deployment.
Disclosure of Invention
The invention provides a multi-dimensional feature acquisition device, which is used for solving the problems that clock synchronization is difficult to realize when a plurality of independent cameras are adopted and distributed deployment cannot be realized when a multi-view integrated camera is adopted in the traditional multi-dimensional feature acquisition scheme.
In a first aspect, an embodiment of the present invention provides a multi-dimensional feature collecting device, including a main control module, a deserializing module, and at least two image collecting modules, where each image collecting module includes an image sensor and a serializing unit connected to the image sensor, the main control module is connected to the deserializing module, and the deserializing module is connected to each serializing unit, where:
the image sensors in the image acquisition modules are used for acquiring image sensing signals, processing the image sensing signals into a plurality of paths of image frame signals according to a protocol and transmitting the image frame signals to the connected serialization units;
each serialization unit is used for serially encoding the multi-channel image frame signals into a single-channel high-frequency signal and transmitting the single-channel high-frequency signal to the deserializing module;
the deserializing module is used for processing the single-path high-frequency signals received from a plurality of serializing units in parallel: and deserializing the single-path high-frequency signal into corresponding multi-path image frame signals, and transmitting the obtained multi-path image frame signals to the main control module in parallel.
An optional implementation manner is that, the multi-dimensional feature acquisition apparatus provided in the embodiment of the present invention further includes a reference clock and a synchronization signal generator, where:
the synchronization signal generator generates a synchronization control signal based on a clock signal of the reference clock;
the deserializing module is also used for respectively transmitting the synchronous control signals to each serializing unit;
the serialization unit is also used for transmitting the received synchronous control signal to the image sensor and controlling the exposure of the image sensor.
In an optional embodiment, the reference clock and the synchronization signal generator are integrated with the master control module; or
The reference clock is integrated with the main control module, and the synchronous signal generator is integrated with the main control module; or
The reference clock and the synchronization signal generator are integrated with the deserializing module.
In an optional implementation manner, an image sensor in the image acquisition module is connected with the serialization unit through an MIPI bus interface;
each serialization unit is connected with the deserializing module in a wired or wireless mode, wherein the wired mode is a coaxial cable or a shielded twisted pair;
the deserializing module is connected with the main control module through an MIPI flat cable interface.
In an optional embodiment, each serialization unit is connected to the deserializing module through a coaxial cable, and each serialization unit is connected to a corresponding BNC interface in the deserializing module through a BNC interface.
In an optional implementation manner, each serialization unit is configured to encode the multiple image frame signals into a single high-frequency signal in a serial manner, modulate the single high-frequency signal to a forward channel, and transmit the modulated single high-frequency signal to the deserializing module;
the deserializing module is used for processing the single-path high-frequency signals received from a plurality of serializing units in parallel: separating the single-path high-frequency signal from a forward channel, deserializing the single-path high-frequency signal into corresponding multi-path image frame signals, and transmitting the obtained multi-path image frame signals to the main control module in parallel;
the deserializing module is also used for modulating the synchronous control signal to a back channel and then respectively transmitting the synchronous control signal to each serializing unit;
the serialization unit is also used for separating a synchronization control signal from the back channel and transmitting the synchronization control signal to the image sensor.
In an optional implementation manner, the serialization unit is configured to number the multiple image frame signals according to a protocol specification, arrange the multiple image frame signals into a single high-frequency signal according to a numbering sequence, and transmit the single high-frequency signal to the deserializing module.
In a second aspect, an embodiment of the present invention further provides an image acquisition method, where the method includes:
respectively acquiring image sensing signals through each image sensor, processing the image sensing signals into a plurality of paths of image frame signals according to a protocol, and transmitting the image frame signals to a serialization unit connected with the image sensors;
the method comprises the steps that multiple paths of image frame signals output by an image sensor connected with a serialization unit are subjected to serial coding through the serialization units, and single-path high-frequency signals are obtained and transmitted to a deserializing module;
and the deserializing module is used for processing the received single-path high-frequency signals output by the serializing units in parallel: and deserializing the single-path high-frequency signal into corresponding multi-path image frame signals, and transmitting the obtained multi-path image frame signals to the main control module in parallel.
An optional embodiment is that, before the image sensor respectively collects the image sensing signals, the method further includes:
generating a clock signal by a reference clock and transmitting the clock signal to a synchronous signal generator;
the synchronization signal generator generates a synchronization control signal based on the clock signal;
transmitting the synchronous control signal to each serialization unit through a deserializing module;
and transmitting the synchronous control signal to an image sensor connected with each serialization unit through each serialization unit, and controlling the exposure of the image sensor.
An optional embodiment is that, a plurality of image frame signals output by an image sensor connected to each serialization unit are serially encoded by each serialization unit to obtain a single high-frequency signal, and the single high-frequency signal is transmitted to the deserializing module, and the method includes:
the image processing method comprises the steps that multiple paths of image frame signals output by an image sensor connected with a serialization unit are numbered through the serialization unit according to protocol regulations, and the multiple paths of image frame signals are arranged into single-path high-frequency signals according to the numbering sequence and are transmitted to a deserializing module.
The multi-dimensional feature acquisition device provided by the embodiment of the invention has the advantages that the plurality of image acquisition modules can be flexibly distributed and deployed, and meanwhile, synchronous exposure can be carried out. The device not only has the flexibility of a common single camera, but also has the cooperative ability among all lenses of a multi-view integrated camera. Based on this multi-dimensional characteristic collection system, can acquire human multi-angle information simultaneously, have very big help to each item intelligence application in the machine vision.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.
Fig. 1 is an overall schematic diagram of a multi-dimensional feature acquisition apparatus according to an embodiment of the present invention;
fig. 2 is a detailed schematic diagram of a multi-dimensional feature acquisition apparatus according to an embodiment of the present invention;
FIG. 3 is a timing diagram of signals in a synchronous exposure according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a synchronous exposure apparatus according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of a synchronous exposure apparatus according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of a synchronous exposure apparatus according to an embodiment of the present invention;
fig. 7 is a flowchart illustrating a multi-dimensional feature collecting apparatus according to an embodiment of the present invention;
fig. 8 is a flowchart of an image capturing method according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the present invention will be described in further detail with reference to the accompanying drawings, and it is apparent that the described embodiments are only a part of the embodiments of the present invention, not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The security industry has obtained very big development in all aspects under the support of technologies such as AI, big data, its function is more diversified, complicated, humanized. However, at present, the higher demand on the artificial intelligence and the bottleneck encountered by the current intelligent algorithm become one of the main contradictions in the security industry, which puts higher requirements on the acquisition side, and more complex functions require more data, and more data not only refer to the number, but also include the dimension, especially some emerging biological feature recognition technologies have stronger dependence on target multi-dimensional information, and typically are gait recognition which is recognized as the future development trend by a public security system.
Therefore, the single-dimensional human body feature collection can not meet the requirements of security users, and the collection side needs to collect multi-directional and multi-angle target materials to meet the requirements of complex functions. In order to collect the multidimensional characteristics of the target material, the following two schemes are generally adopted in the current industry:
the first scheme is as follows: a plurality of cameras are deployed in different scenes to acquire multi-dimensional features, and cameras in corresponding scenes are added by means of network ports of the switches. The switch is provided with a plurality of network ports, so the scheme has expandability, the cameras with corresponding number can be expanded according to the specific scene requirements, and each camera is an independent individual and can be flexibly installed and deployed; however, this solution has a drawback that: because the camera lens is expanded by the aid of the network, the network has delay, and the network delay generally reaches at least dozens of milliseconds, the synchronous exposure of the data acquired among a plurality of lenses is difficult to realize, and thus the related services (such as acquisition of multi-dimensional human body features) of the same space-time scene are completed.
Scheme II: and collecting the multi-dimensional features through a multi-view integrated camera. Because every camera lens of many meshes integration camera in this scheme is direct to be connected with main control unit, consequently do not receive the influence of network delay, the cooperation that can be fine between the camera lens is worked, nevertheless has a shortcoming: the number of supported lenses is determined when the lenses are delivered from a factory, and the expandability and the flexible deployment capability are not provided. When the lens is required to be deployed in a 'surrounding manner' from the periphery of a target for multi-dimensional image feature acquisition, the multi-view integrated camera cannot realize distributed deployment because the lens is limited in the integrated camera body.
The embodiment of the invention provides a multi-dimensional feature acquisition device aiming at the problems existing in the two schemes, and as shown in fig. 1, the multi-dimensional feature acquisition device comprises a main control module 101, a deserializing module 102 and at least two image acquisition modules 103, wherein each image acquisition module comprises an image sensor and a serialization unit connected with the image sensor, the main control module is connected with the deserializing module, and the deserializing module is connected with each serialization unit.
The main control module is connected with the deserializing module through a double-edge connector, and the double-edge connector is a user-defined socket used for connection.
The main control module consists of an SOC main chip and a peripheral circuit thereof, and the deserializing module consists of a deserializing chip.
The image sensors in the image acquisition modules are used for acquiring image sensing signals, processing the image sensing signals into a plurality of paths of image frame signals according to a protocol and transmitting the image frame signals to the connected serialization units;
each serialization unit is used for serially encoding the multi-channel image frame signals into a single-channel high-frequency signal and transmitting the single-channel high-frequency signal to the deserializing module;
the deserializing module is used for processing the single-path high-frequency signals received from a plurality of serializing units in parallel: and deserializing the single-path high-frequency signal into corresponding multi-path image frame signals, and transmitting the obtained multi-path image frame signals to the main control module in parallel.
In the embodiment of the invention, a plurality of image sensors do not adopt an integrated structure like a multi-view integrated camera, but adopt a split structure, each image sensor is connected with one serialization unit to form an image acquisition module, the plurality of image acquisition modules are mutually independent and can be deployed at different positions in practical application, and the serialization units in the image acquisition modules are connected to a deserializing module through long-line cables and perform signal transmission with a main control module through the deserializing module.
The multi-path image frame signals are usually four paths, and after the four paths of image frame signals are serially coded into a single path of signal by the serialization unit, the transmission in a longer distance can be supported, so that in actual deployment, a longer distance can be formed between each image acquisition module and the deserializing module, and in the embodiment of the invention, the single path of high-frequency signal obtained after the serialization modulation can support the long-distance transmission of 15-20 meters.
As an optional implementation manner, the serializing unit is configured to number the multiple image frame signals according to a protocol specification, arrange the multiple image frame signals into a single high-frequency signal according to a numbering sequence, and transmit the single high-frequency signal to the deserializing module.
For example, after the image sensor transmits four image frame signals to the serialization unit connected with the image sensor, the serialization unit respectively numbers the four image frame signals as A1, B1, C1 and D1, arranges the four image frame signals into a single high-frequency signal according to the sequence of A1-B1-C1-D1, and transmits the single high-frequency signal to the deserializing module, and after the four image frame signals of the next frame are transmitted to the serialization unit, the serialization unit processes the four image frame signals according to the same steps.
As an optional implementation manner, each serialization unit serially encodes the multiple image frame signals into a single high-frequency signal, modulates the single high-frequency signal to a forward channel, and transmits the modulated single high-frequency signal to the deserializing module;
because the single-path high-frequency signals serially modulated by the serialization unit do not meet the data format requirement of the interface of the main control module, the deserializing module is required to deserialize each single-path high-frequency signal into corresponding multi-path image frame signals meeting the format requirement, and then the multi-path image frame signals are transmitted to the main control module.
As an optional implementation manner, the deserializing module separates the single high-frequency signal from the forward channel, deserializes the single high-frequency signal into corresponding multiple image frame signals;
the parallel processing of the single-path high-frequency signals received from the plurality of serialization units is that the deserialization and the transmission of each single-path high-frequency signal are independently carried out, and the deserialization and the transmission of any two single-path high-frequency signals are not interfered with each other.
The main control module is used for receiving a plurality of multipath image frame signals sent by the deserializing module through a plurality of interfaces in parallel and sending the signals to the remote server.
The number of the image acquisition modules can be determined according to the acquisition requirements of an actual scene, and the signals transmitted from the image acquisition modules support long-distance transmission, so that the multi-dimensional feature acquisition device provided by the embodiment of the invention can realize distributed deployment.
The specific connection manner between the modules in the embodiment of the present invention is shown in fig. 2, fig. 2 only describes the case where the serialization unit and the deserialization module are connected through a coaxial cable, and other connection manners are similar to the above.
As an optional implementation manner, an image sensor in the image acquisition module is connected with the serialization unit through an MIPI (mobile industry processor interface) flat cable interface, and after acquiring image RAW data, the image sensor transmits a CSI-2 standard multi-channel image frame signal to the serialization unit connected with the image sensor through the MIPI interface;
each serialization unit is connected with the deserializing module in a wired or wireless mode, wherein the wired mode is a coaxial cable or a shielded twisted pair;
the deserializing module is connected with the main control module through an MIPI flat cable interface.
As an optional implementation manner, each serialization unit is connected with the deserializing module through a coaxial cable, each serialization unit is connected with a corresponding BNC interface in the deserializing module through a BNC interface, and each serialization unit transmits a single-path high-frequency signal conforming to the FPD-LINK interface protocol to the deserializing module through the BNC interface.
The number of the BNC interfaces in the deserializing module is the same as that of the serialization units, and the BNC interfaces are in one-to-one correspondence with the BNC interfaces integrated by the serialization units.
As an alternative embodiment, each serialization unit is connected with the deserializing module in a wireless manner, and the serialization unit and the deserializing module also need to be integrated with a signal transceiver.
The multi-dimensional feature acquisition device provided by the embodiment of the invention further comprises: a reference clock and synchronization signal generator, wherein:
the synchronization signal generator generates a synchronization control signal based on a clock signal of the reference clock;
the deserializing module is also used for respectively transmitting the synchronous control signals to each serializing unit;
the serialization unit is also used for transmitting the received synchronous control signal to the image sensor and controlling the exposure of the image sensor.
The reference clock is used for generating a clock signal after being powered on and transmitting the clock signal to the synchronous signal generator, the specific clock signal frequency is set according to the specific SOC system requirement, in implementation, the reference clock selects a 24 MHz high-precision crystal oscillator as the reference clock, the clock generates a 24 MHz clock signal after being powered on, and the 24 MHz clock signal is transmitted to the synchronous signal generator.
The synchronization signal generator generates a first synchronization control signal having the same frequency as the target output frame rate based on the clock signal of 24 MHz and the target output frame rate.
The deserializing module receives the first synchronous control signal through the GPIO port of the deserializing chip, conducts transparent transmission of the synchronous control signal, and transmits a second synchronous control signal to each serializing unit through the low-delay back channel, wherein the second synchronous control signal refers to the synchronous control signal transmitted from the deserializing module to the serializing unit, and the characteristics of the signal such as frequency and the like are not changed and are the same as the first synchronous control signal.
As an optional implementation manner, the deserializing module modulates the synchronization control signal to a back channel and then respectively transmits the modulated synchronization control signal to each serializing unit.
The serialization unit separates a second synchronous control signal from an image data signal (namely, a single-path high-frequency signal), after the second synchronous control signal is separated from the back channel, the transparent transmission of the synchronous control signal is carried out, and a third synchronous control signal is transmitted to a VS pin of an image sensor connected with the VS pin through a GPIO port, wherein the third synchronous control signal refers to the synchronous control signal transmitted from the serialization unit to the image sensor, and the characteristics of the signal such as frequency and the like are not changed and are the same as the second synchronous control signal.
As an alternative embodiment, the serializing unit separates the synchronization control signal from the back channel and transmits the synchronization control signal to the image sensor.
The image sensor receives a third synchronization control signal, and then is controlled by the third synchronization control signal to perform exposure, specifically, the third synchronization control signal is transmitted to a VS pin of the image sensor, and the VS pin detects a rising edge of the third synchronization control signal, so as to perform exposure control of the lens.
Fig. 3 shows a timing chart corresponding to the clock signal, the synchronization signals, and the exposures of the image sensors in the above process.
The reference clock and the synchronization signal generator may be integrated at different locations, and as an alternative implementation, the reference clock and the synchronization signal generator are integrated with the main control module; or
The reference clock is integrated with the main control module, and the synchronous signal generator is integrated with the main control module; or
The reference clock and the synchronization signal generator are integrated with the deserializing module.
Based on the mechanism, the transmission precision of the synchronous control signal of the multi-dimensional feature acquisition device provided by the embodiment of the invention can reach ns level, which is about 1000000 times of network transmission, the high-precision synchronous exposure of a plurality of image acquisition modules can be realized, and the problem that low time delay is needed among the plurality of image acquisition modules is solved.
The above-mentioned process of synchronous exposure is described below by way of example, in which the reference clock is a 24 mhz high-precision crystal oscillator, and the target output frame rate is set to 25 frames, wherein the corresponding timing chart in the whole synchronous exposure is shown in fig. 3:
first, as shown in fig. 4, the reference clock and the synchronization signal generator are integrated with the main control module.
After the reference clock is powered on, a 24 MHz clock signal is generated and transmitted to the synchronous signal generator, and the synchronous signal generator calculates the period of a synchronous control signal according to the target output frame rate and outputs a first synchronous control signal based on the received 24 MHz clock signal;
target output frame rate f in the embodiment of the inventionsFor 25 frames, the reference clock is generated with a clock signal period tbFrequency fbIs 24 mhz. Synchronous control signal period tsThe calculation formula of (a) is as follows:
after substituting the corresponding data into the formula, it is determined that the period configuration of the synchronous control signal should be 960000tb。
And an output pin of the synchronous control signal is electrically connected with a GPIO port of a deserializing chip in the deserializing module, and the synchronous signal generator generates a first synchronous control signal and then transmits the first synchronous control signal to the deserializing chip of the deserializing module through the electrical connection.
After the deserializing module receives the first synchronous control signal, the deserializing chip internal circuit modulates the first synchronous control signal to the back channel to form a plurality of paths of identical second synchronous control signals, and the identical second synchronous control signals are transmitted to the serializing units of the image acquisition modules through long lines.
The specific number of the second synchronous control signals is determined by the number of the image acquisition modules, generally four, and if more paths are needed, the number of the deserializing chips needs to be correspondingly increased by the deserializing module. The third synchronous exposure control signal and the first synchronous control signal have the following correspondence relationship.
The serialization unit demodulates the second synchronous control signal through the serialization chip, separates the second synchronous control signal from the back channel to form a third synchronous control signal, and transmits the third synchronous control signal to an exposure synchronous control pin (VS pin) of the image sensor connected with the third synchronous control signal through the MIPI bus line through a GPIO port corresponding to the serialization chip.
At the moment, the image sensors in the plurality of image acquisition modules are all required to be configured in a slave mode, and when the image sensors are in the slave mode, no synchronous control signal is triggered, and the image sensors do not output image frames; when the image sensor is in the slave mode, after receiving the third synchronous control signal, the image sensor senses the rising edge of the synchronous control signal and triggers to collect and output a frame of image, and each time the rising edge is sensed, the image sensor triggers to generate a frame of image.
In the second example, as shown in fig. 5, the reference clock is integrated with the main control module, and the synchronization signal generator is integrated with the main control module.
After the reference clock is powered on, a 24 MHz clock signal is generated and output to the deserializing module through a clock output interface of the main control module, and after a synchronous signal generator integrated with the deserializing module receives the clock signal, the period of the synchronous control signal is calculated according to the target output frame rate based on the 24 MHz clock signal, and a first synchronous control signal is output;
target output frame rate f in the embodiment of the inventionsFor 25 frames, the reference clock is generated with a clock signal period tbFrequency fbIs 24 mhz. Synchronous control signal period tsThe calculation formula of (a) is as follows:
after substituting the corresponding data into the formula, it is determined that the period configuration of the synchronous control signal should be 960000tb。
And an output pin of the synchronous control signal is electrically connected with a GPIO port of a deserializing chip in the deserializing module, and the synchronous signal generator generates a first synchronous control signal and then transmits the first synchronous control signal to the deserializing chip of the deserializing module through the electrical connection.
After the deserializing module receives the first synchronous control signal, the deserializing chip internal circuit modulates the first synchronous control signal to the back channel to form a plurality of paths of identical second synchronous control signals, and the identical second synchronous control signals are transmitted to the serializing units of the image acquisition modules through long lines.
The specific number of the second synchronous control signals is determined by the number of the image acquisition modules, generally four, and if more paths are needed, the number of the deserializing chips needs to be correspondingly increased by the deserializing module. The third synchronous exposure control signal and the first synchronous control signal have the following correspondence relationship.
The serialization unit demodulates the second synchronous control signal through the serialization chip, separates the second synchronous control signal from the back channel to form a third synchronous control signal, and transmits the third synchronous control signal to an exposure synchronous control pin (VS pin) of the image sensor connected with the third synchronous control signal through the MIPI bus line through a GPIO port corresponding to the serialization chip.
At the moment, the image sensors in the plurality of image acquisition modules are all required to be configured in a slave mode, and when the image sensors are in the slave mode, no synchronous control signal is triggered, and the image sensors do not output image frames; when the image sensor is in the slave mode, after receiving the third synchronous control signal, the image sensor senses the rising edge of the synchronous control signal and triggers to collect and output a frame of image, and each time the rising edge is sensed, the image sensor triggers to generate a frame of image.
Example three, as shown in fig. 6, the reference clock and the synchronization signal generator are integrated with the deserializing module.
After the reference clock is powered on, a 24 MHz clock signal is generated and transmitted to the synchronous signal generator, and the synchronous signal generator calculates the period of a synchronous control signal according to the target output frame rate and outputs a first synchronous control signal based on the received 24 MHz clock signal;
target output frame rate f in the embodiment of the inventionsFor 25 frames, the reference clock is generated with a clock signal period tbFrequency fbIs 24 mhz. Synchronous control signal period tsThe calculation formula of (a) is as follows:
after substituting the corresponding data into the formula, it is determined that the period configuration of the synchronous control signal should be 960000tb。
And an output pin of the synchronous control signal is electrically connected with a GPIO port of a deserializing chip in the deserializing module, and the synchronous signal generator generates a first synchronous control signal and then transmits the first synchronous control signal to the deserializing chip of the deserializing module through the electrical connection.
After the deserializing module receives the first synchronous control signal, the deserializing chip internal circuit modulates the first synchronous control signal to the back channel to form a plurality of paths of identical second synchronous control signals, and the identical second synchronous control signals are transmitted to the serializing units of the image acquisition modules through long lines.
The specific number of the second synchronous control signals is determined by the number of the image acquisition modules, generally four, and if more paths are needed, the number of the deserializing chips needs to be correspondingly increased by the deserializing module. The third synchronous exposure control signal and the first synchronous control signal have the following correspondence relationship.
The serialization unit demodulates the second synchronous control signal through the serialization chip, separates the second synchronous control signal from the back channel to form a third synchronous control signal, and transmits the third synchronous control signal to an exposure synchronous control pin (VS pin) of the image sensor connected with the third synchronous control signal through the MIPI bus line through a GPIO port corresponding to the serialization chip.
At the moment, the image sensors in the plurality of image acquisition modules are all required to be configured in a slave mode, and when the image sensors are in the slave mode, no synchronous control signal is triggered, and the image sensors do not output image frames; when the image sensor is in the slave mode, after receiving the third synchronous control signal, the image sensor senses the rising edge of the synchronous control signal and triggers to collect and output a frame of image, and each time the rising edge is sensed, the image sensor triggers to generate a frame of image.
The workflow of the multidimensional feature acquisition device is explained in the following with reference to fig. 7:
703, after each image sensor receives the synchronous control signal, synchronously exposing and acquiring a target image, and transmitting a plurality of paths of image frame signals to a serialization unit connected with the image frame signals;
704, receiving and serially modulating a plurality of paths of image frame signals by each serialization unit to form a single path of high-frequency signals and transmitting the single path of high-frequency signals to a deserializing module;
An embodiment of the present invention further provides an image acquisition method, as shown in fig. 8, the method includes:
The detailed process of the image acquisition is already described in the introduction of the multi-dimensional feature acquisition apparatus, and is not described herein again.
As an optional implementation manner, before the image sensors respectively acquire the image sensing signals, the method further includes:
generating a clock signal by a reference clock and transmitting the clock signal to a synchronous signal generator;
the synchronization signal generator generates a synchronization control signal based on the clock signal;
transmitting the synchronous control signal to each serialization unit through a deserializing module;
and transmitting the synchronous control signal to an image sensor connected with each serialization unit through each serialization unit, and controlling the exposure of the image sensor.
The specific process of the above-mentioned synchronous exposure has been described in detail in the introduction of the above-mentioned multi-dimensional feature collecting apparatus, and is not described herein again.
As an optional implementation manner, each serialization unit is used for serially encoding a plurality of image frame signals output by an image sensor connected with the serialization unit to obtain a single high-frequency signal, and the single high-frequency signal is transmitted to the deserializing module, and the method includes the following steps:
the image processing method comprises the steps that multiple paths of image frame signals output by an image sensor connected with a serialization unit are numbered through the serialization unit according to protocol regulations, the multiple paths of image frame signals are arranged according to the numbering sequence to form a single path of high-frequency signal, and the single path of high-frequency signal is transmitted to a deserializing module.
Claims (10)
1. The utility model provides a multidimensional feature acquisition device, its characterized in that includes host system, deserialize module and two at least image acquisition modules, including image sensor in each image acquisition module and with the serialization unit that image sensor connects, host system with the deserialize module is connected, deserialize module and each serialization unit connection, wherein:
the image sensors in the image acquisition modules are used for acquiring image sensing signals, processing the image sensing signals into a plurality of paths of image frame signals according to a protocol and transmitting the image frame signals to the connected serialization units;
each serialization unit is used for serially encoding the multi-channel image frame signals into a single-channel high-frequency signal and transmitting the single-channel high-frequency signal to the deserializing module;
the deserializing module is used for processing the single-path high-frequency signals received from a plurality of serializing units in parallel: and deserializing the single-path high-frequency signal into corresponding multi-path image frame signals, and transmitting the obtained multi-path image frame signals to the main control module in parallel.
2. The multi-dimensional feature acquisition apparatus of claim 1 further comprising a reference clock and synchronization signal generator, wherein:
the synchronization signal generator generates a synchronization control signal based on a clock signal of the reference clock;
the deserializing module is also used for respectively transmitting the synchronous control signals to each serializing unit;
the serialization unit is also used for transmitting the received synchronous control signal to the image sensor and controlling the exposure of the image sensor.
3. The multi-dimensional feature acquisition apparatus of claim 2,
the reference clock and the synchronous signal generator are integrated with the main control module; or
The reference clock is integrated with the main control module, and the synchronous signal generator is integrated with the main control module; or
The reference clock and the synchronization signal generator are integrated with the deserializing module.
4. The multi-dimensional feature acquisition apparatus of claim 2,
an image sensor in the image acquisition module is connected with the serialization unit through an MIPI flat cable interface;
each serialization unit is connected with the deserializing module in a wired or wireless mode, wherein the wired mode is a coaxial cable or a shielded twisted pair;
the deserializing module is connected with the main control module through an MIPI flat cable interface.
5. The multi-dimensional feature collection device of claim 1, wherein each serialization element is connected to the deserializing module by a coaxial cable, and wherein each serialization element is connected to a corresponding BNC interface in the deserializing module by a BNC interface.
6. The multi-dimensional feature acquisition apparatus of claim 2,
each serialization unit is used for serially encoding the multi-channel image frame signals into a single-channel high-frequency signal, modulating the single-channel high-frequency signal to a forward channel and then transmitting the single-channel high-frequency signal to the deserializing module;
the deserializing module is used for processing the single-path high-frequency signals received from a plurality of serializing units in parallel: separating the single-path high-frequency signal from a forward channel, deserializing the single-path high-frequency signal into corresponding multi-path image frame signals, and transmitting the obtained multi-path image frame signals to the main control module in parallel;
the deserializing module is also used for modulating the synchronous control signal to a back channel and then respectively transmitting the synchronous control signal to each serializing unit;
the serialization unit is also used for separating a synchronization control signal from the back channel and transmitting the synchronization control signal to the image sensor.
7. The multi-dimensional feature acquisition apparatus of claim 1,
the serialization unit is used for numbering the multi-channel image frame signals according to protocol regulations, arranging the multi-channel image frame signals into single-channel high-frequency signals according to the numbering sequence, and transmitting the single-channel high-frequency signals to the deserializing module.
8. An image acquisition method, characterized in that the method comprises:
respectively acquiring image sensing signals through each image sensor, processing the image sensing signals into a plurality of paths of image frame signals according to a protocol, and transmitting the image frame signals to a serialization unit connected with the image sensors;
the method comprises the steps that multiple paths of image frame signals output by an image sensor connected with a serialization unit are subjected to serial coding through the serialization units, and single-path high-frequency signals are obtained and transmitted to a deserializing module;
and the deserializing module is used for processing the received single-path high-frequency signals output by the serializing units in parallel: and deserializing the single-path high-frequency signal into corresponding multi-path image frame signals, and transmitting the obtained multi-path image frame signals to the main control module in parallel.
9. The method of claim 8, wherein before the image sensor signals are respectively acquired by the image sensors, the method further comprises:
generating a clock signal by a reference clock and transmitting the clock signal to a synchronous signal generator;
the synchronization signal generator generates a synchronization control signal based on the clock signal;
transmitting the synchronous control signal to each serialization unit through a deserializing module;
and transmitting the synchronous control signal to an image sensor connected with each serialization unit through each serialization unit, and controlling the exposure of the image sensor.
10. The method as claimed in claim 8, wherein the serial encoding of the image frame signals outputted from the image sensor connected to the serializing unit by each serializing unit to obtain a single high frequency signal and transmitting the single high frequency signal to the deserializing module comprises:
the image processing method comprises the steps that multiple paths of image frame signals output by an image sensor connected with a serialization unit are numbered through the serialization unit according to protocol regulations, and the multiple paths of image frame signals are arranged into single-path high-frequency signals according to the numbering sequence and are transmitted to a deserializing module.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111632461.7A CN114143513B (en) | 2021-12-29 | 2021-12-29 | Multi-dimensional feature acquisition device and image acquisition method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111632461.7A CN114143513B (en) | 2021-12-29 | 2021-12-29 | Multi-dimensional feature acquisition device and image acquisition method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114143513A true CN114143513A (en) | 2022-03-04 |
| CN114143513B CN114143513B (en) | 2023-04-14 |
Family
ID=80383725
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111632461.7A Active CN114143513B (en) | 2021-12-29 | 2021-12-29 | Multi-dimensional feature acquisition device and image acquisition method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114143513B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040017486A1 (en) * | 2002-07-24 | 2004-01-29 | Cooper Alan Neal | Digital camera synchronization |
| JP2005303673A (en) * | 2004-04-12 | 2005-10-27 | Canon Inc | Image photographing processing system and control program |
| CN202231775U (en) * | 2011-08-15 | 2012-05-23 | 山东山大鸥玛软件有限公司 | Image data high speed transmission circuit |
| CN103037222A (en) * | 2012-12-04 | 2013-04-10 | 中国北方车辆研究所 | Compression transmission device and method of parallel digital video signal |
| CN107249101A (en) * | 2017-07-13 | 2017-10-13 | 浙江工业大学 | A kind of sample of high-resolution image and processing unit |
| CN107707795A (en) * | 2017-09-30 | 2018-02-16 | 杭州雄迈集成电路技术有限公司 | A kind of dual camera imaging acquisition method |
| CN109947681A (en) * | 2019-03-20 | 2019-06-28 | 天津芯海创科技有限公司 | Stringization/deserializer and high speed interface protocol exchange chip |
| CN110275846A (en) * | 2019-08-19 | 2019-09-24 | 广东高云半导体科技股份有限公司 | I3C is bilateral along communicating circuit and electronic equipment |
-
2021
- 2021-12-29 CN CN202111632461.7A patent/CN114143513B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040017486A1 (en) * | 2002-07-24 | 2004-01-29 | Cooper Alan Neal | Digital camera synchronization |
| JP2005303673A (en) * | 2004-04-12 | 2005-10-27 | Canon Inc | Image photographing processing system and control program |
| CN202231775U (en) * | 2011-08-15 | 2012-05-23 | 山东山大鸥玛软件有限公司 | Image data high speed transmission circuit |
| CN103037222A (en) * | 2012-12-04 | 2013-04-10 | 中国北方车辆研究所 | Compression transmission device and method of parallel digital video signal |
| CN107249101A (en) * | 2017-07-13 | 2017-10-13 | 浙江工业大学 | A kind of sample of high-resolution image and processing unit |
| CN107707795A (en) * | 2017-09-30 | 2018-02-16 | 杭州雄迈集成电路技术有限公司 | A kind of dual camera imaging acquisition method |
| CN109947681A (en) * | 2019-03-20 | 2019-06-28 | 天津芯海创科技有限公司 | Stringization/deserializer and high speed interface protocol exchange chip |
| CN110275846A (en) * | 2019-08-19 | 2019-09-24 | 广东高云半导体科技股份有限公司 | I3C is bilateral along communicating circuit and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114143513B (en) | 2023-04-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111404607B (en) | Indoor visible light communication positioning method and system based on machine learning and OFDM | |
| CN108924477A (en) | A kind of long-distance video processing method and system, video processing equipment | |
| CN106817794A (en) | TOF circuit modules and its application | |
| CN114143513B (en) | Multi-dimensional feature acquisition device and image acquisition method | |
| CN106851183A (en) | Multi-channel video processing system and method based on FPGA | |
| CN107911597A (en) | Image collecting device and intelligent terminal | |
| CN211180206U (en) | Multi-source heterogeneous sensor system and automatic driving vehicle | |
| CN109278644A (en) | A kind of smart electronics rearview mirror and its image-pickup method based on high-definition camera | |
| KR20000042653A (en) | Apparatus and method for processing snap shot of usb camera | |
| US11716544B2 (en) | Image capture module and electronic terminal | |
| JP2004180187A (en) | Spatial division infrared communication system | |
| CN220629423U (en) | Network camera integrating UWB antenna array single base station positioning function | |
| EP0416235B1 (en) | Data-communication method for loop-type network having portable slave stations connectable to addressable junction boxes permanently connected in the network | |
| CN218734455U (en) | Image transmission apparatus | |
| CN109996004A (en) | A kind of low power image imaging method, processor and device | |
| CN113442938A (en) | Vehicle-mounted computing system, electronic equipment and vehicle | |
| CN113689707B (en) | Video data processing method, device and computer readable storage medium | |
| CN116521114A (en) | Image processing module, device and system | |
| CN219577124U (en) | Control circuit for camera | |
| CN220022907U (en) | Optical motion capturing lens | |
| CN210927777U (en) | Camera circuit, binocular camera and automatic driving system | |
| CN215773580U (en) | Vehicle positioning module and electronic equipment | |
| CN215344869U (en) | Signal acquisition system | |
| CN214587005U (en) | Image processor, three-dimensional imaging device and electronic equipment | |
| WO2022025457A1 (en) | Electronic device for processing images and method for operating same |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |