CN113572966A - Image acquisition and processing device and method - Google Patents

Image acquisition and processing device and method Download PDF

Info

Publication number
CN113572966A
CN113572966A CN202110976073.4A CN202110976073A CN113572966A CN 113572966 A CN113572966 A CN 113572966A CN 202110976073 A CN202110976073 A CN 202110976073A CN 113572966 A CN113572966 A CN 113572966A
Authority
CN
China
Prior art keywords
field
view
information
image acquisition
processing unit
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110976073.4A
Other languages
Chinese (zh)
Inventor
刘明源
马建设
杨贤文
苏萍
林聪�
刘富泉
彭坤
吕新科
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen International Graduate School of Tsinghua University
Henan Hozel Electronics Co Ltd
Original Assignee
Shenzhen International Graduate School of Tsinghua University
Henan Hozel Electronics Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen International Graduate School of Tsinghua University, Henan Hozel Electronics Co Ltd filed Critical Shenzhen International Graduate School of Tsinghua University
Priority to CN202110976073.4A priority Critical patent/CN113572966A/en
Publication of CN113572966A publication Critical patent/CN113572966A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60Control of cameras or camera modules
    • H04N23/67Focus control based on electronic image sensor signals
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/50Constructional details
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/50Constructional details
    • H04N23/55Optical parts specially adapted for electronic image sensors; Mounting thereof
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60Control of cameras or camera modules
    • H04N23/695Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Studio Devices (AREA)

Abstract

The invention discloses an image acquisition and processing device and method. The large-field-of-view image acquisition unit comprises at least two first cameras arranged at intervals, and the small-field-of-view image acquisition unit comprises at least one second camera arranged between the two first cameras. The information processing unit is in signal communication with the large view field acquisition unit and the small view field acquisition unit, wherein the large view field image acquisition unit is used for acquiring real-time images and transmitting image information to the information processing unit for processing, the information processing unit sends the processed information to the small view field image acquisition unit, and the small view field image acquisition unit acquires images according to the information received from the information processing unit. The invention adopts the combination of the large-view-field lens and the small-view-field lens to greatly improve the image acquisition frequency and effectively reduce the cost of the sensor.

Description

Image acquisition and processing device and method
Technical Field
The invention relates to the field of image acquisition, in particular to an image acquisition and processing device and method.
Background
Currently, a Laser radar (LIDAR or Laser Detection And Range) has been widely applied to the fields of resource development, environmental monitoring, traffic communication, And the like. However, the laser-based image acquisition and processing device has the following problems: 1) the target detection depends heavily on the accuracy of the 3D point, and the position and detection need to approximate the appearance of an object; 2) the method has the advantages that a distant target cannot be detected, and as the vehicle and the person only account for 10% of the image and are limited by the laser range, the problems of the current pseudo laser radar of the distant object can be ignored during training; 3) end-to-end depth prediction and target detection joint training is not achieved. In the vehicle radar image, 90% of the pixels correspond to the background. The 10% pixels associated with cars and people (< 1% people) are mainly within a depth of 20 meters.
Reliable and accurate three-dimensional object detection is a necessary condition for safe autonomous driving. While laser image acquisition and processing devices can provide accurate estimates of the three-dimensional point cloud environment, in many cases their cost is prohibitive. Recently, the introduction of pseudo-laser radar (PL) has made the precision gap between LiDAR sensor-based approaches and inexpensive stereo camera-based approaches to be greatly reduced. Therefore, a low-cost, high-frequency, fast image-based image acquisition and processing device is needed.
Disclosure of Invention
The invention aims to provide a high-frequency image acquisition and processing device and a high-frequency image acquisition and processing method for identifying and tracking a high-frequency and quick-response target by means of image processing.
In order to solve the above-described problems, according to an aspect of the present invention, there is provided an image acquisition and processing apparatus characterized by comprising:
a large-field image acquisition unit, which comprises at least two first cameras arranged at intervals,
a small field of view image acquisition unit comprising at least one second camera arranged between the at least two first cameras, and
an information processing unit in signal communication with the large field of view acquisition unit and the small field of view acquisition unit, wherein
The large-view-field image acquisition unit is used for acquiring a real-time image and transmitting image information to the information processing unit for processing, the information processing unit sends the processed information to the small-view-field image acquisition unit, and the small-view-field image acquisition unit acquires the image according to the information received from the information processing unit.
In one embodiment, the camera fields of view of the at least two first cameras are larger than the camera field of view of the second camera.
In one embodiment, the field of view and the focal length of the at least two first cameras are fixed and the field of view and the focal length of the at least one second camera are adjustable.
In one embodiment, the information processing unit is configured to process information transmitted from the large-field-of-view image acquisition unit and then output distance and orientation information of a target object to the small-field-of-view image acquisition unit, and the small-field-of-view image acquisition unit is configured to rotate the camera according to the orientation information of the target transmitted from the information processing unit and select zoom parameters according to the distance information of the target transmitted from the information processing unit.
In one embodiment, the at least one second camera is arranged in an intermediate position between the at least two first cameras.
In one embodiment, the at least two first cameras are the same in size and model.
In one embodiment, the small-field-of-view image acquisition unit further comprises a rotating mechanism for driving the second camera, and the rotating mechanism is in signal communication with the information processing unit and drives the second camera to rotate according to the information of the information processing unit.
In one embodiment, the small-field-of-view image acquisition unit further comprises a movement mechanism, wherein the movement mechanism is provided with an optical element, is in signal communication with the information processing unit and is set to move according to information transmitted by the information processing unit so that the target enters the second camera through the action of the optical element; preferably, the optical element is a mirror.
According to another aspect of the present invention, there is also provided an image acquisition and processing method, the method comprising the steps of:
step one, a large-view field image acquisition unit is used for acquiring a real-time image and transmitting the image information to an information processing unit,
step two, the information processing unit processes the information transmitted by the large-view-field image acquisition unit and sends the processed information to the small-view-field image acquisition unit,
and step three, the small-field-of-view image acquisition unit acquires images of the target according to the information received from the information processing unit.
In one embodiment, in the second step, the information processing unit processes the information transmitted from the large-field-of-view image acquisition unit and outputs the distance and orientation information of the target object to the small-field-of-view image acquisition unit; and in the third step, the small-field image acquisition unit rotates the camera according to the azimuth information of the target transmitted by the information processing unit and selects zooming parameters according to the distance information of the target transmitted by the information processing unit.
In one embodiment, the large-field-of-view image acquisition unit includes at least two first cameras, and the small-field-of-view image acquisition unit includes at least one second camera, wherein the information processing unit calculates distance and orientation information between the target object and the second camera according to a distance difference between the target and the at least two first cameras.
The invention adopts the combination of the large-view field lens and the small-view field lens to greatly improve the image acquisition frequency, and effectively reduces the cost of the sensor by depending on the mature image processing hardware at present.
Drawings
FIG. 1 is a schematic view of an initial state of an image acquisition and processing apparatus according to an embodiment of the present invention.
Fig. 2 is a schematic view of an operation state of the image acquisition and processing apparatus of fig. 1.
Fig. 3 is a schematic layout of an image acquisition and processing apparatus according to another embodiment of the present invention.
Detailed Description
The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings so that the objects, features and advantages of the invention can be more clearly understood. It should be understood that the embodiments shown in the drawings are not intended to limit the scope of the present invention, but are merely intended to illustrate the spirit of the technical solution of the present invention.
In the following description, for the purposes of illustrating various disclosed embodiments, certain specific details are set forth in order to provide a thorough understanding of the various disclosed embodiments. One skilled in the relevant art will recognize, however, that the embodiments may be practiced without one or more of the specific details. In other instances, well-known devices, structures and techniques associated with this application may not be shown or described in detail to avoid unnecessarily obscuring the description of the embodiments.
Reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
In the following description, for the purposes of clearly illustrating the structure and operation of the present invention, directional terms will be used, but terms such as "front", "rear", "left", "right", "outer", "inner", "outer", "inward", "upper", "lower", etc. should be construed as words of convenience and should not be construed as limiting terms.
The invention aims to provide a high-frequency image acquisition and processing device which carries out high-frequency and quick-response target identification and tracking by means of image processing. Compared with the linear scanning of the laser radar, the image recognition mode improves the target recognition speed. The invention adopts the combination of the large-view field lens and the small-view field lens to greatly improve the image acquisition frequency, and effectively reduces the cost of the sensor by depending on the mature image processing hardware at present.
Fig. 1 is a schematic view of an initial state of an image capturing and processing apparatus 100 according to an embodiment of the present invention, and fig. 2 is a schematic view of an operation state of the image capturing and processing apparatus 100 of fig. 1. As shown in fig. 1-2, the image acquisition and processing apparatus 100 of the present invention generally includes a large-field-of-view image acquisition unit 11, a small-field-of-view image acquisition unit 12, and an information processing unit 13. The large-field image acquisition unit 11 includes at least two first cameras 11A, and the at least two first cameras 11A are arranged at intervals. The small-field-of-view image acquisition unit 12 includes at least one second camera 12A, the at least one second camera 12A being arranged between the at least two first cameras 11A. The information processing unit 13 is in signal communication with the large-view-field image acquisition unit 11 and the small-view-field image acquisition unit, the large-view-field image acquisition unit 11 is used for acquiring real-time images and transmitting image information to the information processing unit 13 for processing, the information processing unit 13 sends the processed information to the small-view-field image acquisition unit 12, and the small-view-field image acquisition unit 12 acquires images according to the information received from the information processing unit 13.
At least two first cameras 11A of the large-field image acquisition unit 11 image a specific target Q within a field range, and because the imaging coordinate positions of the target Q on the image sensors of the two first cameras 11A are different, the distance and the orientation between the target Q and the second camera 12A can be calculated according to the coordinate difference.
It should be noted that the fact that the large-field-of-view image capturing unit 11 of the present invention includes at least two first cameras 11A means that the large-field-of-view image capturing unit 11 may include only two cameras, may include more than two first cameras 11A, and may include other elements besides the first cameras 11A. In addition, the two first cameras 11A may be identical cameras, that is, cameras of the same specification and model, or cameras of different models and specifications, and are not limited herein.
The small-field-of-view image capturing unit 12 includes at least one second camera 12A, which means that the small-field-of-view image capturing unit 12 may include only one second camera 12A, may include a plurality of second cameras 12A, and may include other elements besides the second cameras 12A, such as a rotating mechanism, a moving mechanism, and the like, which will be further described below.
As shown in fig. 1, the specific target Q is positioned on the left side of the sensor in the image sensor C of the first camera 11A on the left side and on the right side of the sensor in the image sensor D of the first camera 11A on the right side, and the distance h between the target Q and the second camera 12A can be calculated from the coordinate difference between the images obtained by the specific target Q on the image sensor C and the image sensor D, and the offset angle ω of the specific target Q with respect to the second camera 12A can also be calculated. The information processing unit 13 determines the optimal zoom parameter of the second camera 12A according to the distance h and the angle ω, and simultaneously gives information of the offset angle ω to a rotating mechanism (not shown) of the second camera 12A, and the rotating mechanism drives the second camera to rotate, so that the target Q can be within the imaging range of the third camera 12A at the optimal angle. The small-field-of-view image acquisition unit 12 completes zooming according to the optimal zooming parameter of the information processing unit 13, adjusts the spatial position of the third camera 12A, enables the target Q to be located at the optimal imaging position of the third camera 12A, and completes track tracking for the target Q according to the subsequent signal of the information processing unit 13.
In the present invention, the information processing unit 13 is mainly based on the principle of binocular ranging, since two first cameras 11A present "parallax" to images presented by the same target. The farther the object distance is, the smaller the parallax error is; conversely, the greater the parallax. Two first cameras 11A collect real-time images, target data in the images are processed through deep learning calculation to give distance and direction information of a front target, after an information processing unit sends the information to a small-field image collecting unit 12, the small-field image collecting unit 12 changes according to the information, a second camera 12A is rotated to point to the direction of the target, and meanwhile, according to the distance information, an optimal zooming parameter is selected, so that the second camera 12A can shoot the target with an optimal optical index for requirements of subsequent image processing, image recognition, image utilization and the like.
It should be noted that the information processing unit 13 not only can obtain the distance h and the angle ω information, calculate and transmit the zoom and angle change signals to the third camera 12A of the small field-of-view image acquisition unit 12, but also can determine the target Q more accurately depending on the image depth learning. In other embodiments, the information processing unit 13 can not only obtain the above information, but also provide the information to other association mechanisms.
Because the large-view-field image acquisition unit is used for acquiring image information in a large range, and the small-view-field image acquisition unit is mainly used for acquiring images of specific targets screened by the large-view-field image acquisition unit, preferably, the camera view fields of the two first cameras 11A of the large-view-field image acquisition unit 11 are larger than the camera view field of the second camera 12A of the small-view-field image acquisition unit 12, so that the camera cost is favorably reduced. Of course, those skilled in the art will understand that the cameras of the two first cameras 11A of the large-field image capturing unit 11 may be equal to or smaller than the camera field of view of the second camera 12A of the small-field image capturing unit 12.
Optionally, the view field and the focal length of the two first cameras 11A of the large view field image acquisition unit 11 are fixed, and the view field and the focal length of the at least one second camera 12A of the small view field image acquisition unit 12 are adjustable, so that the corresponding cameras are selected according to functions, so that the selection of the cameras is matched with the use scene, the image acquisition quality is further improved, and the cost is reduced.
In one embodiment, the information processing unit is configured to process information transmitted from the large-field-of-view image acquisition unit and output distance and orientation information of the target object to the small-field-of-view image acquisition unit, and the small-field-of-view image acquisition unit is configured to rotate the camera according to the orientation information of the target transmitted from the information processing unit and select the zoom parameter according to the distance information of the target transmitted from the information processing unit.
With continued reference to fig. 1, optionally, the second camera 12A is disposed at a position intermediate the two first cameras 11A, which can minimize the rotation angle of the second camera 12A and can image an object within the field of view with the best effect.
In operation, as shown in fig. 1-2, the two first cameras 11A of the large-field image acquisition unit 11 capture images of a field range, and because the coordinate positions of the target imaged on the sensors of the two first cameras 11A are different, the distance and the orientation between the target and the cameras can be calculated according to the coordinate difference, and the distance and the orientation information are sent to the information processing unit 12. Specifically, at present, Q is located at the left side of the sensor in the C sensor and at the right side of the sensor in the D sensor, and the middle h between the target Q and the camera can be calculated according to the coordinate difference of the image obtained by the target Q on the C, D sensor, and the ω angle can also be calculated. The information processing unit determines the optimal zoom parameter of the camera C according to the distance h and the angle ω, and transmits the offset angle ω to the rotating mechanism of the second camera 12A, so that the target Q can be within the imaging range of the second camera 12A at the optimal angle. The second camera 12A in the small field of view zooming image acquisition unit finishes zooming according to the optimal zooming parameter of the information processing unit, and adjusts the spatial position of the second camera 12A, so that the target Q is located at the optimal imaging position of the second camera 12A, and the trajectory tracking for the target Q is finished according to the subsequent signal of the information processing unit.
FIG. 3 is a schematic diagram of an image acquisition and processing apparatus according to another embodiment of the present invention. Another embodiment of the present invention will be described below with reference to fig. 3, and the present embodiment is different from the embodiment shown in fig. 1-2 in that the small-field-of-view image capturing unit 12 of the present embodiment further includes a separate rotating mechanism on which a reflecting mirror 12B such as a prism is mounted, which is disposed between the first camera 11A and the second camera 11B and causes an image of the object Q to be incident on the third camera 12A at an optimum angle.
As can be seen from the above description, the present invention also provides an image acquisition and processing method, comprising the steps of:
step one, a large-view field image acquisition unit is used for acquiring a real-time image and transmitting the image information to an information processing unit,
step two, the information processing unit processes the information transmitted by the large-view-field image acquisition unit and sends the processed information to the small-view-field image acquisition unit,
and step three, the small-field-of-view image acquisition unit acquires images of the target according to the information received from the information processing unit.
Optionally, in the second step, the information processing unit processes the information transmitted from the large-field-of-view image acquisition unit, and outputs the distance and direction information of the target object to the small-field-of-view image acquisition unit; in the third step, the small-field image acquisition unit rotates the camera according to the azimuth information of the target transmitted by the information processing unit and selects the zooming parameters according to the distance information of the target transmitted by the information processing unit.
While the preferred embodiments of the present invention have been illustrated and described in detail, it should be understood that various changes and modifications of the invention can be effected therein by those skilled in the art after reading the above teachings of the invention. Such equivalents are intended to fall within the scope of the claims appended hereto.

Claims (11)

1. An image acquisition and processing device, characterized in that it comprises:
a large-field image acquisition unit, which comprises at least two first cameras arranged at intervals,
a small field of view image acquisition unit comprising at least one second camera arranged between the at least two first cameras, and
an information processing unit in signal communication with the large field of view acquisition unit and the small field of view acquisition unit, wherein
The large-view-field image acquisition unit is used for acquiring a real-time image and transmitting image information to the information processing unit for processing, the information processing unit sends the processed information to the small-view-field image acquisition unit, and the small-view-field image acquisition unit acquires the image according to the information received from the information processing unit.
2. The image acquisition and processing device according to claim 1, wherein the fields of view of the at least two first cameras are larger than the field of view of the second camera.
3. The image acquisition and processing device according to claim 1, characterized in that the field of view and the focal length of said at least two first cameras are fixed and the field of view and the focal length of said at least one second camera are adjustable.
4. The image capturing and processing device of claim 1, wherein the information processing unit is configured to process the information from the large field of view image capturing unit and output the distance and orientation information of the target object to the small field of view image capturing unit, and wherein the small field of view image capturing unit is configured to rotate the camera according to the orientation information of the target from the information processing unit and select the zoom parameter according to the distance information of the target from the information processing unit.
5. The image acquisition and processing device according to claim 1, characterized in that said at least one second camera is arranged in an intermediate position between said at least two first cameras.
6. The image acquisition and processing device according to claim 1, wherein the at least two first cameras are of the same format and model.
7. The image capturing and processing device of claim 1, wherein the small-field-of-view image capturing unit further comprises a rotating mechanism for driving the second camera, and the rotating mechanism is in signal communication with the information processing unit and drives the second camera to rotate according to the information of the information processing unit.
8. The image capture and processing device of claim 4, wherein the small-field-of-view image capture unit further comprises a motion mechanism, the motion mechanism being equipped with an optical element and being in signal communication with the information processing unit and being configured to move the object into the second camera via the action of the optical element in accordance with information transmitted from the information processing unit; preferably, the optical element is a mirror.
9. An image acquisition and processing method, characterized in that it comprises the following steps:
step one, a large-view field image acquisition unit is used for acquiring a real-time image and transmitting the image information to an information processing unit,
step two, the information processing unit processes the information transmitted by the large-view-field image acquisition unit and sends the processed information to the small-view-field image acquisition unit,
and step three, the small-field-of-view image acquisition unit acquires images of the target according to the information received from the information processing unit.
10. The method of claim 9,
in the second step, the information processing unit processes the information transmitted by the large-view-field image acquisition unit and outputs the distance and direction information of the target object to the small-view-field image acquisition unit; and
in the third step, the small-field image acquisition unit rotates the camera according to the azimuth information of the target transmitted by the information processing unit and selects the zooming parameters according to the distance information of the target transmitted by the information processing unit.
11. The method according to claim 9, wherein the large-field image capturing unit comprises at least two first cameras and the small-field image capturing unit comprises at least one second camera, wherein the information processing unit calculates distance and orientation information between the target object and the second camera according to a distance difference between the target object and the at least two first cameras.
CN202110976073.4A 2021-08-24 2021-08-24 Image acquisition and processing device and method Pending CN113572966A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110976073.4A CN113572966A (en) 2021-08-24 2021-08-24 Image acquisition and processing device and method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110976073.4A CN113572966A (en) 2021-08-24 2021-08-24 Image acquisition and processing device and method

Publications (1)

Publication Number Publication Date
CN113572966A true CN113572966A (en) 2021-10-29

Family

ID=78172575

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110976073.4A Pending CN113572966A (en) 2021-08-24 2021-08-24 Image acquisition and processing device and method

Country Status (1)

Country Link
CN (1) CN113572966A (en)

Similar Documents

Publication Publication Date Title
CN109085570A (en) Automobile detecting following algorithm based on data fusion
CN107122770B (en) Multi-camera system, intelligent driving system, automobile, method and storage medium
CN106370884B (en) Vehicle speed measuring method based on binocular camera computer vision technology
CN109120883B (en) Far and near scene-based video monitoring method and device and computer-readable storage medium
RU151809U1 (en) VIDEO SYSTEM FOR SECURITY OF VEHICLES
US11184539B2 (en) Intelligent dual-lens photographing device and photographing method therefor
WO2020061794A1 (en) Vehicle driver assistance device, vehicle and information processing method
KR20160062880A (en) road traffic information management system for g using camera and radar
CN111045000A (en) Monitoring system and method
JP2006086591A (en) Mobile body tracing system, photographing apparatus, and photographing method
CN107071341A (en) The coordinated control system and its control method of miniradar and photoelectric turntable
WO2017117749A1 (en) Follow focus system and method based on multiple ranging approaches, and photographing system
CN110750153A (en) Dynamic virtualization device of unmanned vehicle
CN109816702A (en) A kind of multiple target tracking device and method
CN109444916A (en) The unmanned travelable area determining device of one kind and method
CN108205314A (en) Based on the matched robot navigation device of stereoscopic vision and system
CN112633101A (en) Obstacle speed detection method and device
CN115291219A (en) Method and device for realizing dynamic obstacle avoidance of unmanned aerial vehicle by using monocular camera and unmanned aerial vehicle
KR102484691B1 (en) Vehicle detection system and vehicle detection method using stereo camera and radar
WO2017047873A1 (en) Scanning method using high-speed scanning apparatus
CN216414432U (en) Image acquisition and processing device
CN113572966A (en) Image acquisition and processing device and method
US11747481B2 (en) High performance three dimensional light detection and ranging (LIDAR) system for drone obstacle avoidance
CN113895482B (en) Train speed measuring method and device based on trackside equipment
JP4918675B2 (en) 3D coordinate measurement method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination