WO2018014730A1 - Procédé de réglage de paramètres de caméra, caméra dirigeant la diffusion, et système de tournage dirigeant la diffusion - Google Patents

Procédé de réglage de paramètres de caméra, caméra dirigeant la diffusion, et système de tournage dirigeant la diffusion Download PDF

Info

Publication number
WO2018014730A1
WO2018014730A1 PCT/CN2017/091863 CN2017091863W WO2018014730A1 WO 2018014730 A1 WO2018014730 A1 WO 2018014730A1 CN 2017091863 W CN2017091863 W CN 2017091863W WO 2018014730 A1 WO2018014730 A1 WO 2018014730A1
Authority
WO
WIPO (PCT)
Prior art keywords
camera
video object
target
navigation
target video
Prior art date
Application number
PCT/CN2017/091863
Other languages
English (en)
Chinese (zh)
Inventor
刘源
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2018014730A1 publication Critical patent/WO2018014730A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30204—Marker

Definitions

  • the present invention relates to the field of image processing technologies, and in particular, to a camera parameter adjustment method, a navigation camera, and a system.
  • FIG. 1 is a schematic diagram of a video conference.
  • the conference room adopts a long elliptical conference table
  • the participants' seats surround the conference table
  • the participants include A and B
  • the participants A and B sit opposite each other.
  • Cameras C0 and C1 are arranged on both sides of the projection screen in front of B and B.
  • the camera parameters are usually adjusted manually by remote control or other means to obtain a better shooting result.
  • the manual adjustment method requires the operator to have certain camera expertise and the operation process is cumbersome, which makes the adjustment efficiency low, and the better shooting effect cannot be guaranteed in time.
  • the camera can be determined by sound source positioning and the camera shooting effect can be adjusted.
  • the sound source is positioned by locating and tracking the participant who is speaking (ie, "speaker"), while using a camera to capture the close-up of the speaker, while tracking the speaker's face position and performing PTZ (for the lens) Pan Tilt Zoom, which is the “Pan, Tilt, Zoom” adjustment, so that the speaker's face is in the middle of the image.
  • the sound source localization method only considers adjusting the front side of the speaker to the center of the image, and does not consider the effect of the image being taken, nor can it guarantee a better shooting effect.
  • the embodiment of the invention provides a camera parameter adjustment method, a guide camera and a system, which can improve the efficiency of camera parameter adjustment and improve the camera shooting effect.
  • an embodiment of the present invention provides a camera parameter adjustment method, where the method is applied to a navigation camera, including:
  • first three-dimensional coordinate of the target video object where the first three-dimensional coordinate is a three-dimensional coordinate of the target video object in a first coordinate system corresponding to the target camera;
  • the first three-dimensional coordinate may be a three-dimensional coordinate of the target video object in a first coordinate system corresponding to the target camera.
  • the target camera may be a navigation camera or an ordinary PTZ camera, and the first coordinate system corresponding to the target camera may be a three-dimensional coordinate system established with the optical center of the target camera as an origin, or established by using any other reference object as an origin.
  • the three-dimensional coordinate system is not limited in the embodiment of the present invention.
  • the target video object may be any one or more video objects in the shooting scene corresponding to the navigation camera system where the navigation camera is located.
  • the acquiring the first three-dimensional coordinates of the target video object comprises:
  • the second three-dimensional coordinates are converted into first three-dimensional coordinates according to a pre-calibrated positional relationship between the binocular camera and the target camera.
  • the second three-dimensional coordinates may be two-dimensional coordinates of the binocular camera obtained by respectively acquiring video objects in left and right views of the binocular camera and the binocular camera acquired The internal and external parameters of the data are calculated.
  • the second three-dimensional coordinate is a three-dimensional coordinate of the target video object in a second coordinate system corresponding to the binocular camera
  • the second coordinate system corresponding to the binocular camera may be a binocular camera optical center
  • the two-dimensional coordinates may specifically be corresponding pixel coordinates of the target video object in the left view and the right view of the binocular camera.
  • the determining a target video object that needs to be captured comprises:
  • the target camera for capturing the target video object is filtered out from each camera of the navigation camera system where the navigation camera is located according to a preset navigation policy, including:
  • the camera whose shooting effect parameter satisfies the preset guiding strategy is determined as the target camera for capturing the target video object.
  • determining the target video object from the captured image acquired by the camera comprises:
  • the video object is determined to be the target video object.
  • the current camera is any camera other than the binocular camera that is in a positional relationship with the navigation camera in the navigation camera system, and the third three-dimensional coordinate is that the target video object corresponds to the current camera.
  • the three-dimensional coordinates in the third coordinate system are any camera other than the binocular camera that is in a positional relationship with the navigation camera in the navigation camera system, and the third three-dimensional coordinate is that the target video object corresponds to the current camera.
  • the shooting effect parameter includes any one or more of an eye-to-eye effect parameter, an occlusion relationship parameter, and a scene object parameter of the shooting area of the target video object in a coordinate system corresponding to the current camera.
  • the current camera is any camera other than the binocular camera in the navigation camera system.
  • the eye-to-eye effect parameter may include a rotation angle of the target video object relative to a coordinate system corresponding to the current camera, the rotation angle being according to a rotation angle of the target video object in the second coordinate system and the binocular pre-calibrated The positional relationship between the camera and the current camera is determined. The smaller the angle of rotation, the better the eye-to-eye effect.
  • the occlusion relationship parameter and the scene object parameter may be determined according to a pre-calibrated positional relationship between the binocular camera and the current camera, and the region of the scene object detected by the current camera is re-projected to an imaging plane of the current camera. of.
  • the output image is better when there is no occlusion relationship (the smaller the occlusion relationship parameter is). The smaller the area of the scene object and the smaller the number, the better the output image effect; otherwise, the worse the output image effect.
  • an embodiment of the present invention further provides a navigation camera, including: a memory and a processor, wherein the processor is connected to the memory;
  • the memory is used to store driver software
  • the processor reads the driver software from the memory and performs some or all of the steps of the camera parameter adjustment method of the first aspect described above by the driver software.
  • the embodiment of the present invention further provides a parameter adjustment apparatus, including an object determining unit, a selecting unit, an acquiring unit, and a parameter adjusting unit, wherein the parameter adjusting device implements a part of the camera parameter adjusting method of the first aspect by using the foregoing unit. Or all steps.
  • an embodiment of the present invention further provides a computer storage medium, where the computer storage medium stores a program, and the program includes some or all of the steps of the camera parameter adjustment method of the first aspect.
  • the embodiment of the present invention further provides a navigation camera system, including a first camera and at least one second camera, the first camera includes a navigation camera and a binocular camera, and the navigation camera and the binocular Between the cameras, the first camera and the second camera are connected by a wired interface or a wireless interface;
  • the guidance camera is configured to determine a target video object that needs to be captured, and select a target camera for capturing the target video object from a camera of the navigation camera system according to a preset navigation strategy;
  • the binocular camera is configured to acquire a second three-dimensional coordinate of the target video object, and transmit the second three-dimensional coordinate to the navigation camera; wherein the second three-dimensional coordinate is that the target video object is The three-dimensional coordinates of the binocular camera corresponding to the second coordinate system;
  • the guidance camera is configured to receive the second three-dimensional coordinates transmitted by the binocular camera; convert the second three-dimensional coordinates into a first position according to a pre-calibrated positional relationship between the binocular camera and the target camera a three-dimensional coordinate; adjusting an imaging parameter of the target camera to an imaging parameter corresponding to the first three-dimensional coordinate, and outputting a video image after adjusting the imaging parameter; wherein the first three-dimensional coordinate is the target video object The three-dimensional coordinates in the first coordinate system corresponding to the target camera.
  • the second camera may include a navigation camera and a binocular camera, and the target camera may be any of the navigation cameras in the navigation camera system; or the second camera may also be a normal PTZ camera. Then the target camera can be the guide camera or a normal PTZ camera.
  • the binocular camera can be placed on a preset guide bracket and connected to the guide camera via the guide bracket.
  • the target camera that captures the target video object is selected from the cameras of the navigation camera system according to a preset guiding strategy, and is obtained.
  • a three-dimensional coordinate of the target video object in a coordinate system corresponding to the target camera to control the target camera to perform camera parameter adjustment according to the three-dimensional coordinates of the target video object, and output a video image after adjusting the imaging parameter, so that the guided camera system It can improve the accuracy of video object detection and tracking based on three-dimensional coordinate detection and preset navigation strategy, and improve the efficiency of camera parameter adjustment, and effectively improve the camera's shooting effect.
  • FIG. 1 is a schematic diagram of a scene of a video conference
  • FIG. 2 is a schematic flowchart of a method for adjusting a camera parameter according to an embodiment of the present invention
  • FIG. 3a is a schematic diagram of a camera imaging model according to an embodiment of the present invention.
  • FIG. 3b is a schematic diagram of a calibration scenario of a multi-camera according to an embodiment of the present invention.
  • FIG. 3c is a schematic diagram of a three-dimensional positioning of a binocular camera according to an embodiment of the present invention.
  • FIG. 3 is a schematic diagram of a PTZ camera rotation model according to an embodiment of the present invention.
  • FIG. 4a is a schematic diagram of a video object matching scenario according to an embodiment of the present invention.
  • Figure 4b is an image view of a set of video objects in Figure 4a;
  • FIG. 5 is a schematic structural diagram of a parameter adjustment apparatus according to an embodiment of the present invention.
  • FIG. 6 is a schematic structural diagram of a navigation camera system according to an embodiment of the present invention.
  • FIG. 7 is a schematic structural diagram of a first camera according to an embodiment of the present disclosure.
  • FIG. 8 is a schematic diagram of networking of a navigation camera system according to an embodiment of the present invention.
  • FIG. 9 is a schematic structural diagram of a navigation camera according to an embodiment of the present invention.
  • the navigation camera according to the embodiment of the present invention may be specifically a PTZ camera for performing the technical solution of the embodiment of the present invention, which can be connected to a binocular camera, and the navigation camera can be applied to a scene such as a conference or a training, and The deployment of the position and number of the guided cameras is performed according to different scenarios.
  • the binocular camera can be mounted on a guide bracket, that is, the guide camera can be coupled to the binocular camera via a guide bracket (referred to as a "bracket").
  • the navigation camera is used for guiding shooting and tracking.
  • a microphone can be mounted on the bracket, which can be used for sound source localization, sound source recognition and the like.
  • the camera and the bracket may be separate or integrated, and a communication interface such as a serial interface may be used for communication between the camera and the bracket.
  • the binocular camera can be used for video capture, video pre-processing, motion detection, face detection, humanoid detection, scene object detection, feature detection/matching, binocular camera calibration, multi-camera calibration, etc.
  • microphones are available
  • the camera can be used for audio and video (Audio Video, referred to as "AV") object 3D positioning, AV object modeling, AV object tracking, motion / gesture Identification, guided control and video switching/synthesis, etc.
  • AV Audio Video
  • the video capture includes synchronously collecting video streams of the binocular camera and the guide camera; the video pre-processing includes pre-processing the input binocular image, such as performing noise reduction, changing resolution and frame rate, etc.; motion detection includes detecting the scene. The moving object in the moving object and the stationary background are separated to obtain the moving object area; the face detection includes detecting the face target object in the scene, and outputting the detection information of the face, such as the face position, area, direction The information detection includes detecting the human head and shoulder area in the scene and outputting detection information; the scene object detection includes detecting objects other than people in the scene, such as a lamp, a window, a conference table, etc.; feature detection/matching includes Perform feature detection and matching on the detected moving object area, detect characteristic objects (such as feature points) in one image and match in another image, and output matching feature object information; binocular camera calibration includes binocular camera Perform calibration to obtain the internal and external parameters of the binocular camera for calculating the video image.
  • the video pre-processing includes pre
  • multi-camera calibration comprises relative positional relationship between the plurality of cameras directed be calibrated to obtain the reference information relative to a plurality of external cameras directed, for positioning a plurality of video objects in the camera coordinate system.
  • the audio collection includes synchronously acquiring the multi-channel audio data of the microphone;
  • the audio pre-processing includes performing 3A processing on the input multi-channel audio data, wherein the 3A processing includes automatic exposure control (AE), auto focus control (AF), and auto white Balance control (AWB);
  • sound source localization includes detecting input multi-channel audio data to find two-dimensional position information of the sounding object;
  • sound source behavior recognition includes detecting and counting the voice behavior of the video object in the scene.
  • the 3D positioning of the AV object includes obtaining the depth information of the object feature in the image according to the parallax information obtained by the internal and external parameters of the binocular camera and the feature detection/matching, and combining the result of the audio positioning to obtain the object feature in a single guided camera coordinate system.
  • the three-dimensional position information can obtain the position information of the feature in other navigation camera coordinate systems according to the position of the feature in a single navigation coordinate system and the relative positional relationship of the plurality of navigation cameras; the AV object modeling includes combining the sound source positioning and the human face.
  • Information such as information, feature objects, and scene objects constructs a model of the AV object;
  • the AV object tracking includes tracking a plurality of AV objects in the scene, and updating state information of the object;
  • the motion/gesture recognition includes performing actions, gestures, and the like of the AV object. Identifying, for example, identifying a standing posture of a subject, a gesture action, etc.;
  • the navigation control includes determining a navigation strategy in conjunction with the result of the motion/gesture recognition and the sound source behavior recognition, and the navigation camera controls the control instruction corresponding to the output guidance strategy, the video object and the scene feature information, and the video. Output strategy, etc.
  • the camera control command can be used to control the PTZ camera to perform PTZ operation, that is, pan, tilt, zoom operation, etc., and the video object and scene feature information can be used for information sharing between multiple guide cameras.
  • the frequency output strategy can be used to control the output strategy of a single or multiple camera video streams.
  • the embodiment of the invention provides a camera parameter adjustment method, a guide camera and a system, which can improve the efficiency of camera parameter adjustment and improve the camera shooting effect. The details are explained below.
  • FIG. 2 is a schematic flowchart diagram of a camera parameter adjustment method according to an embodiment of the present invention. Specifically, the method of the embodiment of the present invention may be specifically applied to the above-mentioned navigation camera. As shown in FIG. 2, the camera parameter adjustment method in the embodiment of the present invention may include the following steps:
  • Filter according to a preset guiding strategy, a target camera for capturing the target video object from each camera of the guiding camera system where the guiding camera is located.
  • the determining the target video object that needs to be captured may be specifically: acquiring a captured image transmitted by the binocular camera, the captured image includes at least one video object; and establishing a video object model including the at least one video object And determining a target video object from the at least one video object.
  • the target camera for capturing the target video object is selected from each camera of the navigation camera system in which the navigation camera is located according to a preset navigation strategy, which may be specifically: separately from the navigation camera system Determining the target video object in the captured image acquired by each camera, and acquiring a shooting effect parameter of the target video object in each camera; determining, by the camera that the shooting effect parameter meets the preset guiding strategy, is used for shooting The target camera of the target video object.
  • One or more navigation cameras can be deployed in the navigation camera system, that is, the navigation camera system can be deployed as a guide camera + a guide camera, and can also be deployed as a guide camera + a normal camera (such as a normal PTZ camera).
  • the video object model may include all video objects in the shooting scene corresponding to the navigation camera system where the guidance camera is located. If the other cameras in the navigation camera system also include the guidance camera, the captured images transmitted by the binocular cameras connected to the other navigation cameras may also be received, and the video object model is updated to obtain the video objects of all the video objects in the captured scene. model.
  • the target video object may be any one or more video objects in the shooting scene.
  • the shooting effect parameter may include any one or more of an eye-to-eye effect parameter, an occlusion relationship parameter, and a scene object parameter of the shooting area of the target video object in a coordinate system corresponding to the current camera.
  • the current camera is any camera other than the binocular camera in the navigation camera system, that is, the current camera may be any of the guidance cameras or ordinary PTZ cameras in the guidance camera system.
  • the eye-to-eye effect parameter may include a rotation angle of the target video object with respect to a coordinate system corresponding to the current camera, and the rotation angle may be a rotation angle of the target video object in the second coordinate system. And determining a positional relationship between the binocular camera and the current camera that is pre-calibrated.
  • the rotation angle of the target video object relative to the coordinate system corresponding to the current camera may refer to an optical axis angle of the face or the humanoid object corresponding to the target video object relative to the current camera (the navigation camera or the ordinary PTZ camera). The smaller the angle, the more the face is rendered in a positive face manner, that is, the better the eye-to-eye effect, the better the output image effect.
  • the occlusion relationship parameter and the scene object parameter may be that the area of the scene object detected by the current camera is re-injected to the location according to a pre-calibrated positional relationship between the binocular camera and the current camera.
  • the imaging plane of the current camera is determined. Specifically, if the areas of the two video objects overlap, the depth information may be used to determine an occlusion relationship between the two objects, and the video object that is closer to the binocular camera may block the farther video object.
  • the output image is better when there is no occlusion relationship (the smaller the occlusion relationship parameter is).
  • the scene object indicated by the scene object parameter may include a light tube, a window, a table, and the like, and the smaller the area of the scene object is, the smaller the number is, the better the output image effect is; Conversely, the worse the output image is.
  • the first three-dimensional coordinate may be a three-dimensional coordinate of the target video object in a first coordinate system corresponding to the target camera.
  • the target camera can be configured as the above-mentioned navigation camera or a normal PTZ camera.
  • the first coordinate system corresponding to the target camera can be a three-dimensional coordinate system established with the target camera's optical center as the origin, or any other reference object.
  • the three-dimensional coordinate system established by the origin is not limited in the embodiment of the present invention.
  • the navigation camera can be connected to a preset binocular camera.
  • the acquiring the first three-dimensional coordinates of the target video object may be specifically: acquiring a second three-dimensional coordinate transmitted by the binocular camera connected to the navigation camera; and according to the pre-calibrated the binocular camera and the target The positional relationship of the camera converts the second three-dimensional coordinates into first three-dimensional coordinates.
  • the second three-dimensional coordinates may be two-dimensional coordinates of the target video object acquired in the left view and the right view of the binocular camera and the binocular acquired by the binocular camera respectively The internal and external parameters of the camera are calculated.
  • the second three-dimensional coordinate is a three-dimensional coordinate of the target video object in a second coordinate system corresponding to the binocular camera
  • the second coordinate system corresponding to the binocular camera may be a binocular camera optical center
  • the two-dimensional coordinates may specifically be corresponding pixel coordinates of the target video object in the left view and the right view of the binocular camera.
  • the positional relationship between the binocular cameras, the positional relationship between the camera and the binocular camera, and the positional relationship between the cameras of the multi-camera in the camera system can be calibrated in advance.
  • the parameter obtained by the binocular camera system calibration can be used to calculate the three-dimensional coordinates of the video object in the coordinate system corresponding to the binocular camera; the positional relationship calibration between the navigation camera and the binocular camera can be used to calculate the video object in the navigation camera coordinate system.
  • the three-dimensional coordinates of the lower three-dimensional coordinates; and the positional calibration parameters between the cameras of the multi-camera can be used to calculate the three-dimensional coordinates of the video object in the camera coordinate system of each aircraft position in the multi-camera deployment scenario, so as to facilitate coordinate conversion.
  • the deployment mode of the multi-camera camera may be the deployment mode of the above-mentioned guide camera + guide camera, or the deployment mode of the guide camera + ordinary PTZ camera.
  • Each of the navigation cameras can be referred to as a single position. When multiple cameras are used for cooperative shooting, a host position can be determined therefrom, and the rest is a slave position. As a guide camera of the slave position, the camera can be used.
  • the binocular camera includes a left camera and a right camera.
  • the image acquired by the left camera may be referred to as a left view
  • the image acquired by the right camera may be referred to as a right view.
  • the imaging (projection) model of a single camera can be described by the following formula:
  • x is a pixel coordinate of a certain point in the scene (ie, a video object, specifically a feature point corresponding to the video object) in the image coordinate system, which is a two-dimensional coordinate;
  • X is a certain point in the scene.
  • P is a 3 ⁇ 4 projection matrix.
  • PX means P ⁇ X.
  • K is a 3 ⁇ 3 camera internal reference matrix, which can be expressed as:
  • f x , f y are the equivalent focal lengths in the x and y directions
  • c x , c y are the image coordinates of the optical center
  • s is the skew coefficient of the skew (the sensor and the optical axis are not perpendicular, usually small, during the calibration process) Ignorable).
  • R and t are camera external parameters, which are represented as a 3 ⁇ 3 rotation matrix and a 3 ⁇ 1 translation vector, respectively, as follows:
  • r 1 , r 2 , r 3 are 3 ⁇ 1 column vectors in the rotation matrix.
  • the model of camera image distortion can be described according to the following formula:
  • x p , y p are the corrected pixel positions
  • x d , y d are the pre-correction pixel positions
  • k 1 , k 2 , k 3 are radial distortion coefficients
  • p 1 , p 2 are tangential distortion coefficients.
  • the positional relationship between the binocular cameras and the positional relationship between the navigation camera such as the PTZ camera and the binocular camera are fixed, and the two calibrations can be completed before leaving the factory, that is, The data obtained by the two calibrations is fixed as the internal and external data.
  • the calibration of the camera may adopt various schemes, such as the plane calibration method of Zhang (also referred to as “Zhang's calibration method”), and the distortion parameter calculation adopts the Brown method, which is not described herein. .
  • the above-mentioned binocular camera calibration principle shows that the calibration of the positional relationship of a multi-camera camera such as a multi-camera camera is to find a relative external parameter between two adjacent guide cameras, according to the position between adjacent guide cameras.
  • the relative external parameters calculate the external parameters between any two navigation cameras, thereby obtaining the positional relationship between any two navigation cameras.
  • the multi-guide camera is deployed, a large overlapping area of the camera is required between the two cameras.
  • the multiple positions are similar to the surrounding multi-camera system.
  • the rotation matrix and translation vector of the i-th camera relative to the j-th camera are:
  • R i,i-1 R i-1,i-2 ...R j+1,j denotes R i,i-1 ⁇ R i-1,i-2 ⁇ ... ⁇ R j+1,j . Since the position of the camera for positioning on different guide brackets changes according to the actual deployment scenario when the guide camera is deployed, the positional relationship between the multi-guide cameras cannot be pre-calibrated before the device leaves the factory, and can be deployed in the guide camera. Perform on-site calibration.
  • FIG. 3b is a schematic diagram of a calibration scenario of a multi-guide camera according to an embodiment of the present invention.
  • a local area network or a wireless Fidelity (Local Area Network, simply referred to as "LAN”) or a wireless fidelity can be used between two adjacent navigation cameras or between a binocular camera and a navigation camera.
  • LAN local area network
  • LAN wireless Fidelity
  • a wireless fidelity can be used between two adjacent navigation cameras or between a binocular camera and a navigation camera.
  • the transmission protocol can use a variety of network protocols, such as the HyperText Transfer Protocol ("HTTP").
  • HTTP HyperText Transfer Protocol
  • a global ID number can be assigned in advance to each of the navigation camera and the camera (binocular camera) connected to the navigation camera. For example, you can select a camera as the starting position. For example, you can select the leftmost or rightmost camera as the starting position.
  • the ID numbers of other cameras are incremented counterclockwise or clockwise.
  • a group of cameras is selected among all cameras to participate in the calibration. The principle of selection may be to ensure that the overlapping area between adjacent cameras is the largest. As shown in FIG.
  • each of the positions includes a navigation camera (recorded as PTZ0, PTZ1, and PTZ2, respectively) and a binocular camera (the binocular camera). Includes two left and right cameras, denoted as C0, C1, C2, C3, C4, C5).
  • cameras with ID numbers C0, C2, and C4 are selected for calibration, and one of the navigation cameras is selected as the calibration computing device, such as the above-mentioned navigation camera for the host position.
  • the external reference calibration between the two cameras can be performed from left to right or from right to left, and the relative external parameters between the two cameras are obtained.
  • a camera relative position relationship table can be maintained in each navigation camera, as shown in Table 1 below. Among them, each of the tables will add or update one of the entries, and each entry is uniquely determined by its two camera ID numbers.
  • the positional relationship table which is the calibration computing device, can be sent to all other navigation cameras for storage via the network. Further, according to the position relationship table and the external parameters of the binocular camera (the external reference can be calibrated at the factory), any two or two cameras in the calibration scene can be calculated (including between binocular cameras, binocular) The positional relationship between the camera and the PTZ camera and between the PTZ cameras.
  • the navigation camera D3 in FIG. 3b is a calibration computing device, and the navigation cameras D1 and D2 are cameras to be calibrated, the navigation camera D3 can be set as the calibration host position, and the other cameras are set as slaves. The position is set and the calibration is initiated by the camera D3. Before calibration, it is necessary to ensure that the cameras are interconnected through the network. The overlapping areas can be captured between the cameras that need to be calibrated, and there are calibration templates (such as checkerboard templates) in the overlap area.
  • the calibration camera is required to start the calibration process, and the image acquisition command is sent to the navigation camera D1.
  • the acquisition command includes the ID number of the navigation camera D1 (ie, D1) and the ID number of the binocular camera to be acquired (C4 or C5).
  • the navigation camera D1 After receiving the acquisition command, the navigation camera D1 performs image acquisition of the calibration template, and transmits the collected image data to the navigation camera D3. Similarly, the guide camera D3 obtains the binocular camera on the guide camera D2. The calibration template image taken. If the binocular camera to be calibrated is located on the navigation camera D3, the navigation camera D3 can directly acquire the calibration template image of the binocular camera. After obtaining the calibration template image of the camera to be calibrated, the guide camera D3 can perform checkerboard corner detection on the calibration template images of the two cameras. If the two images can detect all the checkerboard corner points, the collection can be indicated. Successful; otherwise the two images can be discarded and the image reacquired.
  • a plurality of calibration template images of the two cameras that need to be calibrated are cyclically acquired and saved in the navigation camera D3.
  • the camera D3 can be used to perform the camera. Calibration, the internal parameters of each camera have been calibrated at the factory, so it can be used as the initial input value for the calibration.
  • the relative external parameters R and T between the two cameras are obtained, and whether the re-projection (shadow) error is less than a preset threshold is calculated. If the re-injection error is greater than the threshold, the calibration failure may be indicated; otherwise, it may indicate The calibration was successful.
  • the positional relationship table can be updated based on the calculated relative position relationship of the camera and transmitted to other navigation cameras.
  • the video within the shooting range of the navigation camera can be obtained.
  • the object is positioned, and the three-dimensional position information is obtained, so as to determine an appropriate guiding camera position according to the acquired three-dimensional position information, and adjust the parameter of the guiding camera according to the guiding strategy corresponding to the three-dimensional position information, and control the positioning of the guiding camera to be suitable.
  • the location of the video object is taken.
  • the positioning of the video object includes three-dimensional positioning of the binocular camera, single-camera camera such as PTZ camera positioning, and three-dimensional positioning between the cameras of the multi-camera.
  • the stereoscopic image captured by the binocular camera can be used to calculate the depth position information of a certain observation point in the camera coordinate system, thereby determining the three-dimensional position information of the observation point.
  • This method is the same as the principle that the human eye perceives the depth distance, and is called binocular camera ranging.
  • FIG. 3c a three-dimensional positioning schematic diagram of a binocular camera is provided. The following describes the ranging principle of the binocular camera system. Among them, P is an observation point in the world coordinate system, and is imaged by two left and right cameras.
  • the position of the P point in the physical coordinate system of the left camera is X L , Y L , Z L , and the coordinates of the pixel position of the imaging point in the left view are x l , y l ;
  • the position in the physical coordinate system of the right camera is X R , Y R , Z R , the pixel position coordinates of the imaging point in the right view are x r , y r , assuming that the relative external parameters of the left and right cameras are R, T;
  • the focal lengths of the left and right cameras are: f l , f r .
  • the relationship between the imaging model of the left and right cameras and the physical coordinate position of the left and right cameras is as follows:
  • the values of x l , y l , x r , y r can be obtained by image matching, f l , f r , R, T can be obtained by binocular camera calibration, so X L , Y L , Z L can be calculated And the values of X R , Y R , Z R , thereby determining the three-dimensional coordinates of the observation points in the scene in the coordinate system corresponding to the binocular camera.
  • FIG. 3 is a schematic diagram of a PTZ camera rotation model according to an embodiment of the present invention. As shown in Fig.
  • the PTZ camera is a zoom camera, it is necessary to obtain a function relationship of the zoom factor Z and the internal parameters such as the focal length and the distortion coefficient.
  • the polynomial can be used to fit the relationship between the zoom factor Z and the focal lengths f x , f y to obtain the following relationship:
  • the camera internal parameters are obtained, the corresponding f x , f y and distortion coefficients are calculated, and the coefficients are fitted using least squares method.
  • Other internal parameters such as distortion coefficients can also be processed in a similar manner.
  • the values of ⁇ p and ⁇ t can be calculated according to the Pan/Tilt model formula.
  • the captured image sent by the binocular camera connected to other navigation cameras may be acquired, and the video object model is updated after the video object is matched.
  • Determining the target video object from the captured image obtained by the camera which may be specifically: converting the second three-dimensional coordinate into a third three-dimensional coordinate according to a pre-calibrated positional relationship between the binocular camera and the current camera; Determining whether the area of the target video object in the third three-dimensional coordinate and the area of the video object detected by the current camera in the three-dimensional coordinate of the video object exceeds a preset area threshold; if exceeded, The video object is then determined as the target video object. That is, the video object matches successfully.
  • the third three-dimensional coordinate is a three-dimensional coordinate of the target video object in a third coordinate system corresponding to the current camera, and the current camera is any other than the binocular camera in the navigation camera system.
  • a camera for example, in a multi-camera camera scene, the current camera can be a binocular camera other than the binocular camera of the host position.
  • the purpose of the three-dimensional positioning of the multi-camera video object is to calculate the three-dimensional coordinates of the video object in a binocular coordinate system of a navigation camera, and calculate it in other binocular cameras or a PZT camera coordinate system.
  • the three-dimensional coordinates It is known that a certain observation point (ie, a video object, specifically a certain feature point of the video object) has a coordinate vector X 1 in the camera D1, and an external parameter R 21 , T 21 of the camera D2 relative to the camera D1 (through The binocular camera calibration is obtained), and the coordinate vector X 2 of the observation point in the camera D2 can be calculated:
  • FIG. 4a is a schematic diagram of an object matching scenario according to an embodiment of the present invention.
  • Three-dimensional positioning of multi-camera video objects can be used to determine the correspondence of multiple video objects.
  • three-camera cameras D1, D2, and D3 are deployed in the scene.
  • FIG 4b it is a set of video objects in the image of FIG 4a, the participants O 1 which is a different view of the imaging cameras directed D1, D2 and D3 in.
  • the binocular camera in the station D1 detects the video object VO 11 by using an algorithm such as face detection, and then uses the binocular camera three-dimensional positioning algorithm to obtain the three-dimensional position of the object in the D1 binocular coordinate system.
  • D2 and D3 will also detect the video objects VO 12 and VO 13 and calculate the three-dimensional coordinates of the object in the D2 and D3 binocular coordinate systems.
  • the positional relationship between D1, D2 and D3 can be used to calibrate the three-dimensional position of the video object VO 11 in the D1 coordinate system to the D2 and D3 coordinate systems. And detect the coincident area.
  • the VO 11 , VO 12 and VO 13 are considered to be the same video object, and the video object is successfully matched. Further, if there are multiple video objects in the image that are close to each other in the image, simply determining the correspondence relationship of the video objects by using the positional coincidence region may result in a matching error. Thereby, the image information of the video object can be further combined and the accuracy of determining the correspondence can be improved by the matching algorithm.
  • the matching algorithm may include a template matching algorithm and the like.
  • a template matching algorithm may be used to take a two-dimensional image of a video object detected by a binocular camera of a certain host position such as a host bit as a known template, and perform a video object detected by a binocular camera of another station.
  • a matching finds the object that matches the video object by algorithm such as squared difference matching and correlation matching, thereby establishing the correspondence of the object.
  • Video object detection tracking and scene modeling are examples of video objects existing in the scene, and to track and identify these objects.
  • Video objects include participants, as well as scene objects such as lights, windows, conference tables, and more.
  • the system needs to cyclically process the image data of the input binocular camera, including face detection and matching, human shape detection and matching, moving object detection and matching, scene object detection and matching, etc., modeling the video object and updating the model. Parameters to model the entire shooting scene based on the detected object model.
  • the modeled model can be used for subsequent object recognition and guided strategy processing.
  • face detection can be used to detect video objects with close distances, such as participants with relatively close detection distances.
  • the face detection can obtain various parameters of the face video object, including the two-dimensional coordinates of the circumscribed rectangular area of the face, the coordinates of the center point, the area of the rectangle, and the rotation angle of the face around the coordinate axis (representing the left and right deflection of the face, pitching Parameters such as the degree of rotation and the position of the organs such as the eyes, nose, and mouth in the face.
  • the video object needs to be tracked in the video frame sequence to establish a correspondence relationship between the video objects in the time domain.
  • video object tracking algorithms include grayscale based template matching, MeanShift, CamShift, Kalman filtering algorithms and the like.
  • the video object The matching can be applied to the binocular camera, and the video object region detected in one camera image of the binocular camera is used to find a corresponding video image region in another camera image, so that the feature can be performed in the matching region of the video object. Match and calculation of 3D coordinates.
  • the matching algorithm of the video object is similar to the tracking algorithm, and grayscale-based template matching and algorithms such as MeanShift can be used.
  • a video object may be represented by its features, and commonly used features include feature points, image textures, histogram information, and the like.
  • the feature detection and matching can be performed in the detected video object region, so that the three-dimensional position information of the video object, that is, the three-dimensional coordinates can be calculated according to the feature point information, and the video object can be tracked according to the texture information and the histogram information.
  • the feature point is the main feature type, and the feature point detection algorithm includes Harris corner detection and SIFT feature point detection. Further, the feature matching is used to establish the correspondence relationship between the features of the same video object of the binocular camera.
  • the feature points can be matched by using a matching algorithm such as FLANN algorithm and KLT optical flow method, and the image texture can be matched by using a gray template matching algorithm and the like.
  • Graphs can be matched using algorithms such as histogram matching.
  • a plurality of video object models can be established in a single navigation camera coordinate system, and can be passed through a human face or a human form.
  • the motion detection tracking algorithm updates the model data.
  • each video object model can be assigned a unique ID number, and the data in the model represents the attributes of the video object.
  • the data in the model may include attributes such as an object ID, a circumscribed rectangle two-dimensional coordinate, a three-dimensional coordinate of the object feature point, a motion region texture data, a histogram data, and the like.
  • video camera model data can be exchanged between multiple guide cameras through network communication.
  • the above-mentioned multi-guide camera can be used to generate three-dimensional images.
  • the algorithm for locating and matching the video object establishes a correspondence relationship between the video object models, thereby obtaining a guiding strategy for the entire scene.
  • the network communication protocol during communication can adopt a standard protocol such as the HTTP protocol, or a custom protocol, and the data of the video object model is formatted according to a format such as an eXtensible Markup Language (“XML”) format. , packaging and transmission.
  • XML eXtensible Markup Language
  • the scene model contains models of multiple video objects, reflecting the characteristics of the video objects and their distribution in three dimensions.
  • the camera needs to maintain the scene model, including adding and deleting object models and object model properties. For example, when a new participant appears in the scene, when the binocular camera detects a new face or a humanoid object, the object model is created and added to the object model set; when the participant leaves the scene, the object is deleted. The model; after the participant's position changes, the parameters of the corresponding object model are updated. To develop a navigation strategy based on the latest video object model, select the camera with the best position for shooting.
  • one or more guiding camera positions of the best shooting effect can be selected according to the preset guiding strategy.
  • the camera having the better shooting effect is determined according to the eye-to-eye effect parameter, the occlusion relationship parameter, and the scene object parameter of the shooting area.
  • the eye-to-eye effect needs to be determined according to the optical axis angle of the face/human object relative to the PTZ camera. The smaller the angle, the more the face is presented in a positive face manner, and the eye-to-eye effect is better.
  • the face/human shape detection algorithm obtains a rotation angle (left and right deflection, pitch and rotation angle) of the three-dimensional coordinate axis centered on the face/human shape with respect to the binocular camera coordinate system, and uses the aforementioned camera.
  • the formula for the conversion of the coordinate system converts the rotation angle of the face/human shape relative to the binocular camera to the rotation angle of the relative PTZ camera.
  • a PTZ camera priority queue with an eye-to-eye effect can be further established for each video object, and a camera with better eye-to-eye effect has a higher priority.
  • the area of the video object detected by a certain camera may be known according to the projection equation of the camera, and the single-guided camera binocular camera and the calibration may be utilized.
  • the external parameters between the PTZ cameras and the external parameters between the binocular cameras of different positions re-enter the area onto the imaging plane of each position PTZ camera. If the regions of the two video objects overlap, the depth information can be used to determine the occlusion relationship between the two objects, that is, the video object closer to the binocular camera will block the farther video object.
  • a PTZ camera priority queue can be established for each video object with an occlusion relationship, and an unoccluded camera has a higher priority.
  • the system also detects other video objects (scene objects) of interest in the scene, such as lamps, windows, conference tables, and the like.
  • the detection of these objects can employ algorithms based on image color and edge features, and the like.
  • the Canny operator can be used to extract the edge of the tube to obtain its long straight line feature, and then detect whether there is an overexposed pixel area (lighting area) in the adjacent area. According to these two features, it can be detected. Exit the tube object and get the coordinates of its circumscribed rectangle. The detection of the window is similar to the detection of the tube.
  • the quadrilateral feature can be obtained by edge detection, and then whether the window is determined according to whether there is a certain area of the overexposed pixel area in the quadrilateral.
  • the conference table can also be detected using edge features in the image.
  • the guide camera of the navigation camera such as the host position, establishes a priority queue for the PTZ cameras of each station according to the acquired image effect parameters and the preset guide strategy, and can determine the camera to be selected.
  • one or more video objects that need to be photographed that is, a target video object, such as a talking video object determined according to the sound source localization result, may be determined to capture a close-up of the video object; or an AutoFrame strategy is required.
  • Pan/Tilt can be adjusted to include all the video objects in the scene, and Zoom adjusts the object to the appropriate size, and so on.
  • a PTZ camera priority queue using eye-to-eye effect parameters, occlusion relationship parameters, and scene object parameters can determine a comprehensive PTZ camera priority queue according to a certain guiding strategy.
  • the navigation policy may be automatically calculated by the system or preset by the user, which is not limited by the embodiment of the present invention.
  • the unobstructed PTZ camera select the PTZ camera with the best image object parameters and the best image as the target camera. .
  • the host bit can adjust the PTZ parameters of the selected PTZ camera according to the three-dimensional coordinates of the target video object to obtain the best image effect as much as possible. For example, during voice tracking, when shooting a close-up of a participant, avoid shooting an object that affects the brightness of the image, such as a lamp or a window; when adjusting the Zoom size, avoid adjusting the white balance effect of the large-area table object on the image. and many more.
  • the host bit (the host position of the navigation camera) can output the selected PTZ camera video image or ID.
  • the host bit can directly output an image of the selected camera; for a multi-guide camera system output through the video matrix, the host bit can pass through a communication interface (such as a serial port or a network port).
  • the ID of the selected PTZ camera is output to the video matrix, and the camera image is switched by the video matrix.
  • the target camera that captures the target video object is selected from the cameras of the navigation camera system according to a preset guiding strategy, and is obtained.
  • a three-dimensional coordinate of the target video object in a coordinate system corresponding to the target camera to control the target camera to perform camera parameter adjustment according to the three-dimensional coordinates of the target video object, and output a video image after adjusting the imaging parameter, so that the guided camera system It can improve the accuracy of video object detection and tracking based on three-dimensional coordinate detection and preset navigation strategy, and improve the efficiency of camera parameter adjustment, and effectively improve the camera's shooting effect.
  • FIG. 5 is a schematic structural diagram of a parameter adjustment apparatus according to an embodiment of the present invention.
  • the device in the embodiment of the present invention may be specifically configured in the above-mentioned navigation camera.
  • the parameter adjustment device in the embodiment of the present invention may include an object determining unit 10, a selecting unit 20, and an acquiring unit. 30 and parameter adjustment unit 40. among them,
  • the object determining unit 10 is configured to determine a target video object that needs to be photographed.
  • the selecting unit 20 is configured to select, from a camera of the navigation camera system where the navigation camera is located, a target camera for capturing the target video object according to a preset navigation policy.
  • the shooting effect parameter may include any one or more of an eye-to-eye effect parameter, an occlusion relationship parameter, and a scene object parameter of the shooting area of the target video object in a coordinate system corresponding to the current camera.
  • the current camera is any camera other than the binocular camera in the navigation camera system.
  • the eye-to-eye effect parameter may include a rotation angle of the target video object with respect to a coordinate system corresponding to the current camera, and the rotation angle may be a rotation angle of the target video object in the second coordinate system. And determining a positional relationship between the binocular camera and the current camera that is pre-calibrated.
  • the occlusion relationship parameter and the scene object parameter may be that the area of the scene object detected by the current camera is re-injected to the location according to a pre-calibrated positional relationship between the binocular camera and the current camera.
  • the imaging plane of the current camera is determined.
  • the acquiring unit 30 is configured to acquire first three-dimensional coordinates of the target video object.
  • the first three-dimensional coordinate may be a three-dimensional coordinate of the target video object in a first coordinate system corresponding to the target camera.
  • the target camera may be the above-mentioned navigation camera or an ordinary PTZ camera.
  • the first coordinate system corresponding to the target camera may refer to a three-dimensional coordinate system established with the optical center of the target camera as the origin, or the origin of any other reference object.
  • the established three-dimensional coordinate system is not limited in the embodiment of the present invention.
  • the parameter adjustment unit 40 is configured to adjust an imaging parameter of the target camera to an imaging parameter corresponding to the first three-dimensional coordinate, and output a video image after adjusting the imaging parameter.
  • the obtaining unit 30 may be specifically configured to:
  • the second three-dimensional coordinates are converted into first three-dimensional coordinates according to a pre-calibrated positional relationship between the binocular camera and the target camera.
  • the second three-dimensional coordinates may be two-dimensional coordinates of the target video object acquired in the left view and the right view of the binocular camera and the binocular acquired by the binocular camera respectively The internal and external parameters of the camera are calculated.
  • the second three-dimensional coordinate is a three-dimensional coordinate of the target video object in a second coordinate system corresponding to the binocular camera, and the second coordinate system corresponding to the binocular camera may be a binocular camera optical center
  • the two-dimensional coordinates may specifically be corresponding pixel coordinates of the target video object in the left view and the right view of the binocular camera.
  • the object determining unit 10 may be specifically configured to:
  • the selecting unit 20 can be specifically configured to:
  • the camera whose shooting effect parameter satisfies the preset guiding strategy is determined as the target camera for capturing the target video object.
  • the selecting unit 20 performs the determination of the target video object from the captured image acquired by the camera.
  • the third three-dimensional coordinate is a three-dimensional coordinate of the target video object in a third coordinate system corresponding to the current camera;
  • the video object is determined to be the target video object.
  • the target camera that captures the target video object is selected from the cameras of the navigation camera system according to a preset guiding strategy, and is obtained.
  • a three-dimensional coordinate of the target video object in a coordinate system corresponding to the target camera to control the target camera to perform camera parameter adjustment according to the three-dimensional coordinates of the target video object, and output a video image after adjusting the imaging parameter, so that the guided camera system It can improve the accuracy of video object detection and tracking based on three-dimensional coordinate detection and preset navigation strategy, and improve the efficiency of camera parameter adjustment, and effectively improve the camera's shooting effect.
  • FIG. 6 is a schematic structural diagram of a navigation camera system according to an embodiment of the present invention.
  • the navigation camera system of the embodiment of the present invention may include a first camera 1 and at least one second camera 2, the first camera 1 includes a navigation camera 11 and a binocular camera 12, and the navigation camera 11 and the camera Between the binocular cameras 12, the first camera 1 and the second camera 2 can be connected through a wired interface or a wireless interface;
  • the guidance camera 11 is configured to determine a target video object that needs to be captured, and select a target camera for capturing the target video object from a camera of the navigation camera system according to a preset navigation strategy;
  • the binocular camera 12 is configured to acquire a second three-dimensional coordinate of the target video object, and transmit the second three-dimensional coordinate to the navigation camera 11; wherein the second three-dimensional coordinate is the target video The three-dimensional coordinates of the object in the second coordinate system corresponding to the binocular camera 12;
  • the navigation camera 11 is configured to receive the second three-dimensional coordinates transmitted by the binocular camera 12; and according to a pre-calibrated positional relationship between the binocular camera 12 and the target camera, the second three-dimensional coordinates Converting to a first three-dimensional coordinate; adjusting an imaging parameter of the target camera to an imaging parameter corresponding to the first three-dimensional coordinate, and outputting a video image after adjusting the imaging parameter; wherein the first three-dimensional coordinate is the The three-dimensional coordinates of the target video object in the first coordinate system corresponding to the target camera.
  • the second camera 2 may also include a navigation camera and a binocular camera, and the target camera may be any of the navigation cameras in the navigation camera system; or the second camera 2 is a normal PTZ camera.
  • the target camera can be the guide camera or a normal PTZ camera.
  • the binocular camera 12 can be disposed on a preset guide bracket and connected to the guide camera 11 through the guide bracket.
  • FIG. 7 it is a schematic structural diagram of a first camera provided by an embodiment of the present invention.
  • the first camera includes a binocular camera and one or more guide cameras. It is assumed that in the embodiment of the present invention, the first camera is equipped with two navigation cameras for guiding shooting and tracking, which can be wired or wirelessly connected to the binocular camera through a guiding bracket (referred to as "cradle").
  • the binocular camera is mounted on the bracket.
  • a microphone can be mounted on the bracket, and the installed microphone can be in the form of an array.
  • the microphone in the array can be used for realizing sound source positioning, sound source identification and the like. Includes a horizontal array of microphones and a vertical array of microphones.
  • the guide camera and the bracket may be separated or integrated, and a communication interface such as a serial interface may be used for communication between the guide camera and the bracket.
  • a communication interface such as a serial interface
  • the above-mentioned navigation camera and the guide bracket may be integrated into one guide device.
  • the connection form of each device in the navigation camera system is not limited in the embodiment of the present invention.
  • FIG. 8 is a schematic diagram of networking of a guided camera system according to an embodiment of the present invention.
  • the multi-camera network includes multiple inter-camera networking with guided cameras, and the camera and guide brackets are installed + multiple common PTZ camera groups. Net, the position of the camera and the guide bracket is installed + the position of the station without PTZ camera (that is, only the guide bracket), and the position without the PTZ camera + the network of multiple ordinary PTZ cameras (ie, no guide bracket).
  • the cameras of each station can be interconnected by LAN or Wi-Fi to transmit control messages, including camera switching messages, audio and video data such as video object model data, and the like.
  • control message may be transmitted through an Internet Protocol (IP), such as an IP Camera protocol stack.
  • IP Internet Protocol
  • a binning camera is required between the binocular cameras in the two positions.
  • IP Camera IP Camera protocol stack.
  • a binning camera is required between the binocular cameras in the two positions.
  • the switching policy of the video matrix may be controlled by any specified navigation camera in the scenario, such as a navigation camera as a host, or controlled by a third-party device, which is not limited in the embodiment of the present invention.
  • the video image output by the video matrix is encoded by the codec device and transmitted to the far end for video conferencing.
  • the video data can be processed in cascade (the navigation bracket supports video cascading); if the number is large, the video of multiple cameras is output to the video matrix for processing, by the video matrix. Switch or synthesize one or more camera video sources.
  • the bracket can provide external video input / Output interface, LAN/Wi-Fi network port and serial interface.
  • the video input interface is used for external input video of other cameras; the video output interface is used to connect terminals or video matrix devices to output video images; the serial interface provides control and debugging interface for the bracket; LAN/Wi-Fi
  • the network port is used for cascading multiple camera positions, and can transmit audio and video data and control data.
  • the plurality of navigation cameras have video object detection capability and PTZ camera function, and one of the navigation cameras can be used as a host position, and is responsible for outputting the position selection and PTZ camera control, and other cameras.
  • the slave position; the position of the guide camera and the guide bracket + multiple common PTZ cameras only one guide camera has video object detection capability, which is responsible for output position selection and PTZ camera control, and the ordinary camera is only used as a PTZ camera. Since only the navigation camera has the video object detection capability, the data of the video object model of the camera position is not obtained through the network, and the matching process of the multi-camera video object model is performed.
  • FIG. 9 is a schematic structural diagram of a navigation camera according to an embodiment of the present invention, for performing the above camera parameter adjustment method.
  • the navigation camera of the embodiment of the present invention includes: a communication interface 300, a memory 200, and a processor 100, and the processor 100 is respectively connected to the communication interface 300 and the memory 200.
  • the memory 200 may be a high speed RAM memory or a non-volatile memory such as at least one disk memory.
  • the communication interface 300, the memory 200, and the processor 100 may be connected to each other through a bus, or may be connected by other means. In the present embodiment, a bus connection will be described.
  • the device structure shown in FIG. 10 does not constitute a limitation on the embodiments of the present invention, and may include more or less components than those illustrated, or a combination of certain components, or different component arrangements. among them:
  • the processor 100 is a control center of the device, which connects various parts of the entire device using various interfaces and lines, by running or executing programs and/or units stored in the memory 200, and calling driver software stored in the memory 200, Perform various functions and process data of the device.
  • the processor 100 may be composed of an integrated circuit ("IC"), for example, may be composed of a single packaged IC, or may be composed of a plurality of packaged ICs that have the same function or different functions.
  • the processor 100 may include only a central processing unit (“CPU"), or may be a CPU, a digital signal processor ("DSP"), or a graphics processor ( Graphic Processing Unit (referred to as "GPU") and a combination of various control chips.
  • the CPU may be a single operation core, and may also include multiple operation cores.
  • Communication interface 300 can include a wired interface, a wireless interface, and the like.
  • the memory 200 can be used to store driver software (or software programs) and units, and the processor 100 and the communication interface 300 perform various functional applications of the devices and implement data processing by calling the driver software and the units stored in the memory 200.
  • the memory 200 mainly includes a program storage area and a data storage area, wherein the program storage area can store driver software and the like required for at least one function; the data storage area can store data according to the parameter adjustment process, such as the above-described three-dimensional coordinate information.
  • the processor 100 reads the driver software from the memory 200 and executes it under the action of the driver software:
  • first three-dimensional coordinate of the target video object where the first three-dimensional coordinate is a three-dimensional coordinate of the target video object in a first coordinate system corresponding to the target camera;
  • the processor 100 reads the driver software from the memory 200 and performs the acquiring the first three-dimensional coordinates of the target video object by using the driver software, and specifically performing the following steps:
  • the second three-dimensional coordinates are converted into first three-dimensional coordinates according to a pre-calibrated positional relationship between the binocular camera and the target camera.
  • the processor 100 reads the driver software from the memory 200 and performs the determining of the target video object that needs to be captured by the driver software, and specifically performs the following steps:
  • the processor 100 reads the driver software from the memory 200 and performs the function of the driver software to select from the cameras of the navigation camera system where the camera is located according to a preset navigation policy. For the target camera for capturing the target video object, perform the following steps:
  • the camera whose shooting effect parameter satisfies the preset guiding strategy is determined as the target camera for capturing the target video object.
  • the processor 100 reads the driver software from the memory 200 and performs the determination of the target video object from the captured image acquired by the camera under the action of the driver software, and specifically performs the following steps:
  • the third three-dimensional coordinate is a three-dimensional coordinate of the target video object in a third coordinate system corresponding to the current camera;
  • the video object is determined to be the target video object.
  • the shooting effect parameter may include any one or more of an eye-to-eye effect parameter, an occlusion relationship parameter, and a scene object parameter of the shooting area of the target video object in a coordinate system corresponding to the current camera.
  • the current camera is any camera other than the binocular camera in the navigation camera system.
  • the eye-to-eye effect parameter may include a rotation angle of the target video object relative to a coordinate system corresponding to the current camera, the rotation angle being according to a rotation angle of the target video object in the second coordinate system and The pre-calibrated positional relationship between the binocular camera and the current camera is determined.
  • the occlusion relationship parameter and the scene object parameter may be according to the binocular camera pre-calibrated. And a positional relationship between the current camera, and re-casting an area of the scene object detected by the current camera to an imaging plane of the current camera.
  • the disclosed apparatus and method may be implemented in other manners.
  • the device embodiments described above are merely illustrative.
  • the division of the unit is only a logical function division.
  • there may be another division manner for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored or not executed.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be in an electrical, mechanical or other form.
  • the units described as separate components may or may not be physically separated, and the components displayed as the unit may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. . Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.
  • each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
  • the above integrated unit can be implemented in the form of hardware or in the form of hardware plus software functional units.
  • the above-described integrated unit implemented in the form of a software functional unit can be stored in a computer readable storage medium.
  • the above software functional unit is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor to perform the methods of the various embodiments of the present invention. Part of the steps.
  • the foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (Read-Only Memory, simply referred to as "ROM"), a random access memory (RAM), a magnetic disk, or an optical disk, and the like.
  • the medium of the program code includes: a U disk, a mobile hard disk, a read-only memory (Read-Only Memory, simply referred to as "ROM"), a random access memory (RAM), a magnetic disk, or an optical disk, and the like.
  • the medium of the program code includes: a U disk, a mobile hard disk, a read-only

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Studio Devices (AREA)

Abstract

La présente invention concerne, dans ses modes de réalisation, un procédé de réglage des paramètres d'une caméra, une caméra dirigeant la diffusion, et un système de tournage dirigeant la diffusion. Le procédé comporte les étapes consistant à: déterminer un objet vidéo cible à filmer; selon une politique de direction prédéfinie, sélectionner une caméra cible pour filmer l'objet vidéo cible, parmi des caméras d'un système de tournage dirigeant la diffusion dans lequel est située la caméra dirigeant la diffusion; acquérir les premières coordonnées tridimensionnelles de l'objet vidéo cible, les premières coordonnées tridimensionnelles étant les coordonnées tridimensionnelles de l'objet vidéo cible dans un premier système de coordonnées correspondant à la caméra cible; et régler les paramètres de tournage de la caméra cible pour qu'ils soient des paramètres de tournage correspondant aux premières coordonnées tridimensionnelles, et délivrer des images vidéo après que les paramètres de tournage ont été réglés. En utilisant la présente solution, le rendement de réglage de paramètres de caméra peut être amélioré, et l'effet de tournage de la caméra peut être renforcé.
PCT/CN2017/091863 2016-07-18 2017-07-05 Procédé de réglage de paramètres de caméra, caméra dirigeant la diffusion, et système de tournage dirigeant la diffusion WO2018014730A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201610562671.6 2016-07-18
CN201610562671.6A CN106251334B (zh) 2016-07-18 2016-07-18 一种摄像机参数调整方法、导播摄像机及系统

Publications (1)

Publication Number Publication Date
WO2018014730A1 true WO2018014730A1 (fr) 2018-01-25

Family

ID=57613157

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/091863 WO2018014730A1 (fr) 2016-07-18 2017-07-05 Procédé de réglage de paramètres de caméra, caméra dirigeant la diffusion, et système de tournage dirigeant la diffusion

Country Status (2)

Country Link
CN (1) CN106251334B (fr)
WO (1) WO2018014730A1 (fr)

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110969662A (zh) * 2018-09-28 2020-04-07 杭州海康威视数字技术股份有限公司 鱼眼摄像机内参标定方法、装置、标定装置控制器和系统
CN111080679A (zh) * 2020-01-02 2020-04-28 东南大学 一种对大型场所室内人员动态跟踪定位的方法
CN111243029A (zh) * 2018-11-28 2020-06-05 驭势（上海）汽车科技有限公司 视觉传感器的标定方法以及装置
CN111325790A (zh) * 2019-07-09 2020-06-23 杭州海康威视系统技术有限公司 目标追踪方法、设备及系统
CN112106110A (zh) * 2018-04-27 2020-12-18 上海趋视信息科技有限公司 标定相机的系统和方法
CN112468680A (zh) * 2019-09-09 2021-03-09 上海御正文化传播有限公司 一种广告拍摄现场合成处理系统的处理方法
CN112819770A (zh) * 2021-01-26 2021-05-18 中国人民解放军陆军军医大学第一附属医院 碘对比剂过敏监测方法及系统
CN113129376A (zh) * 2021-04-22 2021-07-16 青岛联合创智科技有限公司 一种基于棋盘格的相机实时定位方法
CN113587895A (zh) * 2021-07-30 2021-11-02 杭州三坛医疗科技有限公司 双目测距方法及装置
CN113610932A (zh) * 2021-08-20 2021-11-05 苏州智加科技有限公司 双目相机外参标定方法和装置
CN113838146A (zh) * 2021-09-26 2021-12-24 昆山丘钛光电科技有限公司 验证摄像头模组标定精度、摄像头模组测试方法及装置
CN114025107A (zh) * 2021-12-01 2022-02-08 北京七维视觉科技有限公司 图像重影的拍摄方法、装置、存储介质和融合处理器
CN117523431A (zh) * 2023-11-17 2024-02-06 中国科学技术大学 一种烟火检测方法、装置、电子设备及存储介质

Families Citing this family (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106251334B (zh) * 2016-07-18 2019-03-01 华为技术有限公司 一种摄像机参数调整方法、导播摄像机及系统
US10091412B1 (en) * 2017-06-30 2018-10-02 Polycom, Inc. Optimal view selection method in a video conference
CN109413359B (zh) * 2017-08-16 2020-07-28 华为技术有限公司 摄像跟踪方法、装置及设备
CN109922251B (zh) * 2017-12-12 2021-10-22 华为技术有限公司 快速抓拍的方法、装置及系统
CN109031201A (zh) * 2018-06-01 2018-12-18 深圳市鹰硕技术有限公司 基于行为识别的语音定位方法以及装置
CN108900860A (zh) * 2018-08-23 2018-11-27 佛山龙眼传媒科技有限公司 一种导播控制方法及装置
CN109360250A (zh) * 2018-12-27 2019-02-19 爱笔（北京）智能科技有限公司 一种对摄像装置的标定方法、设备及系统
CN109712188A (zh) * 2018-12-28 2019-05-03 科大讯飞股份有限公司 一种目标跟踪方法及装置
CN111787243B (zh) * 2019-07-31 2021-09-03 北京沃东天骏信息技术有限公司 导播方法、装置及计算机可读存储介质
CN110456829B (zh) * 2019-08-07 2022-12-13 深圳市维海德技术股份有限公司 定位跟踪方法、装置及计算机可读存储介质
CN110737798B (zh) * 2019-09-26 2022-10-14 万翼科技有限公司 室内巡检方法及相关产品
CN111080698B (zh) * 2019-11-27 2023-06-06 上海新时达机器人有限公司 长型板材位置标定方法、系统和存储装置
CN111131697B (zh) * 2019-12-23 2022-01-04 北京中广上洋科技股份有限公司 一种多摄像机智能跟踪拍摄方法、系统、设备及存储介质
CN113516717A (zh) * 2020-04-10 2021-10-19 富华科精密工业（深圳）有限公司 摄像装置外参标定方法、电子设备及存储介质
CN111698467B (zh) * 2020-05-08 2022-05-06 北京中广上洋科技股份有限公司 基于多摄像机的智能跟踪方法及系统
CN113808199B (zh) * 2020-06-17 2023-09-08 华为云计算技术有限公司 定位方法、电子设备与定位系统
CN111800590B (zh) * 2020-07-06 2022-11-25 深圳博为教育科技有限公司 一种导播控制方法、装置、系统及控制主机
CN112887653B (zh) * 2021-01-25 2022-10-21 联想(北京)有限公司 一种信息处理方法和信息处理装置
CN113453021B (zh) * 2021-03-24 2022-04-29 北京国际云转播科技有限公司 人工智能导播方法、系统、服务器和计算机可读存储介质
CN113271482A (zh) * 2021-05-17 2021-08-17 广东彼雍德云教育科技有限公司 一种便捷式全幅抠像黑板
CN116389660B (zh) * 2021-12-22 2024-04-12 广州开得联智能科技有限公司 录播的导播方法、装置、设备及存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090303329A1 (en) * 2008-06-06 2009-12-10 Mitsunori Morisaki Object image displaying system
CN101630406A (zh) * 2008-07-14 2010-01-20 深圳华为通信技术有限公司 摄像机的标定方法及摄像机标定装置
CN102638672A (zh) * 2011-02-09 2012-08-15 宝利通公司 用于多流多站点远程呈现会议系统的自动视频布局
CN102843540A (zh) * 2011-06-20 2012-12-26 宝利通公司 用于视频会议的自动摄像机选择
CN106251334A (zh) * 2016-07-18 2016-12-21 华为技术有限公司 一种摄像机参数调整方法、导播摄像机及系统

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104869365B (zh) * 2015-06-02 2018-12-18 阔地教育科技有限公司 一种基于直录播系统的鼠标跟踪方法及装置
CN105049764B (zh) * 2015-06-17 2018-05-25 武汉智亿方科技有限公司 一种基于多个定位摄像头的教学用图像跟踪方法及系统
CN105718862A (zh) * 2016-01-15 2016-06-29 北京市博汇科技股份有限公司 一种单摄像头教师自动跟踪方法、装置及录播系统

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090303329A1 (en) * 2008-06-06 2009-12-10 Mitsunori Morisaki Object image displaying system
CN101630406A (zh) * 2008-07-14 2010-01-20 深圳华为通信技术有限公司 摄像机的标定方法及摄像机标定装置
CN102638672A (zh) * 2011-02-09 2012-08-15 宝利通公司 用于多流多站点远程呈现会议系统的自动视频布局
CN102843540A (zh) * 2011-06-20 2012-12-26 宝利通公司 用于视频会议的自动摄像机选择
CN106251334A (zh) * 2016-07-18 2016-12-21 华为技术有限公司 一种摄像机参数调整方法、导播摄像机及系统

Cited By (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11468598B2 (en) 2018-04-27 2022-10-11 Shanghai Truthvision Information Technology Co., Ltd. System and method for camera calibration
CN112106110A (zh) * 2018-04-27 2020-12-18 上海趋视信息科技有限公司 标定相机的系统和方法
CN110969662B (zh) * 2018-09-28 2023-09-26 杭州海康威视数字技术股份有限公司 鱼眼摄像机内参标定方法、装置、标定装置控制器和系统
CN110969662A (zh) * 2018-09-28 2020-04-07 杭州海康威视数字技术股份有限公司 鱼眼摄像机内参标定方法、装置、标定装置控制器和系统
CN111243029B (zh) * 2018-11-28 2023-06-23 驭势（上海）汽车科技有限公司 视觉传感器的标定方法以及装置
CN111243029A (zh) * 2018-11-28 2020-06-05 驭势（上海）汽车科技有限公司 视觉传感器的标定方法以及装置
CN111325790B (zh) * 2019-07-09 2024-02-20 杭州海康威视系统技术有限公司 目标追踪方法、设备及系统
CN111325790A (zh) * 2019-07-09 2020-06-23 杭州海康威视系统技术有限公司 目标追踪方法、设备及系统
CN112468680A (zh) * 2019-09-09 2021-03-09 上海御正文化传播有限公司 一种广告拍摄现场合成处理系统的处理方法
CN111080679A (zh) * 2020-01-02 2020-04-28 东南大学 一种对大型场所室内人员动态跟踪定位的方法
CN112819770A (zh) * 2021-01-26 2021-05-18 中国人民解放军陆军军医大学第一附属医院 碘对比剂过敏监测方法及系统
CN113129376A (zh) * 2021-04-22 2021-07-16 青岛联合创智科技有限公司 一种基于棋盘格的相机实时定位方法
CN113587895A (zh) * 2021-07-30 2021-11-02 杭州三坛医疗科技有限公司 双目测距方法及装置
CN113610932A (zh) * 2021-08-20 2021-11-05 苏州智加科技有限公司 双目相机外参标定方法和装置
CN113838146A (zh) * 2021-09-26 2021-12-24 昆山丘钛光电科技有限公司 验证摄像头模组标定精度、摄像头模组测试方法及装置
CN114025107B (zh) * 2021-12-01 2023-12-01 北京七维视觉科技有限公司 图像重影的拍摄方法、装置、存储介质和融合处理器
CN114025107A (zh) * 2021-12-01 2022-02-08 北京七维视觉科技有限公司 图像重影的拍摄方法、装置、存储介质和融合处理器
CN117523431A (zh) * 2023-11-17 2024-02-06 中国科学技术大学 一种烟火检测方法、装置、电子设备及存储介质

Also Published As

Publication number Publication date
CN106251334A (zh) 2016-12-21
CN106251334B (zh) 2019-03-01

Similar Documents

Publication Publication Date Title
WO2018014730A1 (fr) Procédé de réglage de paramètres de caméra, caméra dirigeant la diffusion, et système de tournage dirigeant la diffusion
WO2017215295A1 (fr) Procédé de réglage de paramètres de caméra, caméra robotisée et système
US10368011B2 (en) Camera array removing lens distortion
US20200275079A1 (en) Generating three-dimensional video content from a set of images captured by a camera array
US9832583B2 (en) Enhancement of audio captured by multiple microphones at unspecified positions
CN105247881B (zh) 信息处理设备、显示控制方法以及程序
US8749607B2 (en) Face equalization in video conferencing
JP7179515B2 (ja) 装置、制御方法、及びプログラム
JP2024056955A (ja) 光学式捕捉によるパーソナライズされたｈｒｔｆ
CN112311965A (zh) 虚拟拍摄方法、装置、系统及存储介质
JP2021514573A (ja) マルチセンサを使用してオムニステレオビデオを捕捉するためのシステム及び方法
JP2019083402A (ja) 画像処理装置、画像処理システム、画像処理方法、及びプログラム
WO2016184131A1 (fr) Procédé et appareil de photographie d'image basés sur des appareils photographiques doubles et support de stockage informatique
US10186301B1 (en) Camera array including camera modules
KR20050084263A (ko) 비디오 폰 이미지에서 머리 자세를 보정하기 위한 방법 및장치
JP2019191989A (ja) 仮想視点画像を生成するシステム、方法及びプログラム
JP5963006B2 (ja) 画像変換装置、カメラ、映像システム、画像変換方法およびプログラムを記録した記録媒体
JP2019022151A (ja) 情報処理装置、画像処理システム、制御方法、及び、プログラム
TW201824178A (zh) 全景即時影像處理方法
JP2023502552A (ja) ウェアラブルデバイス、インテリジェントガイド方法及び装置、ガイドシステム、記憶媒体
CN108053376A (zh) 一种语义分割信息指导深度学习鱼眼图像校正方法
JP7040511B2 (ja) 情報処理装置および方法
EP4075794A1 (fr) Ajustement des paramètres d'une caméra basé sur une région d'intérêt dans un environnement de téléconférence
JP2005142765A (ja) 撮像装置及び方法
WO2016197788A1 (fr) Procédé et dispositif de photographie

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17830364

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17830364

Country of ref document: EP

Kind code of ref document: A1