CN111131713A - Lens switching method, device, equipment and computer readable storage medium - Google Patents
Lens switching method, device, equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN111131713A CN111131713A CN201911422375.6A CN201911422375A CN111131713A CN 111131713 A CN111131713 A CN 111131713A CN 201911422375 A CN201911422375 A CN 201911422375A CN 111131713 A CN111131713 A CN 111131713A
- Authority
- CN
- China
- Prior art keywords
- close
- shot
- determining
- optical flow
- panoramic
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/698—Control of cameras or camera modules for achieving an enlarged field of view, e.g. panoramic image capture
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/695—Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Studio Devices (AREA)
Abstract
The invention discloses a lens switching method, which comprises the following steps: if an uprising object is detected based on the panoramic lens, determining the position information of the uprising object; determining a down tilt of the panoramic shot relative to a close-up shot and calculating coordinate information of the standing object relative to the close-up shot based on the position information and the down tilt; determining a number of steps of the close-up shot based on the coordinate information, and controlling the close-up shot to move based on the number of steps; and switching the close-up shot collection close-up picture if the close-up shot movement is detected to be completed. The invention also discloses a lens switching device, equipment and a computer readable storage medium. The invention firstly determines the position information of the standing object through the panoramic shot, then converts the position information into the movement information of the close-up shot, then controls the close-up shot to accurately move to a corresponding position according to the movement information, and then switches the close-up shot to acquire the close-up shot, thereby realizing the automatic switching of the shot.
Description
Technical Field
The present invention relates to the field of recording and broadcasting technologies, and in particular, to a method, an apparatus, a device, and a computer-readable storage medium for switching a lens.
Background
In recent years, recording and broadcasting systems are widely recognized and used in the teaching field, and each recording and broadcasting system comprises a recording and broadcasting host, a teacher close-up, a teacher full view, student close-up, a student full view, a blackboard writing and other multi-digit digital cameras, and a teacher microphone, a student microphone and other equipment. In practical application, in order to achieve a better recording and broadcasting effect, the recording and broadcasting system needs to switch the shots to full-shot or close-up shot collection pictures at all times, for example, when a student answers a question immediately, the close-up shot of the student needs to be switched to collect the close-up pictures of the student, and the like.
With the development of intelligent terminal technology, the traditional manual lens switching mode is gradually eliminated due to the consumption of manpower and delay in the switching process. Although the existing recording and broadcasting system can realize automatic switching of the lens, the number of students is large, the distribution is dense, the students are overlapped in a picture, and the standing up process of the students is short, so that the lens is difficult to master the switching opportunity and the switching precision.
Disclosure of Invention
The invention mainly aims to provide a lens switching method, a device, equipment and a computer readable storage medium, aiming at realizing automatic switching of lenses.
In order to achieve the above object, the present invention provides a lens switching method, including the steps of:
if an uprising object is detected based on the panoramic lens, determining the position information of the uprising object;
determining a down tilt of the panoramic shot relative to a close-up shot and calculating coordinate information of the standing object relative to the close-up shot based on the position information and the down tilt;
determining a number of steps of the close-up shot based on the coordinate information, and controlling the close-up shot to move based on the number of steps;
and switching the close-up shot collection close-up picture if the close-up shot movement is detected to be completed.
Preferably, if the standing object is detected based on the panoramic lens, the step of determining the position information of the standing object includes:
based on a panoramic lens, acquiring a panoramic picture, and configuring angular points in the panoramic picture as first optical flow points so as to perform optical flow tracking on the first optical flow points;
if the first optical flow point has an upward-shifted target optical flow point, performing density clustering on the target optical flow point based on a density clustering algorithm to obtain a preselected region;
configuring the corner points in the preselected area as second optical flow points, recording the initial positions of the second optical flow points, and carrying out optical flow tracking on the second optical flow points;
and if the vertical direction displacement difference between the current position of the second optical flow point and the initial position is greater than a first preset threshold value within a preset time, determining that the standing object is detected, and determining the position information of the standing object.
Preferably, after the step of switching the close-up shot capture frame based on a preset director strategy if the close-up shot movement is detected to be completed, the shot switching method further comprises:
continuing to perform optical flow tracking on the second optical flow points;
and if the vertical direction displacement difference between the current position of the second optical flow point and the initial position is smaller than a second preset threshold value, determining that the standing object is in the home position, and switching the panoramic lens to acquire a panoramic picture.
Preferably, the step of determining a down tilt of the panoramic view relative to the close-up view, and calculating coordinate information of the standing object relative to the close-up view based on the position information and the down tilt comprises:
determining a down tilt angle of the panoramic shot relative to a close-up shot and calculating a horizontal offset angle and a vertical offset angle of the standing object relative to the close-up shot based on the position information;
determining coordinate information of the standing object with respect to the close-up based on the downtilt angle, the horizontal offset angle, and the vertical offset angle.
Preferably, the step of determining the number of steps of the close-up based on the coordinate information, and controlling the close-up movement based on the number of steps includes:
determining a step angle of the close-up;
determining a horizontal direction step number of the close-up shot based on the horizontal offset angle and the step angle;
determining a vertical direction step count for the close-up shot based on the vertical offset angle, the downtilt angle, and the step angle;
controlling the close-up movement based on the horizontal direction step number and the vertical direction step number.
Preferably, the step of switching the close-up shot close-up view if completion of the close-up shot movement is detected comprises:
and if the close-up shot movement is detected to be completed, determining a director strategy corresponding to the standing object, and switching the close-up shot collection close-up picture based on the director strategy.
Preferably, the step of switching the close-up shot close-up view if completion of the close-up shot movement is detected comprises:
and if the close-up shot movement is detected to be completed, switching the close-up shot, amplifying the position information of the standing object by preset times based on the close-up shot to obtain a close-up picture of the standing object, and acquiring the close-up picture.
Further, to achieve the above object, the present invention also provides a lens switching device including:
the panoramic camera comprises a detection module, a storage module and a control module, wherein the detection module is used for determining the position information of an upright object if the upright object is detected based on a panoramic lens;
a conversion module for determining a down tilt angle of the panoramic shot relative to a close-up shot and calculating coordinate information of the standing object relative to the close-up shot based on the position information and the down tilt angle;
a tracking module for determining a number of steps of the close-up based on the coordinate information and controlling the close-up to move based on the number of steps;
and the directing module is used for switching the close-up shot collection close-up picture if the close-up shot movement is detected to be completed.
Preferably, the detection module is configured to:
based on a panoramic lens, acquiring a panoramic picture, and configuring angular points in the panoramic picture as first optical flow points so as to perform optical flow tracking on the first optical flow points;
if the first optical flow point has an upward-shifted target optical flow point, performing density clustering on the target optical flow point based on a density clustering algorithm to obtain a preselected region;
configuring the corner points in the preselected area as second optical flow points, recording the initial positions of the second optical flow points, and carrying out optical flow tracking on the second optical flow points;
and if the vertical direction displacement difference between the current position of the second optical flow point and the initial position is greater than a first preset threshold value within a preset time, determining that the standing object is detected, and determining the position information of the standing object.
Preferably, the detection module is further configured to:
continuing to perform optical flow tracking on the second optical flow points;
and if the vertical direction displacement difference between the current position of the second optical flow point and the initial position is smaller than a second preset threshold value, determining that the standing object is in the home position, and switching the panoramic lens to acquire a panoramic picture.
Preferably, the scaling module is further configured to:
determining a down tilt angle of the panoramic shot relative to a close-up shot and calculating a horizontal offset angle and a vertical offset angle of the standing object relative to the close-up shot based on the position information;
determining coordinate information of the standing object with respect to the close-up based on the downtilt angle, the horizontal offset angle, and the vertical offset angle.
Preferably, the tracking module is further configured to:
determining a step angle of the close-up;
determining a horizontal direction step number of the close-up shot based on the horizontal offset angle and the step angle;
determining a vertical direction step count for the close-up shot based on the vertical offset angle, the downtilt angle, and the step angle;
controlling the close-up movement based on the horizontal direction step number and the vertical direction step number.
Preferably, the director module is further configured to:
and if the close-up shot movement is detected to be completed, determining a director strategy corresponding to the standing object, and switching the close-up shot collection close-up picture based on the director strategy.
Preferably, the director module is further configured to:
and if the close-up shot movement is detected to be completed, switching the close-up shot, amplifying the position information of the standing object by preset times based on the close-up shot to obtain a close-up picture of the standing object, and acquiring the close-up picture.
Further, to achieve the above object, the present invention also provides a lens switching apparatus comprising: a memory, a processor and a shot-cut program stored on the memory and executable on the processor, the shot-cut program when executed by the processor implementing the steps of the shot-cut method as described above.
Further, to achieve the above object, the present invention also provides a computer-readable storage medium having a lens switching program stored thereon, which when executed by a processor, implements the steps of the lens switching method as described above.
According to the lens switching method, if the standing object is detected based on the panoramic lens, the position information of the standing object is determined; determining a down tilt of the panoramic shot relative to a close-up shot and calculating coordinate information of the standing object relative to the close-up shot based on the position information and the down tilt; determining a number of steps of the close-up shot based on the coordinate information, and controlling the close-up shot to move based on the number of steps; and switching the close-up shot collection close-up picture if the close-up shot movement is detected to be completed. The invention firstly determines the position information of the standing object through the panoramic shot, then converts the position information into the movement information of the close-up shot, then controls the close-up shot to accurately move to a corresponding position according to the movement information, and then switches the close-up shot to acquire the close-up shot, thereby realizing the automatic switching of the shot.
Drawings
FIG. 1 is a schematic diagram of an apparatus architecture of a hardware operating environment according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating a lens switching method according to a first embodiment of the present invention;
FIG. 3 is a schematic view of a lens shifting apparatus according to a first embodiment of the lens shifting method of the present invention;
FIG. 4 is a schematic diagram illustrating the calculation of the horizontal offset angle and the vertical offset angle in the first embodiment of the lens switching method according to the present invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
As shown in fig. 1, fig. 1 is a schematic device structure diagram of a hardware operating environment according to an embodiment of the present invention.
The device of the embodiment of the invention can be a PC or a server device.
As shown in fig. 1, the apparatus may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, a communication bus 1002. Wherein a communication bus 1002 is used to enable connective communication between these components. The user interface 1003 may include a Display screen (Display), an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface). The memory 1005 may be a high-speed RAM memory or a non-volatile memory (e.g., a magnetic disk memory). The memory 1005 may alternatively be a storage device separate from the processor 1001.
Those skilled in the art will appreciate that the configuration of the apparatus shown in fig. 1 is not intended to be limiting of the apparatus and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components.
As shown in fig. 1, a memory 1005, which is a kind of computer storage medium, may include therein an operating system, a network communication module, a user interface module, and a lens changeover program.
The operating system is a program for managing and controlling the lens switching equipment and software resources, and supports the running of a network communication module, a user interface module, a lens switching program and other programs or software; the network communication module is used for managing and controlling the network interface 1002; the user interface module is used to manage and control the user interface 1003.
In the lens barrel switching apparatus shown in fig. 1, the lens barrel switching apparatus calls a lens barrel switching program stored in a memory 1005 by a processor 1001 and performs operations in the respective embodiments of the lens barrel switching method described below.
Based on the hardware structure, the embodiment of the lens switching method is provided.
Referring to fig. 2, fig. 2 is a schematic flowchart of a lens switching method according to a first embodiment of the present invention, where the method includes:
step S10, if an upright object is detected based on the panoramic lens, determining position information of the upright object;
a step S20 of determining a down tilt of the panoramic view with respect to the close-up view, and calculating coordinate information of the standing object with respect to the close-up view based on the position information and the down tilt;
a step S30 of determining the number of steps of the close-up shot based on the coordinate information, and controlling the close-up shot to move based on the number of steps;
and step S40, if the close-up shot movement is detected to be completed, switching the close-up shot collection close-up picture.
In the specific implementation, as shown in fig. 3, the switching device may be an integrated binocular camera including a panoramic camera and a close-up camera, wherein the panoramic camera includes a start/sit detection module (detection module for short) for detecting whether there is a target, i.e., a standing object, standing up or returning when the panoramic camera collects a panoramic picture; a coordinate conversion module (conversion module for short) for calculating the position information of the target relative to the close-up shot, namely the coordinate information of the close-up shot, according to the position information of the target relative to the panoramic picture; the tracking module is used for acquiring the current position of the close-up camera, namely the downward inclination angle of the close-up camera with the panoramic lens, and then controlling the close-up camera to perform mobile tracking according to the calculated coordinate information; and the director module is used for outputting director switching information according to the target tracking state, namely whether the close-up shot moves completely, and switching the close-up shot to collect pictures so as to continuously provide the recording and broadcasting host (namely the third-party equipment) for continuous recording and broadcasting.
In this embodiment, the panoramic lens gathers panoramic video image and is used for detecting whether the standing object stands up, and the close-up lens comprises cloud platform and optics zoom camera, can realize that the standing object fixed point zoom shoots, and wherein, the cloud platform can be two degrees of freedom cloud platforms. In an embodiment, the switching device is provided with a haisi Hi3516a/3519v101 Chip, so that the switching device can be accelerated by combining a hardware IVE (intelligent video acceleration engine, which is a hardware module in a System-on-a-Chip) of a built-in SOC (System-on-a-Chip) of the haisi Hi3516a/3519v101 in the process of realizing automatic lens switching, and real-time multi-target detection and tracking are realized.
The switching equipment of this embodiment gathers panoramic video image through panoramic lens earlier, like the student panorama in the classroom, then, through the panoramic video image of gathering, real-time supervision exists the standing object, also whether also has the student to stand up, and after detecting the standing object, confirm the positional information of standing object, also determine specifically which student stands up, thereby control close-up lens tracks the standing object, and switch the shot from panoramic lens to close-up lens, adopt close-up lens to gather close-up picture, make in the recorded broadcast process, can switch over panoramic lens and close-up lens automatically, the video image of recording both includes panoramic lens, also includes close-up lens.
The respective steps will be described in detail below:
in step S10, if an upright object is detected based on the panoramic lens, position information of the upright object is determined.
In this embodiment, the switching device is used for recording in a classroom, and in order to clearly reflect the attendance state of all students in the classroom, the switching device first captures a panoramic image in the classroom using a panoramic lens, wherein the panoramic image includes all objects to be raised, that is, all students, and it can be understood that during the class, the students may stand up to speak, and in order to clearly reflect the spiritual features of the raised objects, that is, the students, the switching device needs to switch the panoramic lens to a close-up lens, and capture the close-up image of the raised students using the close-up lens, and therefore, when capturing the panoramic image, the switching device needs to monitor in real time whether the raised objects exist, so as to determine whether the panoramic image needs to be switched to the close-up lens.
If the switching device detects the standing object based on the panoramic lens, that is, the switching device detects that a student stands up through the panoramic lens, the switching device determines the position information of the standing object, wherein the position information is the position information of the standing object relative to the panoramic picture, that is, which student stands up in a classroom is specifically determined.
Specifically, step S10 includes:
step S11, collecting a panoramic picture based on a panoramic lens, and configuring corner points in the panoramic picture as first optical flow points so as to perform optical flow tracking on the first optical flow points;
in this step, the switching device captures the panoramic image through the panoramic lens, and configures ST (Shi-Tomasi) corner points in the panoramic image as first optical flow points to perform optical flow tracking on the first optical flow points, where the corner points are interest points, and are customized by a user, such as points with gray values exceeding a threshold value, and in a specific implementation, the optical flow tracking may be performed on the first optical flow points by using an optical flow method, such as Lucas-kanade (lk) optical flow method, and since the optical flow method is a relatively mature algorithm for calculating the motion of the same object between different frames, detailed description is omitted here.
Step S12, if the first optical flow point has an upward-shifted target optical flow point, performing density clustering on the target optical flow point based on a density clustering algorithm to obtain a preselected region;
if the target optical flow points which are displaced upwards exist, density clustering is performed on the target optical flow points, that is, each optical flow point is detected according to an optical flow method, whether each optical flow point is displaced upwards is determined, if yes, the upwardly displaced optical flow points are reserved, the reserved optical flow points are the target optical flow points, then density clustering is performed on the target optical flow points according to a density clustering algorithm, so as to obtain a clustering result, wherein the density clustering algorithm is a density-based clustering algorithm.
Further, in another embodiment, the step S12 includes:
step a1, if there is a target optical flow point which is displaced upwards, recording the number of the target optical flow point;
step a2, if the number is larger than the preset number, density clustering is carried out on the target light stream points based on a density clustering algorithm to obtain a preselected region.
In another embodiment, it is considered that a panoramic picture captured by a panoramic lens may affect tracking of optical flow points due to existence of noise, and therefore, if it is determined that there are target optical flow points that are displaced upward, the number of the target optical flow points is recorded, and density clustering is performed on the target optical flow points based on a density clustering algorithm when the number of the target optical flow points reaches a preset number, that is, when the number of the target optical flow points is not greater than the preset number, density clustering is considered to be caused by the noise and is not required.
Step S13, configuring the corner points in the preselected area as second optical flow points, recording the initial positions of the second optical flow points, and carrying out optical flow tracking on the second optical flow points;
after obtaining the preselected area, the ST angular points are re-taken from the preselected area, the initial positions of the ST angular points are recorded, the angular points are configured into second optical flow points, the optical flow tracking is continuously carried out on the second optical flow points, and the current positions of the ST angular points successfully tracked by the optical flow tracking are recorded
Step S14, if the vertical displacement difference between the current position of the second optical flow point and the initial position is greater than a first preset threshold within a preset time, determining that an upright object is detected, and determining the position information of the upright object.
If the vertical displacement difference between the current position and the initial position of the second optical flow point is greater than a first preset threshold value within a period of time, it is determined that an object stands at the position, that is, it is determined that the standing object is detected, so as to determine the position information of the standing object.
Step S20, determining a down tilt of the panoramic view relative to the close-up view, and calculating coordinate information of the standing object relative to the close-up view based on the position information and the down tilt.
In this embodiment, after determining the position information of the standing object, determining a down tilt angle of the panoramic lens with respect to the close-up lens, and calculating coordinate information of the standing object with respect to the close-up lens according to the determined position information of the standing object and the down tilt angle, where the down tilt angle is an included angle γ between the close-up lens and the panoramic lens, and the down tilt angle γ is a determined value obtained by measurement.
Specifically, step S20 includes:
a step b1 of determining the downward inclination angle of the panoramic shot relative to the close-up shot and calculating the horizontal offset angle and the vertical offset angle of the standing object relative to the close-up shot based on the position information;
in this step, the downtilt angle γ of the panoramic lens with respect to the close-up lens is determined first, and since the switching apparatus of the present embodiment is an integral binocular camera, the value of the downtilt angle γ can be determined by measurement.
And then according to the position information of the standing object, calculating the horizontal offset angle and the vertical offset angle of the standing object relative to the close-up shot. Referring to fig. 4, the character "a" indicates the detected standing object, the target rectangle indicates a close-up image of the standing object a, and in the implementation, a rectangle enlarged by a certain factor according to the position information of the standing object is used as the target rectangle, and since the coordinates of the target rectangle are known, the linear lengths of AC, AB, BC, OC, OA, OB can be calculated, and in the implementation, the linear lengths are expressed in pixels.
Finally, θ ═ atan (BC/AC × (α/2)) can be obtained from tan (α/2) ═ AC/OC and tan (θ) ═ BC/OC, and similarly, from β, Φ can be calculated, θ representing the horizontal offset angle and Φ representing the vertical offset angle.
Step b2, determining coordinate information of the standing object relative to the close-up shot based on the down tilt angle, the horizontal offset angle and the vertical offset angle.
And finally, determining the coordinate information of the standing object relative to the close-up shot according to the downward inclination angle gamma, the horizontal offset angle theta and the vertical offset angle phi, wherein the finally calculated coordinate information of the standing object relative to the close-up shot is (theta, phi + gamma).
Step S30, determining the number of steps of the close-up shot based on the coordinate information, and controlling the close-up shot to move based on the number of steps.
In the embodiment, the step number of the close-up shot, that is, the movement information of the close-up shot is determined according to the calculated coordinate information, so that the close-up shot is controlled to move according to the movement information.
Specifically, step S30 includes:
step c1, determining the step angle of the close-up shot;
in this embodiment, the angle variation amount of the two-degree-of-freedom pan/tilt head of the specific close-up shot, which is the physical characteristic of the two-degree-of-freedom pan/tilt head of the close-up shot, is rotated one step forward, and the switching device may determine the step angle of the close-up shot according to the model of the pan/tilt head of the read close-up shot.
Step c2, determining the horizontal direction step number of the close-up shot based on the horizontal offset angle and the step angle;
and then, determining the horizontal direction step number of the close-up shot according to the horizontal offset angle and the step angle, and specifically dividing the horizontal offset angle by the step angle to obtain the horizontal direction step number, wherein the horizontal direction step number is the step number of the two-degree-of-freedom holder of the close-up shot rotating in the horizontal direction, and taking the current step angle as 0.069 degree/step as an example, the horizontal direction step number P of the close-up shot is theta/0.069.
Step c3 determining a vertical direction step number for the close-up shot based on the vertical offset angle, the downtilt angle, and the step angle;
and then, determining the vertical direction step number of the close-up shot according to the vertical offset angle, the step angle and the declination angle, and specifically dividing the sum of the vertical offset angle and the declination angle by the step angle to obtain the vertical direction step number, wherein the vertical direction step number is the step number of the two-degree-of-freedom holder of the close-up shot rotating in the vertical direction, and taking the current step angle as 0.069 degree/step as an example, the vertical direction step number T of the close-up shot is (gamma + phi)/0.069.
Step c4, controlling the close-up movement based on the number of steps in the horizontal direction and the number of steps in the vertical direction.
And finally, controlling the close-up shot to move according to the obtained horizontal direction stepping number and the vertical direction stepping number, namely controlling the two-degree-of-freedom holder of the close-up shot to move according to the horizontal direction stepping number and the vertical direction stepping number.
And step S40, if the close-up shot movement is detected to be completed, switching the close-up shot collection close-up picture.
In this embodiment, if it is detected that the close-up shot has moved to the target position, that is, the close-up shot has moved to the target position, indicating that the close-up shot has completed tracking the standing object, the close-up shot is switched to collect the close-up shot, and recording is continued.
In this embodiment, if an uprising object is detected based on a panoramic lens, position information of the uprising object is determined; determining a down tilt of the panoramic shot relative to a close-up shot and calculating coordinate information of the standing object relative to the close-up shot based on the position information and the down tilt; determining a number of steps of the close-up shot based on the coordinate information, and controlling the close-up shot to move based on the number of steps; and switching the close-up shot collection close-up picture if the close-up shot movement is detected to be completed. The invention firstly determines the position information of the standing object through the panoramic shot, then converts the position information into the movement information of the close-up shot, then controls the close-up shot to accurately move to a corresponding position according to the movement information, and then switches the close-up shot to acquire the close-up shot, thereby realizing the automatic switching of the shot.
Further, a second embodiment of the lens switching method of the present invention is provided based on the first embodiment of the lens switching method of the present invention.
The second embodiment of the lens switching method is different from the first embodiment of the lens switching method in that step S40 includes:
and d, if the close-up shot movement is detected to be completed, determining a directing strategy corresponding to the standing object, and switching the close-up shot collection close-up picture based on the directing strategy.
In this embodiment, if it is detected that the close-up shot movement is completed, a director policy corresponding to the standing object is determined, where the director policy may be a custom director policy.
In specific implementation, the director strategy is arranged in the order of priority from high to low:
1. close-up of students: the close-up shot is performed by the horizontal/vertical stepping number, the student close-up shooting is completed, and the director can automatically switch to the student close-up picture at the moment;
2. panoramic view of students: the system indicates that the director can be automatically switched to a panoramic picture of the student when the standing up/returning of the student is detected;
3. and (3) close-up of a teacher: indicating that the director can automatically switch to a teacher close-up view;
4. panoramic teacher: the presentation director may automatically switch to the instructor panorama picture.
In another embodiment, if it is detected that the close-up shot movement is completed, the identity of the standing-up object is determined, and a corresponding director strategy is determined according to the identity of the standing-up object, that is, a plurality of director strategies can be customized, and then a plurality of flexible director strategies, such as a panoramic close-up exchange director and the like, are implemented according to the identity of the current standing-up object.
Further, step S40 includes:
and if the close-up shot movement is detected to be completed, switching the close-up shot, amplifying the position information of the standing object by preset times based on the close-up shot to obtain a close-up picture of the standing object, and acquiring the close-up picture.
And switching the close-up shot to collect the picture after detecting that the close-up shot moves, specifically determining the amplification position according to the position information of the standing object, and amplifying the picture by a certain multiple if the face of the standing object is used as the amplification center to obtain the picture which is the collected close-up picture.
Further, after step S40, the lens switching method further includes:
step e, continuing to perform optical flow tracking on the second optical flow points;
and f, if the vertical direction displacement difference between the current position of the second optical flow point and the initial position is smaller than a second preset threshold value, determining that the standing object is in the home position, and switching the panoramic lens to collect panoramic pictures.
The switching device of this embodiment needs to switch the shots according to the actual situation, and therefore, after switching the shot from the panoramic shot to the close-up shot, it needs to continue to detect whether the condition for switching the shot from the close-up shot back to the panoramic shot exists, and therefore, it needs to continue to perform optical flow tracking on the second optical flow point, and determine whether the shot needs to be switched back to the panoramic shot from the close-up shot, and if so, switch the panoramic shot to capture the panoramic picture.
The respective steps will be described in detail below:
and e, continuing to perform optical flow tracking on the second optical flow points.
In this embodiment, it can be understood that the time when the student stands up to speak is generally short, so that the student can sit back to the seat at last, and when the student sits back to the seat, the panoramic lens needs to be used for collecting the panoramic image, so that the optical flow tracking needs to be continuously performed on the second optical flow point, thereby determining whether the standing object is parked, and the specific tracking manner is not described herein again. In a specific implementation, the switching device may continue to perform optical flow tracking on the second optical flow points through the panoramic lens.
And f, if the vertical direction displacement difference between the current position of the second optical flow point and the initial position is smaller than a second preset threshold value, determining that the standing object is in the home position, and switching the panoramic lens to collect panoramic pictures.
In this embodiment, if it is determined that the vertical direction displacement difference between the current position and the initial position of the second optical flow point is smaller than the second preset threshold, it is determined that the standing object is parked, that is, the student finishes speaking and sits back to the original position, at this time, a close-up shot is no longer needed for close-up, and therefore, the panoramic shot needs to be switched to collect the panoramic picture, and recording and playing continue.
In the lens switching process, the director switching is performed according to a director strategy except for the switching time, so that the lenses are switched flexibly while the lenses are switched automatically, and the intelligence is improved.
The invention also provides a lens switching device. The lens switching device of the invention comprises:
the panoramic camera comprises a detection module, a storage module and a control module, wherein the detection module is used for determining the position information of an upright object if the upright object is detected based on a panoramic lens;
a conversion module for determining a down tilt angle of the panoramic shot relative to a close-up shot and calculating coordinate information of the standing object relative to the close-up shot based on the position information and the down tilt angle;
a tracking module for determining a number of steps of the close-up based on the coordinate information and controlling the close-up to move based on the number of steps;
and the directing module is used for switching the close-up shot collection close-up picture if the close-up shot movement is detected to be completed.
Further, the detection module is configured to:
based on a panoramic lens, acquiring a panoramic picture, and configuring angular points in the panoramic picture as first optical flow points so as to perform optical flow tracking on the first optical flow points;
if the first optical flow point has an upward-shifted target optical flow point, performing density clustering on the target optical flow point based on a density clustering algorithm to obtain a preselected region;
configuring the corner points in the preselected area as second optical flow points, recording the initial positions of the second optical flow points, and carrying out optical flow tracking on the second optical flow points;
and if the vertical direction displacement difference between the current position of the second optical flow point and the initial position is greater than a first preset threshold value within a preset time, determining that the standing object is detected, and determining the position information of the standing object.
Further, the detection module is further configured to:
continuing to perform optical flow tracking on the second optical flow points;
and if the vertical direction displacement difference between the current position of the second optical flow point and the initial position is smaller than a second preset threshold value, determining that the standing object is in the home position, and switching the panoramic lens to acquire a panoramic picture.
Further, the scaling module is further configured to:
determining a down tilt angle of the panoramic shot relative to a close-up shot and calculating a horizontal offset angle and a vertical offset angle of the standing object relative to the close-up shot based on the position information;
determining coordinate information of the standing object with respect to the close-up based on the downtilt angle, the horizontal offset angle, and the vertical offset angle.
Further, the tracking module is further configured to:
determining a step angle of the close-up;
determining a horizontal direction step number of the close-up shot based on the horizontal offset angle and the step angle;
determining a vertical direction step count for the close-up shot based on the vertical offset angle, the downtilt angle, and the step angle;
controlling the close-up movement based on the horizontal direction step number and the vertical direction step number.
Further, the director module is further configured to:
and if the close-up shot movement is detected to be completed, determining a director strategy corresponding to the standing object, and switching the close-up shot collection close-up picture based on the director strategy.
Further, the director module is further configured to:
and if the close-up shot movement is detected to be completed, switching the close-up shot, amplifying the position information of the standing object by preset times based on the close-up shot to obtain a close-up picture of the standing object, and acquiring the close-up picture.
The invention also provides a computer readable storage medium.
The computer readable storage medium of the present invention has stored thereon a lens switching program, which when executed by a processor implements the steps of the lens switching method as described above.
The method implemented when the lens switching program executed on the processor is executed may refer to various embodiments of the lens switching method of the present invention, and details thereof are not repeated herein.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) as described above and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present invention.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (10)
1. A lens switching method is characterized by comprising the following steps:
if an uprising object is detected based on the panoramic lens, determining the position information of the uprising object;
determining a down tilt of the panoramic shot relative to a close-up shot and calculating coordinate information of the standing object relative to the close-up shot based on the position information and the down tilt;
determining a number of steps of the close-up shot based on the coordinate information, and controlling the close-up shot to move based on the number of steps;
and switching the close-up shot collection close-up picture if the close-up shot movement is detected to be completed.
2. The shot-cut method as claimed in claim 1, wherein the step of determining the position information of the standing object if the standing object is detected based on the panorama shot comprises:
based on a panoramic lens, acquiring a panoramic picture, and configuring angular points in the panoramic picture as first optical flow points so as to perform optical flow tracking on the first optical flow points;
if the first optical flow point has an upward-shifted target optical flow point, performing density clustering on the target optical flow point based on a density clustering algorithm to obtain a preselected region;
configuring the corner points in the preselected area as second optical flow points, recording the initial positions of the second optical flow points, and carrying out optical flow tracking on the second optical flow points;
and if the vertical direction displacement difference between the current position of the second optical flow point and the initial position is greater than a first preset threshold value within a preset time, determining that the standing object is detected, and determining the position information of the standing object.
3. The shot-cut method as claimed in claim 2, wherein after the step of switching the close-up shot capture shot based on a preset director strategy if the close-up shot movement is detected to be completed, the shot-cut method further comprises:
continuing to perform optical flow tracking on the second optical flow points;
and if the vertical direction displacement difference between the current position of the second optical flow point and the initial position is smaller than a second preset threshold value, determining that the standing object is in the home position, and switching the panoramic lens to acquire a panoramic picture.
4. The shot-cut method as claimed in claim 1, wherein the step of determining a down-tilt of the panoramic shot with respect to a close-up shot, and calculating coordinate information of the standing object with respect to the close-up shot based on the position information and the down-tilt comprises:
determining a down tilt angle of the panoramic shot relative to a close-up shot and calculating a horizontal offset angle and a vertical offset angle of the standing object relative to the close-up shot based on the position information;
determining coordinate information of the standing object with respect to the close-up based on the downtilt angle, the horizontal offset angle, and the vertical offset angle.
5. The shot-cut method as claimed in claim 4, wherein the step of determining the number of steps of the close-up shot based on the coordinate information and controlling the movement of the close-up shot based on the number of steps comprises:
determining a step angle of the close-up;
determining a horizontal direction step number of the close-up shot based on the horizontal offset angle and the step angle;
determining a vertical direction step count for the close-up shot based on the vertical offset angle, the downtilt angle, and the step angle;
controlling the close-up movement based on the horizontal direction step number and the vertical direction step number.
6. The shot changing method as claimed in claim 1, wherein the step of changing the close-up taking close-up picture if the close-up movement completion is detected comprises:
and if the close-up shot movement is detected to be completed, determining a director strategy corresponding to the standing object, and switching the close-up shot collection close-up picture based on the director strategy.
7. The shot changing method as claimed in any one of claims 1 to 6, wherein the step of changing over the close-up shot close-up view if completion of the close-up shot movement is detected comprises:
and if the close-up shot movement is detected to be completed, switching the close-up shot, amplifying the position information of the standing object by preset times based on the close-up shot to obtain a close-up picture of the standing object, and acquiring the close-up picture.
8. A lens switching device, characterized by comprising:
the panoramic camera comprises a detection module, a storage module and a control module, wherein the detection module is used for determining the position information of an upright object if the upright object is detected based on a panoramic lens;
a conversion module for determining a down tilt angle of the panoramic shot relative to a close-up shot and calculating coordinate information of the standing object relative to the close-up shot based on the position information and the down tilt angle;
a tracking module for determining a number of steps of the close-up based on the coordinate information and controlling the close-up to move based on the number of steps;
and the directing module is used for switching the close-up shot collection close-up picture if the close-up shot movement is detected to be completed.
9. A lens switching apparatus, characterized in that the lens switching apparatus comprises: memory, processor and a shot-cut program stored on the memory and executable on the processor, the shot-cut program when executed by the processor implementing the steps of the shot-cut method as claimed in any one of claims 1 to 7.
10. A computer-readable storage medium, having a shot-cut program stored thereon, which when executed by a processor implements the steps of a shot-cut method as claimed in any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911422375.6A CN111131713B (en) | 2019-12-31 | 2019-12-31 | Lens switching method, device, equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911422375.6A CN111131713B (en) | 2019-12-31 | 2019-12-31 | Lens switching method, device, equipment and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111131713A true CN111131713A (en) | 2020-05-08 |
| CN111131713B CN111131713B (en) | 2022-03-08 |
Family
ID=70507053
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911422375.6A Active CN111131713B (en) | 2019-12-31 | 2019-12-31 | Lens switching method, device, equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111131713B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113055599A (en) * | 2021-03-29 | 2021-06-29 | 维沃软件技术有限公司 | Camera switching method and device, electronic equipment and readable storage medium |
| CN115171200A (en) * | 2022-09-08 | 2022-10-11 | 深圳市维海德技术股份有限公司 | Target tracking close-up method and device based on zooming, electronic equipment and medium |
| CN115205981A (en) * | 2022-09-08 | 2022-10-18 | 深圳市维海德技术股份有限公司 | Standing posture detection method and device, electronic equipment and readable storage medium |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103281492A (en) * | 2013-05-23 | 2013-09-04 | 深圳锐取信息技术股份有限公司 | Video picture switching method, video picture switching system, recording and broadcasting server and video recording and broadcasting system |
| CN104301669A (en) * | 2014-09-12 | 2015-01-21 | 重庆大学 | Suspicious target detection tracking and recognition method based on dual-camera cooperation |
| CN105955311A (en) * | 2016-05-11 | 2016-09-21 | 阔地教育科技有限公司 | Tracking control method, tracking control device and tracking control system |
| CN106780565A (en) * | 2016-11-15 | 2017-05-31 | 天津大学 | A kind of many students based on light stream and k means clusters rise and sit detection method |
| CN206237504U (en) * | 2016-11-30 | 2017-06-09 | 深圳市维海德技术股份有限公司 | A kind of intelligent tutoring tracking camera |
| CN206541199U (en) * | 2017-03-10 | 2017-10-03 | 武汉东信同邦信息技术有限公司 | A kind of cloud platform control system of monopod video camera |
| CN107330924A (en) * | 2017-07-07 | 2017-11-07 | 郑州仁峰软件开发有限公司 | A kind of method that moving object is recognized based on monocular cam |
| CN107396059A (en) * | 2017-08-24 | 2017-11-24 | 杭州凡龙科技有限公司 | A kind of panorama position adaptive student graphical analysis control method |
| CN206674094U (en) * | 2017-04-17 | 2017-11-24 | 上海景瑞信息技术有限公司 | Video camera automatic control system based on recognition of face |
| CN107452021A (en) * | 2016-04-19 | 2017-12-08 | 深圳正谱云教育技术有限公司 | Camera to automatically track system and method based on single-lens image Dynamic Recognition |
| CN206988735U (en) * | 2017-07-12 | 2018-02-09 | 浙江理工大学 | A kind of real training class course teaching device for being used to various visual angles image be presented |
| CN108810394A (en) * | 2017-04-28 | 2018-11-13 | 惠忠州 | A kind of recording and broadcasting system of intelligence instructor in broadcasting |
| CN110456829A (en) * | 2019-08-07 | 2019-11-15 | 深圳市维海德技术股份有限公司 | Positioning and tracing method, device and computer readable storage medium |
-
2019
- 2019-12-31 CN CN201911422375.6A patent/CN111131713B/en active Active
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103281492A (en) * | 2013-05-23 | 2013-09-04 | 深圳锐取信息技术股份有限公司 | Video picture switching method, video picture switching system, recording and broadcasting server and video recording and broadcasting system |
| CN104301669A (en) * | 2014-09-12 | 2015-01-21 | 重庆大学 | Suspicious target detection tracking and recognition method based on dual-camera cooperation |
| CN107452021A (en) * | 2016-04-19 | 2017-12-08 | 深圳正谱云教育技术有限公司 | Camera to automatically track system and method based on single-lens image Dynamic Recognition |
| CN105955311A (en) * | 2016-05-11 | 2016-09-21 | 阔地教育科技有限公司 | Tracking control method, tracking control device and tracking control system |
| CN106780565A (en) * | 2016-11-15 | 2017-05-31 | 天津大学 | A kind of many students based on light stream and k means clusters rise and sit detection method |
| CN206237504U (en) * | 2016-11-30 | 2017-06-09 | 深圳市维海德技术股份有限公司 | A kind of intelligent tutoring tracking camera |
| CN206541199U (en) * | 2017-03-10 | 2017-10-03 | 武汉东信同邦信息技术有限公司 | A kind of cloud platform control system of monopod video camera |
| CN206674094U (en) * | 2017-04-17 | 2017-11-24 | 上海景瑞信息技术有限公司 | Video camera automatic control system based on recognition of face |
| CN108810394A (en) * | 2017-04-28 | 2018-11-13 | 惠忠州 | A kind of recording and broadcasting system of intelligence instructor in broadcasting |
| CN107330924A (en) * | 2017-07-07 | 2017-11-07 | 郑州仁峰软件开发有限公司 | A kind of method that moving object is recognized based on monocular cam |
| CN206988735U (en) * | 2017-07-12 | 2018-02-09 | 浙江理工大学 | A kind of real training class course teaching device for being used to various visual angles image be presented |
| CN107396059A (en) * | 2017-08-24 | 2017-11-24 | 杭州凡龙科技有限公司 | A kind of panorama position adaptive student graphical analysis control method |
| CN110456829A (en) * | 2019-08-07 | 2019-11-15 | 深圳市维海德技术股份有限公司 | Positioning and tracing method, device and computer readable storage medium |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113055599A (en) * | 2021-03-29 | 2021-06-29 | 维沃软件技术有限公司 | Camera switching method and device, electronic equipment and readable storage medium |
| CN113055599B (en) * | 2021-03-29 | 2023-02-07 | 维沃软件技术有限公司 | Camera switching method and device, electronic equipment and readable storage medium |
| CN115171200A (en) * | 2022-09-08 | 2022-10-11 | 深圳市维海德技术股份有限公司 | Target tracking close-up method and device based on zooming, electronic equipment and medium |
| CN115205981A (en) * | 2022-09-08 | 2022-10-18 | 深圳市维海德技术股份有限公司 | Standing posture detection method and device, electronic equipment and readable storage medium |
| CN115205981B (en) * | 2022-09-08 | 2023-01-31 | 深圳市维海德技术股份有限公司 | Standing posture detection method and device, electronic equipment and readable storage medium |
| CN115171200B (en) * | 2022-09-08 | 2023-01-31 | 深圳市维海德技术股份有限公司 | Target tracking close-up method and device based on zooming, electronic equipment and medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111131713B (en) | 2022-03-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111131713B (en) | Lens switching method, device, equipment and computer readable storage medium | |
| CN103716594B (en) | Panorama splicing linkage method and device based on moving target detecting | |
| CN103716595B (en) | Panoramic mosaic video camera and ball machine inter-linked controlling method and device | |
| CN106791485B (en) | Video switching method and device | |
| US10277798B2 (en) | Multiple lenses system, operation method and electronic device employing the same | |
| CN1156151C (en) | Automatic determining preset position corresponding to television conference attendance | |
| CN102148965B (en) | Video monitoring system for multi-target tracking close-up shooting | |
| US7512883B2 (en) | Portable solution for automatic camera management | |
| US20230021863A1 (en) | Monitoring method, electronic device and storage medium | |
| CN109495686B (en) | Shooting method and equipment | |
| CN106713740B (en) | Positioning tracking camera shooting method and system | |
| JP2015208001A (en) | Method and electronic device for implementing refocusing | |
| CN108259752A (en) | A kind of image pickup method and system | |
| CN104919791A (en) | Image processing device, image processing method and program | |
| CN110166694B (en) | Photographing prompting method and device | |
| US20160314596A1 (en) | Camera view presentation method and system | |
| CN112738397A (en) | Shooting method, shooting device, electronic equipment and readable storage medium | |
| CN107645628B (en) | Information processing method and device | |
| JP2013153375A (en) | Image processing apparatus, image processing method, and recording medium | |
| EP3062506B1 (en) | Image switching method and apparatus | |
| CN112019759A (en) | Method, device and equipment for tracking and shooting students in recorded and broadcast classes | |
| CN112995507A (en) | Method and device for prompting object position | |
| CN112839165B (en) | Method and device for realizing face tracking camera shooting, computer equipment and storage medium | |
| WO2012158017A1 (en) | Method and system for multiple objects tracking and display | |
| US8345125B2 (en) | Object detection using an in-sensor detector |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |