WO2021026782A1 - Control method and control apparatus for handheld cradle head, handheld cradle head and storage medium - Google Patents

Abstract

A control method and control apparatus for a handheld cradle head, the handheld cradle head and a storage medium. A specific posture in an image (50) collected by a photographing device (13) on a handheld cradle head is detected, and an object to follow in the image (50) is determined according to the specific posture, the handheld cradle head is further controlled to follow said object, and during the process of following, the photographing device (13) is controlled to perform image capture on the said object, so that when a user is at a certain distance from the handheld cradle head, the user may control the handheld cradle head by means of the specific posture to follow and perform image capture so as to capture a photo or video with a better composition, which improves the user experience.

Description

Hand-held pan/tilt control method, control device, handheld pan/tilt and storage medium Technical field

The embodiments of the present application relate to the field of handheld pan/tilt, and in particular, to a control method, control device, handheld pan/tilt, and storage medium of a handheld pan/tilt.

Background technique

In the prior art, the photographing equipment can be mounted on a handheld pan-tilt, so as to enhance the stability of the photographed image through the handheld pan-tilt.

In some scenes, the user may take a selfie by carrying a camera on the handheld gimbal. However, because the handheld gimbal may be far away from the user, the user cannot control the handheld gimbal, which prevents the user from taking better photos Or video, which reduces the user experience.

Summary of the invention

The embodiment of the application provides a control method, control device, hand-held PTZ and storage medium of a handheld PTZ. When the user is at a certain distance from the handheld PTZ, the user can control the handheld PTZ through a specific posture to follow and shoot, Take photos or videos with better composition to improve user experience.

The first aspect of the embodiments of the present application is to provide a method for controlling a handheld PTZ, the handheld PTZ is equipped with a photographing device, and the method includes:

Acquiring an image collected by the photographing device;

Detecting a specific posture in the image;

Determining the following object in the image according to the specific posture;

Control the handheld pan/tilt to follow the following object, and control the photographing device to photograph the following object during the following process.

The second aspect of the embodiments of the present application is to provide a method for controlling a handheld PTZ, the handheld PTZ is equipped with a photographing device, and the method includes:

Acquiring an image collected by the photographing device;

Detecting a specific posture in the image;

Controlling the parameters of the handheld pan/tilt and/or the photographing device according to the specific posture.

A third aspect of the embodiments of the present application is to provide a control device for a handheld pan/tilt, the handheld pan/tilt mount is equipped with photographing equipment, and the control device includes: a memory and a processor;

The memory is used to store program codes;

The processor calls the program code, and when the program code is executed, is used to perform the following operations:

Acquiring an image collected by the photographing device;

Detecting a specific posture in the image;

Determining the following object in the image according to the specific posture;

Control the handheld pan/tilt to follow the following object, and control the photographing device to photograph the following object during the following process.

A fourth aspect of the embodiments of the present application is to provide a control device for a handheld pan/tilt, the handheld pan/tilt mount is equipped with photographing equipment, and the control device includes: a memory and a processor;

The memory is used to store program codes;

The processor calls the program code, and when the program code is executed, is used to perform the following operations:

Acquiring an image collected by the photographing device;

Detecting a specific posture in the image;

Controlling the parameters of the handheld pan/tilt and/or the photographing device according to the specific posture.

The fifth aspect of the embodiments of the present application is to provide a handheld PTZ, including:

A pan/tilt, on which photographing equipment is carried;

A base, the base station is connected to the pan-tilt; and

The control device described in the third or fourth aspect above.

The sixth aspect of the embodiments of the present application is to provide a computer-readable storage medium having a computer program stored thereon, and the computer program is executed by a processor to implement the method described in the first aspect or the second aspect.

The control method, control device, handheld PTZ and storage medium provided in this embodiment detect a specific posture in an image collected by a shooting device on the handheld PTZ, and determine the image in the image according to the specific posture. Follow the object, further control the handheld PTZ to follow the follow object, and control the shooting device to shoot the follow object during the following process, so that when the user is a certain distance from the handheld PTZ, the user can control the handheld through a specific posture The pan/tilt follows and shoots in order to take a better-composed photo or video, which improves the user experience.

Description of the drawings

In order to more clearly describe the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed in the description of the embodiments. Obviously, the drawings in the following description are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative labor.

FIG. 1 is a schematic diagram of a handheld pan/tilt provided by an embodiment of the application;

2 is a flowchart of a method for controlling a handheld PTZ provided by an embodiment of the application;

FIG. 3 is a schematic diagram of an application scenario provided by an embodiment of the application;

FIG. 4 is a schematic diagram of an image provided by an embodiment of the application;

FIG. 5 is a schematic diagram of another image provided by an embodiment of the application;

Figure 6 is a schematic diagram of the head and shoulders area provided by an embodiment of the application;

FIG. 7 is a flowchart of a method for controlling a handheld PTZ according to another embodiment of the application;

FIG. 8 is a schematic diagram of an expanded area provided by an embodiment of the application;

FIG. 9 is a schematic diagram of another expanded area provided by an embodiment of the application;

FIG. 10 is a schematic diagram of another expanded area provided by an embodiment of the application;

11 is a flowchart of a method for controlling a handheld PTZ according to another embodiment of the application;

FIG. 12 is a schematic diagram of cropping and resizing an enlarged area according to an embodiment of the application;

FIG. 13 is a flowchart of cropping and resizing an enlarged area according to an embodiment of the application;

FIG. 14 is a flowchart of a method for controlling a handheld PTZ according to another embodiment of the application;

FIG. 15 is a schematic diagram of an expanded area provided by an embodiment of the application;

FIG. 16 is a flowchart of a method for controlling a handheld PTZ according to another embodiment of the application;

FIG. 17 is a schematic diagram of an expanded area provided by an embodiment of the application;

FIG. 18 is a flowchart of a method for controlling a handheld PTZ according to another embodiment of the application;

Figure 19 is a structural diagram of a handheld pan/tilt control device provided by an embodiment of the application;

Fig. 20 is a structural diagram of a handheld pan/tilt control device provided by an embodiment of the application.

Reference signs:

11: Base; 12: PTZ; 13: Camera equipment;

40: image; 50: image; 51: object;

52: object; 61: head and shoulders area; 62: head and shoulders area;

81: Expand the area; 82: Expand the area; 83: Specific one-handed gestures;

91: an enlarged area of a preset size corresponding to the head and shoulders area 61;

92: an enlarged area of a preset size corresponding to the head and shoulders area 62;

190: control device; 191: memory; 192: processor;

200: control device; 201: memory; 202: processor.

detailed description

The technical solutions in the embodiments of the present application will be clearly described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of this application.

It should be noted that when a component is said to be "fixed to" another component, it can be directly on the other component or a central component may also exist. When a component is considered to be "connected" to another component, it can be directly connected to another component or there may be a centered component at the same time.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field of this application. The terms used in the description of the application herein are only for the purpose of describing specific embodiments, and are not intended to limit the application. The term "and/or" as used herein includes any and all combinations of one or more related listed items.

Hereinafter, some embodiments of the present application will be described in detail with reference to the drawings. In the case of no conflict, the following embodiments and features in the embodiments can be combined with each other.

The embodiment of the application provides a method for controlling a handheld PTZ. The handheld PTZ is equipped with photographing equipment. Figure 1 is a schematic diagram of a handheld pan/tilt provided by an embodiment of the application. As shown in FIG. 1, the handheld pan/tilt may specifically include a base 11 and a pan/tilt 12, and the pan/tilt 12 carries a photographing device 13. Among them, the shooting device 13 may specifically be a camera, a video camera, or a smart terminal with a shooting function, such as a device such as a mobile phone. The base 11 may specifically be a handheld pole that holds a pan/tilt, and the user can hold the pole with hands to take pictures, for example, to take pictures of distant scenery or selfies. The method described in the embodiment of the present application may be suitable for the user to place the hand-held pan/tilt at a place not far away from him to take pictures, for example, self-portraits. The method will be introduced below in conjunction with specific embodiments.

Fig. 2 is a flowchart of a method for controlling a handheld pan/tilt provided by an embodiment of the application. As shown in Figure 2, the method in this embodiment may include:

S201. Acquire an image collected by the shooting device.

In this embodiment, the user can interact with the handheld pan/tilt through gestures. For example, the user may control the handheld pan/tilt through gestures to follow, take photos, and record videos. As shown in Figure 3, the user poses a specific posture, the specific posture can be a specific hand posture and/or a specific body posture, the camera on the handheld camera captures an image of the user, the image can be a photo or a video . The control device in the handheld PTZ is in communication connection with the shooting device, and the control device obtains the image collected by the shooting device through the communication connection with the shooting device.

S202: Detect a specific posture in the image.

Specifically, the control device may have an image processing function. After the control device acquires the image collected by the photographing device, it detects a specific posture in the image through image processing.

S203: Determine a follow object in the image according to the specific posture.

Further, the control device can determine the following object in the image according to the specific posture. The following object can be the user who issued the specific posture. As shown in FIG. 4, 40 represents an image captured by the shooting device, and the control device detects To the specific hand posture in the image, it is determined that the object matching the specific hand posture is the follower object, that is, the user who issued the specific hand posture.

In other embodiments, the following object may also be other objects around the user making the specific gesture. As shown in Figure 5, 50 represents an image captured by a shooting device. The control device detects a specific hand gesture in the image and determines that the object closest to the hand gesture is the follower object. As shown in Figure 5, the object 51 is the user who issued the hand gesture, and the object 52 is the closest object to the hand gesture. Therefore, the object 52 is the following object in the image.

S204. Control the handheld pan/tilt to follow the following object, and control the photographing device to photograph the following object during the following process.

In addition, the control device also has the functions of controlling the handheld PTZ and controlling the shooting equipment. Specifically, when the control device determines the following object in the image, it controls the handheld pan/tilt to follow the following object, and controls the photographing device to photograph the following object during the following process. Alternatively, in other embodiments, the control device may first control the handheld pan/tilt to follow the following object, and then control the photographing device to photograph the following object.

As a possible way, the controlling the handheld pan/tilt to follow the following object includes: controlling the handheld pan/tilt to follow the following object so that the following object is located on the camera The preset position in the shooting screen.

As shown in FIG. 4, the following object is not in the center of the frame. At this time, the control device can control the handheld pan/tilt to follow the following object so that the following object is located in the center of the shooting frame of the shooting device. It is understandable that positioning the following object in the center of the shooting frame is only a follow-up effect. In other embodiments, the handheld pan/tilt can also be controlled to follow the following object so that the following object is located in other positions of the shooting frame. .

As another possible manner, the controlling the handheld pan/tilt to follow the following object includes: controlling the handheld pan/tilt to follow the following object in a direction with a preset rotation angle.

As shown in Figure 3, the control device can control the yaw angle of the handheld gimbal so that the handheld gimbal rotates in the yaw direction, and follows the following object at a preset rotation angle, for example, the following object Perform 180-degree rotating follow or 360-degree rotating follow. At this time, the following object may be at the preset position of the shooting screen or not at the preset position of the shooting screen.

As a possible implementation, the controlling the photographing device to photograph the following object during the following process includes: controlling the photographing device to focus on the position of the following object during the following process. The following object is photographed.

During the process of the handheld pan/tilt following the following object, the control device controls the photographing device to take the following object with the position of the following object as the center, for example, single-shot photographing, video recording or panoramic shooting. For example, in the process of the handheld pan-tilt to follow the following object, the control device controls the shooting device to take the position of the following object as the panoramic center to complete 180-degree or 3*3 panoramic shooting.

In other embodiments, the method further includes: according to the specific posture, controlling the photographing device to photograph the following object after a preset period of time.

For example, the user's specific gestures can also control the shooting device to delay shooting the following objects. For example, when the control device detects a specific posture in the image, after a preset period of time, for example, 3 seconds later, the following object is photographed as described above, for example, a single photograph, a video or a panoramic photograph.

In this embodiment, by detecting a specific posture in an image collected by a photographing device on a handheld pan/tilt, and determining a follower object in the image according to the specific posture, the handheld gimbal is further controlled to follow the follower object, and is following During the process, the camera is controlled to shoot the following object, so that when the user is at a certain distance from the handheld pan/tilt, the user can control the handheld pan/tilt through a specific posture to follow and shoot, so as to take a better composition photo or video, and improve Improve the user experience.

The embodiment of the application provides a method for controlling a handheld PTZ. On the basis of the foregoing embodiment, as a possible way, the determining a follower object in the image according to the specific posture includes: detecting at least one specific body region in the image; determining the at least one An object in the specific body area that matches the target object issuing the specific posture in the specific body area is the follower object.

As shown in Fig. 6, 50 represents an image collected by a photographing device. When the control device of the handheld pan/tilt head acquires the image, at least one specific body area in the image is detected. The specific body area may specifically be a head and shoulder area. The head and shoulders area 61 and the head and shoulders area 62 are shown in FIG. 6. Wherein, the head and shoulders area 61 includes the object 51 but does not include the specific posture, and the head and shoulders area 62 includes the object 52 and the specific posture. Further, it is detected which of the object 51 and the object 52 the specific posture matches, for example, the detection result If the specific posture matches the object 51, the object 51 is the target object that issued the specific posture, and the head and shoulders area matching the target object is the head and shoulders area 61. Therefore, the object in the head and shoulders area 61 can be used as the follow object. In other words, the target object issuing the specific gesture is the following object.

As another possible manner, the determining the follower object in the image according to the specific posture includes: detecting at least one specific body region in the image; The object in the specific body region where the distance of the target object in the specific posture meets the first preset condition is the follow object.

As shown in Fig. 6, 50 represents an image collected by a photographing device. When the control device of the handheld pan/tilt head acquires the image, at least one specific body area in the image is detected. The specific body area may specifically be a head and shoulder area. The head and shoulders area 61 and the head and shoulders area 62 are shown in FIG. 6. Wherein, the head and shoulders area 61 includes the object 51 but does not include the specific posture, and the head and shoulders area 62 includes the object 52 and the specific posture. Further, it is detected which of the object 51 and the object 52 the specific posture matches, for example, the detection result If the specific posture matches the object 51, then the object 51 is the target object issuing the specific posture. Further, it is determined in the image that the head and shoulders area meeting the first preset condition from the target object is determined. The first preset condition may be the head and shoulders area closest to the target object. As shown in FIG. 6, the head and shoulders area 62 is The head and shoulders area closest to the target object, at this time, the object 52 in the head and shoulders area 62 can be used as the follow object.

It can be understood that the specific body area in this embodiment is not limited to the head and shoulders area. For example, it may also be the upper body area, the hand area, the entire body area, and the like.

In this embodiment, by detecting at least one specific body area in the image, it is determined that the object in the specific body area matching the target object issuing the specific posture in the at least one specific body area is a follower object, or the The object in the specific body area whose distance from the target object issuing the specific posture in at least one specific body area satisfies the first preset condition is the follower object, which improves the flexibility of detecting the follower object.

The embodiment of the application provides a method for controlling a handheld PTZ. FIG. 7 is a flowchart of a method for controlling a handheld pan/tilt head provided by another embodiment of the application. On the basis of the above-mentioned embodiment, the method further includes: determining a target object that emits the specific posture in the image; wherein the specific posture issued by the target object is used to instruct the handheld platform to follow the The object executes the following control instruction for following, and/or is used to instruct the shooting device to execute the shooting control instruction for the following object.

As shown in FIG. 6, when the control device acquires the image 50 collected by the shooting device, the control device also needs to determine the target object in the image 50 that emits a specific posture. As shown in FIG. 6, the object 51 is a target object that issues a specific posture, and the specific posture issued by the target object can control the handheld pan-tilt to execute the following control instructions corresponding to the following objects, and/or the The specific posture can also control the shooting device to execute the shooting control instructions corresponding to the above-mentioned shooting on the following object. It can be seen from the above embodiment that the target object issuing the specific posture may be a follower object, or it may not be a follower object.

Specifically, the determining the target object in the image that emits the specific posture includes the following steps as shown in FIG. 7:

S701. Detect at least one specific body area in the image.

As shown in FIG. 8, when the control device acquires the image 50 collected by the photographing device, it detects at least one specific body area in the image 50, for example, the head and shoulders area 61 and the head and shoulders area 62.

S702. Determine an enlarged area of each of the at least one specific body area in the image.

Further, determine the enlarged areas of the head and shoulders area 61 and the head and shoulders area 62 in the image, for example, multiply the head and shoulders area by a fixed scale factor to obtain the enlarged area of the head and shoulders area, as shown in Figure 8, 81 It represents the enlarged area of the head and shoulders area 61 in the image, and 82 represents the enlarged area of the head and shoulders area 62 in the image. In this embodiment, the reason why it is necessary to determine the enlarged area of the head and shoulders area in the image is to ensure that the enlarged area includes the hand.

S703. Determine a target object issuing the specific posture according to the enlarged area of each specific body region in the image.

For example, according to the enlarged area 81 and the enlarged area 82, the target object that emits a specific posture in the image can be determined.

Optionally, the determining the target object issuing the specific posture according to the enlarged area of each of the specific body regions in the image includes: detecting the enlargement of each of the specific body regions in the image Whether the specific posture exists in the area; if the specific posture exists in the enlarged area of the specific body area in the image, it is determined that the object in the specific body area is the target object.

For example, it is detected whether a specific posture is included in the enlarged area 81 and the enlarged area 82 respectively. If one of the enlarged area 81 and the enlarged area 82 includes a specific posture, for example, the enlarged area 81 includes a specific posture, it is determined that the head and shoulders area 61 is The object 51 of is the target object issuing the specific gesture.

In a possible situation, the specific posture includes a specific one-handed gesture, and the position of the specific one-handed gesture is arbitrary; correspondingly, if the specific body area is in the image If the specific posture exists in the enlarged area, determining that the object in the specific body area is the target object includes: if the specific body area has the specific one-hand gesture in the enlarged area in the image , And the specific one-hand gesture matches the body in the specific body area, then it is determined that the object in the specific body area is the target object.

As shown in FIG. 8, the specific posture includes a specific one-hand gesture 83, and the specific one-hand gesture 83 may be any one-hand gesture, for example, not limited to palm, scissor hand, one-handed comparison, and fist. In addition, the position of the one-handed gesture can be arbitrary. Therefore, after determining that the enlarged area 81 includes the specific one-handed gesture 83, it is necessary to further determine whether the specific one-handed gesture 83 corresponds to the head and shoulders area 61. If the specific one-hand gesture 83 matches the body in the head-and-shoulders area 61, the object 51 in the head-and-shoulders area 61 is determined to be the target object of the one-hand gesture. Optionally, the target object is the follow object.

In addition, if the specific one-handed gesture exists in the enlarged area of the image in the specific body area, and the specific one-handed gesture does not match the body in the specific body area, it is determined that the The object in the specific body area is the follower object, and the object in the specific body area corresponding to the body matching the specific one-hand gesture is determined to be the target object.

As shown in FIG. 9, if the enlarged area 82 includes a specific one-handed gesture, and the specific one-handed gesture does not match the body in the head and shoulders area 62, it is further determined that the body that matches the specific one-handed gesture is The body in the head and shoulders area 61, therefore, it can be determined that the object 52 in the head and shoulders area 62 is the follower object, and the object 51 in the head and shoulders area 61 is the target object for issuing the specific one-handed gesture.

In addition, the method further includes: if a plurality of the specific body regions have the specific posture in the enlarged area in the image, determining a specific body that satisfies a second preset condition among the plurality of specific body regions The object in the area is the target object.

As shown in FIG. 10, the enlarged area 81 and the enlarged area 82 respectively include specific postures. For example, for a specific one-hand gesture, it is necessary to determine one of the head and shoulders area 61 and the head and shoulders area 62 that meets the second preset condition. The head and shoulders area, and the object in the head and shoulders area meeting the second preset condition is taken as the target object.

Optionally, the second preset condition includes that the distance from the center of the image is less than a first preset threshold and/or the proportion of frames in the image is greater than a second preset threshold; wherein, the frame The proportion is determined according to the size of the specific body area or the enlarged area of the specific body area in the image.

For example, the distance between the head and shoulders area 61 and the center of the image and the distance between the head and shoulders area 62 and the center of the image are calculated separately. The distance between the head and shoulders area 61 and the center of the image may specifically be the center of the head and shoulders area 61 and the image. The distance between the center of the head and shoulders area 62 and the center of the image may specifically be the distance between the center of the head and shoulders area 62 and the center of the image. And/or, determine the proportion of the head and shoulders area 61 in the image according to the size of the head and shoulders area 61 or the enlarged area 81, and determine the proportion of the head and shoulders area 62 in the image according to the size of the head and shoulders area 62 or the enlarged area 82 ratio. For example, it is determined from the head and shoulders area 61 and the head and shoulders area 62 that the object in the head and shoulders area whose distance from the center of the image is less than the first preset threshold and/or the frame ratio in the image is greater than the second preset threshold is taken as target. For example, if the distance between the head and shoulders area 61 and the center of the image is less than the first preset threshold, and the proportion of the head and shoulders area 61 in the image is greater than the second preset threshold, it is determined that the object in the head and shoulders area 61 is the target object .

In this embodiment, by detecting at least one specific body area in the image, the enlarged area of each specific body area in the at least one specific body area is determined in the image to ensure that the enlarged area includes the hand. Further, according to each The enlarged area of the specific body area in the image determines the target object that issued the specific posture, which improves the detection accuracy of the specific posture. In addition, when the specific posture includes a specific one-hand gesture, and the position of the specific one-hand gesture is arbitrary, based on the presence of the specific one-hand gesture in the enlarged area corresponding to the specific body area, further Matching the specific one-hand gesture with the body in the specific body area ensures that the specific one-hand gesture is made by an object in the specific body area, which further improves the detection accuracy of the specific one-hand gesture.

The embodiment of the application provides a method for controlling a handheld PTZ. FIG. 11 is a flowchart of a method for controlling a handheld pan/tilt head according to another embodiment of the application. As shown in FIG. 11, on the basis of the foregoing embodiment, the detecting whether each of the specific body regions has the specific posture in the enlarged area in the image includes:

S1101: Adjust the size of the enlarged area of each specific body area in the image to a preset size to obtain the enlarged area of the preset size corresponding to each specific body area, and the preset size The size of the input image that meets the detection model.

As shown in FIG. 12, the control device determines the enlarged area 81 of the head and shoulder area 61 in the image and the enlarged area 82 of the head and shoulder area 62 in the image, and adjusts the enlarged area 81 and the enlarged area 82 to preset Size to obtain an enlarged area 81 of a preset size and an enlarged area 82 of a preset size. The preset size may specifically be the size of the input image of the detection model. That is, the size of the input image of the detection model can be fixed, and the detection model can be used to detect a specific posture, for example, a specific hand posture and/or a specific body posture. Among them, the detection module can be a neural network model, for example.

Optionally, the adjusting the size of the enlarged area of each specific body area in the image to a preset size includes: setting each specific body area in the image in the image Performing cropping processing on the expanded area of each of the expanded areas to obtain a cropped area corresponding to each of the expanded areas; adjusting the size of the cropped area corresponding to each of the expanded areas to the preset size.

Taking the expanded area 81 as an example, as shown in FIG. 12, the expanded area 81 is cropped in the image to obtain the cropped area corresponding to the expanded area 81, and further, the size of the cropped area corresponding to the expanded area 81 is adjusted to The preset size, 91 represents the enlarged area of the preset size corresponding to the head and shoulders area 61. In the same way, the size of the cropping area corresponding to the enlarged area 82 is adjusted to a preset size, and 92 represents the enlarged area of the preset size corresponding to the head and shoulders area 62.

Optionally, the size of the cropping area corresponding to the enlarged area is the same as the size of the enlarged area. As shown in FIG. 12, since the cropped area corresponding to the enlarged area 81 is in the image, the size of the cropped area corresponding to the enlarged area 81 is the same as the size of the enlarged area 81.

Optionally, a part of the cropped area corresponding to the enlarged area is outside the image. As shown in FIG. 12, since the head and shoulders area 62 is near the edge of the image, when determining the expanded area of the head and shoulders area 62 in the image, the expanded area 82 corresponding to the head and shoulders area 62 may exceed the range of the image. In other words, it is necessary to fill a part of pixels on the upper and right sides of the head and shoulders area 62 to form the enlarged area 82 corresponding to the head and shoulders area 62. Among them, the filled part of the pixels is shown as the shaded part in Figure 10. Therefore, a part of the cropped area corresponding to the enlarged area 82, for example, a part of the filled pixels is outside the image.

S1102: Input the expansion area of the preset size corresponding to each specific body area into the detection model, and detect the expansion of the preset size corresponding to each specific body area through the detection model Whether the specific posture exists in the area.

For example, the expanded area 91 of the preset size corresponding to the head and shoulders area 61 and the expanded area 92 of the preset size corresponding to the head and shoulders area 62 are respectively input into the detection model, and the preset corresponding to the head and shoulders area 61 is detected by the detection model. Whether there is a specific posture in the enlarged area 91 of a large size, and whether there is a specific posture in the enlarged area 92 of a preset size corresponding to the head and shoulders area 62 is detected. When the specific posture is a specific hand posture, the detection model may specifically be a gesture detection model. The specific hand posture may be a specific one-hand posture or a specific two-hand posture. For example, the specific two-hand gesture may be a two-hands frame (for example, a quadrilateral frame on the chest). When the specific posture is a specific body posture, the detection model may specifically be a body posture detection model, where the specific body posture may be, for example, a forearm posture when the hands are in front of the face compared to a frame (such as a quadrilateral frame).

As shown in Figure 13, an image is acquired from a shooting device, and the image is subjected to head-and-shoulders detection to obtain at least one head-and-shoulders area in the image, and determine the enlarged area of each head-and-shoulder area in the image. Further, for each enlarged area Perform cropping processing, adjust the size of the cropped area to a preset size, and input the expanded area of the preset size into the detection model.

In this embodiment, by adjusting the size of the enlarged area of each specific body area in the image to a preset size, the enlarged area of the preset size corresponding to each specific body area is obtained, and each The enlarged area of the preset size corresponding to the specific body area is input into the detection model, and the detection model is used to detect whether there is any expansion area of the preset size corresponding to each specific body area. The specific posture is described to improve the detection accuracy of the specific posture.

The embodiment of the application provides a method for controlling a handheld PTZ. FIG. 14 is a flowchart of a method for controlling a handheld PTZ according to another embodiment of the application. As shown in FIG. 14, on the basis of the above embodiment, the specific posture includes a specific hand posture, and the specific body area includes a head and shoulder area. As shown in FIG. 15, the image 50 includes a specific hand posture. , The hand position is between the shoulders. The determining the enlarged area of each of the at least one specific body area in the image includes:

S1401, according to the width of the shoulder in each of the at least one specific body area, determine the width of the enlarged area of each specific body area in the image.

As shown in Figure 15, the image 50 includes a head and shoulders area 61. When determining the enlarged area of the head and shoulders area 61 in the image, the shoulder width in the head and shoulders area 61 can be determined first, for example, the shoulders The width is denoted as w, and further, according to the shoulder width w, the width of the enlarged area of the head and shoulder area 61 in the image is determined.

Optionally, the determining the width of the enlarged area of each specific body region in the image according to the shoulder width in each of the at least one specific body region includes: Determine the width of the enlarged area of each specific body area in the image according to the width of the shoulder in each specific body area in the at least one specific body area and the expansion multiple of the width of the shoulder .

For example, the expansion factor of the shoulder width w is 3 times. Therefore, according to the expansion factor of the shoulder width w and the shoulder width w, the width of the enlarged area of the head and shoulder area 61 in the image is determined, and the width of the enlarged area Denoted as W. For example, the width of the enlarged area may be 3 times the width of the head and shoulders area 61.

S1402, according to the width of the enlarged area of each specific body area in the image, determine the height of the enlarged area of each specific body area in the image.

Further, according to the width W of the enlarged area of the head and shoulders area 61 in the image, the height H of the enlarged area of the head and shoulders area 61 in the image is determined.

The determining the height of the expanded area of each specific body area in the image according to the width of the expanded area of each specific body area in the image includes: according to each specific body area The width of the enlarged area in the image and the ratio between the width and the height of the enlarged area of each specific body area in the image determine how much each specific body area is in the image Increase the height of the area.

For example, based on the ratio between the width W of the enlarged area of the head and shoulders area 61 in the image and the ratio between the width W and the height H of the enlarged area of the head and shoulders area 61 in the image, it is determined that the head and shoulders area 61 is in the image The height of the enlarged area H.

S1403. Determine an enlarged area of each specific body area in the image according to the width and the height.

Based on the width W and the height H, the enlarged area of the head and shoulders area 61 in the image is determined, such as the enlarged area 81 shown in FIG. 15. In addition, in other embodiments, in the process of determining the enlarged area of the head and shoulders area 61 in the image, the position of the head and shoulders center point in the head and shoulders area 61 and the enlarged area may also be kept unchanged.

In this embodiment, the width of the enlarged area of each specific body area in the image is determined according to the width of the shoulder in each specific body area in the at least one specific body area, and the width of each specific body area in the image is determined according to the Determine the width of the enlarged area in the image, determine the height of the enlarged area of each specific body area in the image, and determine the enlarged area of each specific body area in the image according to the width and the height, and realize the definite expansion A way of region.

The embodiment of the application provides a method for controlling a handheld PTZ. FIG. 16 is a flowchart of a method for controlling a handheld PTZ according to another embodiment of the application. As shown in FIG. 16, based on the foregoing embodiment, the specific posture includes a specific body posture, and the specific body area includes a head and shoulder area. As shown in FIG. 17, the image 50 includes a specific body posture.

The determining the enlarged area of each of the at least one specific body area in the image includes:

S1601, according to the height of the head and shoulders in each of the at least one specific body area, determine the height of the enlarged area of each specific body area in the image.

As shown in FIG. 17, the image 50 includes a head and shoulders area 61. When determining the enlarged area of the head and shoulders area 61 in the image, the head and shoulders height in the head and shoulders area 61 can be determined first, for example, head and shoulders. The height is recorded as h. Further, according to the head and shoulder height h, the height of the enlarged area of the head and shoulder area 61 in the image is determined.

Optionally, the determining the height of the enlarged area of each specific body region in the image according to the height of the head and shoulders in each of the at least one specific body region includes: Determine the height of the enlarged area of each specific body area in the image according to the height of the head and shoulders in each specific body area in the at least one specific body area and the magnification factor of the head and shoulders height .

For example, the magnification factor of the head and shoulder height h is 4 times. Therefore, according to the magnification factor of the head and shoulder height h and the head and shoulder height h, the height of the enlarged area of the head and shoulder area 61 in the image is determined, and the height of the enlarged area is recorded For H. For example, the height of the enlarged area may be 4 times the height of the head and shoulders area 61.

S1602, according to the height of the enlarged area of each specific body area in the image, determine the width of the enlarged area of each specific body area in the image.

Further, according to the height H of the enlarged area of the head and shoulders area 61 in the image, the width W of the enlarged area of the head and shoulders area 61 in the image is determined.

Optionally, the determining the width of the expanded area of each specific body area in the image according to the height of the expanded area of each specific body area in the image includes: The height of the enlarged area of the specific body area in the image, and the ratio between the height and the width of the enlarged area of each specific body area in the image, determine whether each specific body area is in the The width of the enlarged area in the image.

For example, according to the height H of the enlarged area of the head and shoulders area 61 in the image, and the ratio between the height H and the width W of the enlarged area of the head and shoulders area 61 in the image, it is determined that the head and shoulders area 61 is in the image. The width W of the enlarged area in.

S1603. Determine an enlarged area of each specific body area in the image according to the width and the height.

According to the width W and the height H, the enlarged area of the head and shoulders area 61 in the image is determined, as shown in the enlarged area 81 in FIG. 17. Optionally, the enlarged area 81 as shown in FIG. 17 may include limbs of the target object. The expanded area 81 as shown in FIG. 15 may include the palm of the target object. That is, the enlarged area required to detect a specific body posture may be larger than the enlarged area required to detect a specific hand posture.

In this embodiment, the height of the enlarged area of each specific body area in the image is determined according to the height of the head and shoulders in each of the at least one specific body area, and the height of each specific body area in the image The height of the enlarged area in the image determines the width of the enlarged area of each specific body area in the image, and determines the enlarged area of each specific body area in the image according to the width and the height to realize the definite expansion Another way of regionalization improves the flexibility of determining the expansion region.

The embodiment of the application provides a method for controlling a handheld PTZ. The handheld PTZ is equipped with photographing equipment. FIG. 18 is a flowchart of a method for controlling a handheld PTZ according to another embodiment of the application. As shown in FIG. 18, the control method of the handheld PTZ includes:

S1801: Acquire an image collected by the photographing device.

The implementation mode and specific principles of S1801 are consistent with the above-mentioned S201, and will not be repeated here.

S1802. Detect a specific posture in the image.

The implementation manner and specific principle of detecting the specific posture in the image are specifically as described in the foregoing embodiment, and will not be repeated here.

S1803. Control the parameters of the handheld PTZ and/or the photographing device according to the specific posture.

In this embodiment, when the control device in the handheld pan/tilt head determines a specific posture in the image, it can also control the parameters of the handheld pan/tilt head and/or the shooting device according to the specific posture.

Optionally, the parameters of the handheld PTZ include at least one of the following: the posture, rotation speed, and rotation direction of the handheld PTZ.

Optionally, the parameters of the shooting device include at least one of the following: a shutter parameter of the shooting device, a lens focal length of the shooting device, and a follow focus parameter of the shooting device.

It can be understood that in addition to the above description, the parameters of the handheld pan/tilt head or the parameters of the photographing device may also include other parameters in practical applications. For example, for the photographing device, the parameters may also include white balance parameters, which are not specifically limited here.

This embodiment detects a specific posture in the image collected by the camera on the handheld camera, and further controls the parameters of the camera and/or the camera according to the specific posture, so that the user is at a certain distance from the camera. The user does not need to take back the handheld pan/tilt for parameter setting. After the setting is complete, place the handheld pan/tilt at a certain distance from the user. The user only needs to control the handheld pan/tilt and/or the shooting through a specific posture. The parameters of the device improve the user experience.

The embodiment of the application provides a method for controlling a handheld PTZ. The handheld PTZ is equipped with photographing equipment. On the basis of the foregoing embodiment, as a possible implementation, the detecting the specific posture in the image includes: detecting the specific posture in the image according to the overall area of the image.

When the target object takes up a relatively large frame in the image, the specific posture issued by the target object will also occupy a relatively large frame in the image. At this time, the specific posture in the image can be detected according to the overall area of the image. posture.

Optionally, the detecting a specific posture in the image according to the overall area of the image includes: when the distance of the shooting device relative to the shooting object is less than or equal to a preset distance, according to the overall image Area, to detect a specific posture in the image.

For example, when the distance of the shooting device relative to the shooting object is less than or equal to the preset distance, it means that the shooting device is closer to the shooting object. At this time, the shooting object occupies a larger proportion in the shooting screen. To detect a specific posture in the image, the shooting object may specifically be the target object in the foregoing embodiment.

Optionally, the detecting a specific posture in the image according to the overall area of the image includes: when the photographing device uses a front camera to collect the image, detecting the entire area according to the image Describe the specific posture in the image.

When the shooting device is a smart phone, if the front camera is currently turned on by the smart phone, and the user himself makes a specific gesture, it means that the user himself is the target object and the user is taking a selfie. At this time, the user accounts for a relatively large portion of the image collected by the front camera. Therefore, the specific posture in the image can be detected according to the overall area of the image.

As another possible implementation manner, the detecting the specific posture in the image includes: detecting the specific posture in the image according to a partial area of the image.

When the target object has a relatively small frame in the image, the specific posture issued by the target object will also occupy a relatively small frame in the image. At this time, it is necessary to extract some areas of the image, such as the head and shoulders area, Hand area, upper body area, entire body area, etc. Further, the partial area is subjected to the above-mentioned cropping process, resized to a preset size, and then input to the detection model, and the detection model determines whether there is a specific posture in the partial area of the preset size.

Optionally, the detecting a specific posture in the image according to a partial area of the image includes: when the distance of the photographing device relative to the shooting object is greater than a preset distance, according to the partial area of the image, Detect specific gestures in the image.

For example, when the distance between the shooting device and the shooting object is greater than the preset distance, it means that the shooting device is far away from the shooting object. At this time, the shooting object occupies a relatively small proportion in the shooting frame. In order to accurately detect the image For the specific posture in the image, the specific posture in the image needs to be detected according to a partial area of the image, and the shooting object may specifically be the target object in the foregoing embodiment.

Optionally, the detecting a specific posture in the image according to a partial area of the image includes: when the photographing device uses a rear camera to capture the image, detecting all positions according to the partial area of the image Describe the specific posture in the image.

When the shooting device is a smart phone, if the smart phone is currently turning on the rear camera, it means that the smart phone is shooting a distant scene. At this time, the target object accounts for a relatively small proportion of the images collected by the rear camera. Therefore, it is necessary to detect the specific posture in the image according to the partial area of the image.

Optionally, the partial area of the image includes at least one specific body area in the image; the detecting the specific posture in the image according to the partial area of the image includes: determining the at least one specific body An enlarged area of each of the specific body regions in the image; and detecting a specific posture in the image according to the enlarged area of each of the specific body regions in the image.

For example, the partial area of the image includes at least one specific body area in the image, and the specific body area may specifically be the head and shoulders area, the hand area, the upper body area, the entire body area, etc. as described above. Take the head and shoulders area as an example. At this time, according to at least one head and shoulders area in the image, when detecting a specific posture in the image, it can be specifically determined that each head and shoulders area in the at least one head and shoulders area is in the image. The process and method of determining the enlarged area in the above embodiments are as described in the above embodiment, and will not be repeated here. Further, according to the enlarged area of each head and shoulder area in the image, a specific posture in the image is detected.

Optionally, the detecting the specific posture in the image according to the enlarged area of each of the specific body regions in the image includes: arranging the enlarged area of each specific body region in the image Adjust the size of the to a preset size to obtain an enlarged area of the preset size corresponding to each of the specific body regions, where the preset size meets the size of the input image of the detection model; and each specific body region corresponds to The expanded area of the preset size is input into the detection model, and the specific posture in the expanded area of the preset size corresponding to each specific body area is detected by the detection model.

For example, according to the enlarged area of each head and shoulder area in the image, when detecting a specific posture in the image, the enlarged area of each head and shoulder area in the image may be adjusted to a preset size. In order to detect the size that the input image of the model needs to meet, further, input each expansion area of a preset size into the detection model, and detect whether there is a specific posture in the expansion area of the preset size corresponding to each head and shoulder area through the detection model.

Optionally, the adjusting the size of the enlarged area of each specific body area in the image to a preset size includes: setting each specific body area in the image in the image Performing cropping processing on the expanded area of each of the expanded areas to obtain a cropped area corresponding to each of the expanded areas; adjusting the size of the cropped area corresponding to each of the expanded areas to the preset size.

For example, when the expanded area of each head and shoulders area in the image is adjusted to a preset size, the expanded area of each head and shoulders area in the image can be cropped out of the image, and further, the cropped The size of the region is adjusted to a preset size, and the specific cropping process and size adjustment process are as described in the foregoing embodiment, and will not be repeated here.

In this embodiment, when the target object occupies a relatively large frame in the image, the specific posture in the image is detected according to the overall area of the image; when the target object occupies a relatively small frame in the image, according to the partial area of the image Detecting the specific posture in the image realizes that in different scenes, the image area adapted to the scene is used to detect the specific posture in the image, thereby improving the flexibility of detecting the specific posture.

The embodiment of the present application provides a control device for a handheld PTZ. The handheld PTZ is equipped with photographing equipment. FIG. 19 is a structural diagram of a control device for a handheld pan/tilt head provided by an embodiment of the application. As shown in FIG. 19, the control device 190 includes: a memory 191 and a processor 192; the memory 191 is used to store program codes; the processor 192, The program code is called, and when the program code is executed, it is used to perform the following operations: acquire the image collected by the photographing device; detect a specific posture in the image; determine the following in the image according to the specific posture Object; controlling the handheld pan/tilt to follow the following object, and controlling the photographing device to photograph the following object during the following process.

Optionally, when the processor 192 determines the follower object in the image according to the specific posture, it is specifically configured to: detect at least one specific body region in the image; determine whether the target is in the at least one specific body region. The object in the specific body region matched by the target object in the specific posture is the following object.

Optionally, when the processor 192 determines the follower object in the image according to the specific posture, it is specifically configured to: detect at least one specific body region in the image; determine whether the target is in the at least one specific body region. The object in the specific body region where the distance of the target object in the specific posture meets the first preset condition is the follow object.

Optionally, when the processor 192 controls the handheld PTZ to follow the following object, it is specifically configured to: control the handheld PTZ to follow the follow object so that the follow object is located on the photographing device The preset position in the shooting screen. Optionally, when the processor 192 controls the handheld pan/tilt to follow the following object, it is specifically configured to: control the handheld pan/tilt to follow the following object in a direction with a preset rotation angle.

Optionally, when the processor 192 controls the photographing device to photograph the following object during the following process, it is specifically configured to: control the photographing device to focus on the position of the following object during the following process. The following object is photographed.

Optionally, the processor 192 is further configured to: according to the specific posture, control the photographing device to photograph the following object after a preset period of time.

Optionally, the processor 192 is further configured to: determine a target object in the image that emits the specific posture; wherein the specific posture issued by the target object is used to instruct the handheld pan/tilt to perform the following object The following control instruction for following, and/or, is used to instruct the photographing device to execute the photographing control instruction for photographing the following object.

Optionally, when the processor 192 determines the target object emitting the specific posture in the image, it is specifically configured to: detect at least one specific body region in the image; determine each of the at least one specific body region An enlarged area of the specific body area in the image; and a target object issuing the specific posture is determined according to the enlarged area of each specific body area in the image.

Optionally, when the processor 192 determines the target object issuing the specific posture according to the enlarged area of each specific body area in the image, it is specifically configured to: detect that each specific body area is in the Whether the specific posture exists in the enlarged area in the image; if the specific body area has the specific posture in the enlarged area in the image, determining that the object in the specific body area is the target object.

Optionally, the specific posture includes a specific one-handed gesture, and the position of the specific one-handed gesture is arbitrary; if the specific body area exists in the enlarged area in the image, the specific single-handed gesture A hand gesture, and the specific one-hand gesture matches the body in the specific body area, the processor 192 determines that the object in the specific body area is the target object.

Optionally, the target object is the follow object.

Optionally, the processor 192 is further configured to: if the specific one-handed gesture exists in the enlarged area of the image, and the specific one-handed gesture and the specific body area If the body does not match, it is determined that the object in the specific body area is the follow object, and the object in the specific body area corresponding to the body matched by the specific one-hand gesture is determined as the target object.

Optionally, the processor 192 is further configured to: if the specific posture exists in the enlarged area of the plurality of specific body regions in the image, determine that a second preset condition is satisfied among the plurality of specific body regions The object in the specific body area of is the target object.

Optionally, the second preset condition includes that the distance from the center of the image is less than a first preset threshold and/or the proportion of frames in the image is greater than a second preset threshold; wherein, the frame The proportion is determined according to the size of the specific body area or the enlarged area of the specific body area in the image.

Optionally, when the processor 192 detects whether each specific body area has the specific posture in the enlarged area in the image, it is specifically configured to: place each specific body area in the image. The size of the expanded area is adjusted to a preset size to obtain the expanded area of the preset size corresponding to each specific body area, and the preset size meets the size of the input image of the detection model; The expanded area of the preset size corresponding to the area is input into the detection model, and the detection model is used to detect whether the specific posture exists in the expanded area of the preset size corresponding to each specific body area.

Optionally, when the processor 192 adjusts the size of the enlarged area of each specific body area in the image to a preset size, it is specifically configured to: The expanded area in the image is cropped to obtain a cropped area corresponding to each expanded area; the size of the cropped area corresponding to each expanded area is adjusted to the preset size.

Optionally, the size of the cropping area corresponding to the enlarged area is the same as the size of the enlarged area.

Optionally, a part of the cropped area corresponding to the enlarged area is outside the image.

Optionally, the specific posture includes a specific hand posture, the specific body area includes a head and shoulders area, and the processor 192 determines that each of the at least one specific body area is in the image When the enlarged area of the at least one specific body area is specifically used to: determine the width of the enlarged area of each specific body area in the image according to the width of the shoulder in each of the at least one specific body area According to the width of the enlarged area of each specific body area in the image, determine the height of the enlarged area of each specific body area in the image; determine each according to the width and the height The enlarged area of the specific body area in the image.

Optionally, when the processor 192 determines the width of the enlarged area of each specific body area in the image according to the shoulder width in each of the at least one specific body area, It is specifically used to determine the position of each specific body area in the image according to the width of the shoulder in each specific body area in the at least one specific body area and the expansion factor of the shoulder width. Expand the width of the area.

Optionally, when the processor 192 determines the height of the expanded area of each specific body area in the image according to the width of the expanded area of each specific body area in the image, it is specifically configured to: According to the width of the enlarged area of each specific body area in the image, and the ratio between the width and height of the enlarged area of each specific body area in the image, each specific The height of the enlarged area of the body area in the image.

Optionally, the specific posture includes a specific body posture, the specific body area includes a head and shoulders area, and the processor 192 determines that each of the at least one specific body area is in the image When the area is enlarged, it is specifically used to: determine the height of the enlarged area of each specific body area in the image according to the height of the head and shoulders in each of the at least one specific body area; According to the height of the enlarged area of each specific body area in the image, determine the width of the enlarged area of each specific body area in the image; determine each specific body area according to the width and height The enlarged area of the specific body area in the image.

Optionally, when the processor 192 determines the height of the enlarged area of each specific body area in the image according to the height of the head and shoulders in each of the at least one specific body area, It is specifically used to determine the height of each specific body area in the image according to the head and shoulder height of each specific body area in the at least one specific body area and the magnification factor of the head and shoulder height Increase the height of the area.

Optionally, when the processor 192 determines the width of the expanded area of each specific body area in the image according to the height of the expanded area of each specific body area in the image, it is specifically configured to: According to the height of the enlarged area of each specific body area in the image and the ratio between the height and the width of the enlarged area of each specific body area in the image, each specific body area is determined The width of the enlarged area of the body area in the image.

The specific principles and implementation manners of the control device provided in the embodiments of the present application are similar to the foregoing embodiments, and will not be repeated here.

The embodiment of the present application provides a control device for a handheld PTZ. The handheld PTZ is equipped with photographing equipment. FIG. 20 is a structural diagram of a control device for a handheld platform provided by an embodiment of the application. As shown in FIG. 20, the control device 200 includes: a memory 201 and a processor 202; the memory 201 is used to store program codes; and the processor 202, The program code is called, and when the program code is executed, it is used to perform the following operations: acquire the image collected by the photographing device; detect a specific posture in the image; control the handheld PTZ and the handheld platform according to the specific posture /Or the parameters of the shooting device.

Optionally, the parameters of the handheld PTZ include at least one of the following: the posture, rotation speed, and rotation direction of the handheld PTZ.

Optionally, the parameters of the shooting device include at least one of the following: a shutter parameter of the shooting device, a lens focal length of the shooting device, and a follow focus parameter of the shooting device.

Optionally, when the processor 202 detects the specific posture in the image, it is specifically configured to: detect the specific posture in the image according to the overall area of the image.

Optionally, when the processor 202 detects a specific posture in the image according to the overall area of the image, it is specifically configured to: when the distance of the shooting device relative to the shooting object is less than or equal to a preset distance, The entire area of the image is used to detect a specific posture in the image.

Optionally, when the processor 202 detects a specific posture in the image according to the overall area of the image, it is specifically configured to: when the photographing device uses a front camera to collect the image, according to the overall image Area, to detect a specific posture in the image.

Optionally, when the processor 202 detects the specific posture in the image, it is specifically configured to: detect the specific posture in the image according to a partial area of the image.

Optionally, when the processor 202 detects a specific posture in the image according to a partial area of the image, it is specifically configured to: when the distance between the shooting device and the shooting object is greater than a preset distance, according to the image Part of the area to detect a specific posture in the image.

Optionally, when the processor 202 detects a specific posture in the image according to a partial area of the image, it is specifically configured to: when the shooting device uses a rear camera to capture the image, according to the partial area of the image Area, to detect a specific posture in the image.

Optionally, the partial area of the image includes at least one specific body area in the image; when the processor 202 detects the specific posture in the image according to the partial area of the image, it is specifically configured to: determine the An enlarged area of each of the specific body areas in the image in at least one specific body area; and detecting a specific posture in the image according to the enlarged area of each of the specific body areas in the image.

Optionally, when the processor 202 detects a specific posture in the image according to the enlarged area of each specific body area in the image, it is specifically configured to: place each specific body area in the image Adjust the size of the enlarged area in, to a preset size, to obtain the enlarged area of the preset size corresponding to each specific body area, and the preset size meets the size of the input image of the detection model; The enlarged area of the preset size corresponding to a specific body area is input into the detection model, and the specific posture in the enlarged area of the preset size corresponding to each specific body area is detected by the detection model .

Optionally, when the processor 202 adjusts the size of the enlarged area of each specific body area in the image to a preset size, it is specifically configured to: The expanded area in the image is cropped to obtain a cropped area corresponding to each expanded area; the size of the cropped area corresponding to each expanded area is adjusted to the preset size.

The specific principles and implementation manners of the control device provided in the embodiments of the present application are similar to the foregoing embodiments, and will not be repeated here.

The embodiment of the present invention provides a handheld PTZ. The handheld pan/tilt includes: a pan/tilt, a base, and the control device as described in the above embodiment, wherein the pan/tilt is used to carry a photographing device, the pan/tilt is connected to the base, and the control device The specific principles and implementation manners of the executed control method of the handheld pan/tilt head are consistent with the foregoing embodiment, and will not be repeated here.

In addition, this embodiment also provides a computer-readable storage medium on which a computer program is stored, and the computer program is executed by a processor to implement the method for controlling the handheld pan/tilt head described in the foregoing embodiment.

In the several embodiments provided in this application, it should be understood that the disclosed device and method may be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division, and there may be other divisions in actual implementation, for example, multiple units or components can be combined or It can be integrated into another system, or some features can be ignored or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple detection units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.

In addition, the functional units in each embodiment of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware, or may be implemented in the form of hardware plus software functional units.

The above-mentioned integrated unit implemented in the form of a software functional unit may be stored in a computer readable storage medium. The above-mentioned software functional unit is stored in a storage medium and includes several instructions to make a computer device (which may be a personal computer, a server, or a detection device, etc.) or a processor execute the method described in the various embodiments of this application. Part of the steps. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program code .

Those skilled in the art can clearly understand that, for the convenience and conciseness of the description, only the division of the above-mentioned functional modules is used as an example. In practical applications, the above-mentioned functions can be allocated by different functional modules as required, namely, the device The internal structure is divided into different functional modules to complete all or part of the functions described above. For the specific working process of the device described above, reference may be made to the corresponding process in the foregoing method embodiment, which is not repeated here.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the application, not to limit them; although the application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand: It is still possible to modify the technical solutions described in the foregoing embodiments, or equivalently replace some or all of the technical features; these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the technical solutions of the embodiments of the application range.

Claims (76)

1. A method for controlling a handheld PTZ, characterized in that a photographing device is mounted on the handheld PTZ, and the method includes:

   Acquiring an image collected by the photographing device;

   Detecting a specific posture in the image;

   Determining the following object in the image according to the specific posture;

   Control the handheld pan/tilt to follow the following object, and control the photographing device to photograph the following object during the following process.
2. The method according to claim 1, wherein the determining a follower object in the image according to the specific posture comprises:

   Detecting at least one specific body area in the image;

   It is determined that an object in the specific body area matching the target object issuing the specific posture in the at least one specific body area is a follow object.
3. The method according to claim 1, wherein the determining a follower object in the image according to the specific posture comprises:

   Detecting at least one specific body area in the image;

   It is determined that an object in the specific body area whose distance from the target object issuing the specific posture satisfies a first preset condition in the at least one specific body area is a follower object.
4. The method according to any one of claims 1 to 3, wherein the controlling the handheld pan/tilt to follow the following object comprises:

   Control the handheld pan/tilt to follow the following object, so that the following object is located at a preset position in the shooting frame of the shooting device.
5. The method according to any one of claims 1 to 3, wherein the controlling the handheld pan/tilt to follow the following object comprises:

   Control the handheld pan/tilt to follow the following object in a direction with a preset rotation angle.
6. The method according to claim 4 or 5, wherein the controlling the photographing device to photograph the following object during the following process comprises:

   During the following process, the photographing device is controlled to photograph the following object with the position of the following object as the center.
7. The method according to any one of claims 1-6, wherein the method further comprises:

   According to the specific posture, control the photographing device to photograph the following object after a preset period of time.
8. The method according to any one of claims 1-7, wherein the method further comprises:

   Determining the target object emitting the specific posture in the image;

   Wherein, the specific posture issued by the target object is used to instruct the handheld pan/tilt to execute the following control instruction for the following object, and/or to instruct the photographing device to perform the following control instruction on the following object Describe shooting control commands for shooting.
9. The method according to claim 8, wherein the determining the target object in the image that emits the specific posture comprises:

   Detecting at least one specific body area in the image;

   Determining an enlarged area of each of the at least one specific body area in the image;

   According to the enlarged area of each specific body area in the image, the target object issuing the specific posture is determined.
10. The method according to claim 9, wherein the determining the target object issuing the specific posture according to the enlarged area of each specific body area in the image comprises:

    Detecting whether each of the specific body regions has the specific posture in the enlarged area in the image;

    If the specific body area has the specific posture in the enlarged area in the image, it is determined that the object in the specific body area is the target object.
11. The method according to claim 10, wherein the specific posture comprises a specific one-handed gesture, and the issuing position of the specific one-handed gesture is arbitrary;

    The determining that the object in the specific body area is the target object if the specific posture exists in the enlarged area in the image of the specific body area includes:

    If the specific one-handed gesture exists in the enlarged area of the image for the specific body area, and the specific one-handed gesture matches the body in the specific body area, then the specific body area is determined The object in is the target object.
12. The method according to claim 11, wherein the target object is the following object.
13. The method of claim 11, wherein the method further comprises:

    If the specific one-handed gesture exists in the enlarged area of the image for the specific body area, and the specific one-handed gesture does not match the body in the specific body area, determine the specific body The object in the area is the follow object, and it is determined that the object in the specific body area corresponding to the body matching the specific one-hand gesture is the target object.
14. The method according to claim 10, wherein the method further comprises:

    If a plurality of the specific body regions have the specific posture in the enlarged area in the image, determine an object in the specific body region that satisfies a second preset condition among the plurality of specific body regions as the target Object.
15. The method according to claim 14, wherein the second preset condition comprises that the distance from the center of the image is less than a first preset threshold and/or the frame ratio in the image is greater than the second Preset threshold;

    Wherein, the frame ratio is determined according to the size of the specific body area or the enlarged area of the specific body area in the image.
16. The method according to any one of claims 10-15, wherein the detecting whether each of the specific body regions has the specific posture in the enlarged area in the image comprises:

    The size of the enlarged area of each specific body area in the image is adjusted to a preset size to obtain the enlarged area of the preset size corresponding to each specific body area, and the preset size satisfies the detection The size of the input image of the model;

    The enlarged area of the preset size corresponding to each specific body area is input into the detection model, and the enlarged area of the preset size corresponding to each specific body area is detected by the detection model Whether the specific posture exists.
17. The method according to claim 16, wherein the adjusting the size of the enlarged area of each specific body area in the image to a preset size comprises:

    Performing cropping processing on the enlarged region of each specific body region in the image in the image, to obtain a cropped region corresponding to each of the enlarged regions;

    Adjusting the size of the cropping area corresponding to each of the enlarged areas to the preset size.
18. The method according to claim 17, wherein the size of the cropped area corresponding to the enlarged area is the same as the size of the enlarged area.
19. The method according to claim 17, wherein a part of the cropped area corresponding to the enlarged area is outside the image.
20. The method according to claim 10, wherein the specific posture includes a specific hand posture, the specific body area includes a head and shoulders area, and the determining each of the at least one specific body area The enlarged area of a specific body area in the image includes:

    Determining the width of the enlarged area of each specific body area in the image according to the width of the shoulder in each of the at least one specific body area;

    Determining the height of the enlarged area of each specific body area in the image according to the width of the enlarged area of each specific body area in the image;

    The enlarged area of each specific body area in the image is determined according to the width and the height.
21. 22. The method according to claim 20, wherein the determining that each of the specific body regions is in the image according to the shoulder width in each of the at least one specific body region The width of the enlarged area in includes:

    Determine the width of the enlarged area of each specific body area in the image according to the width of the shoulder in each specific body area in the at least one specific body area and the expansion multiple of the width of the shoulder .
22. The method according to claim 20 or 21, characterized in that, according to the width of the enlarged area of each of the specific body regions in the image, determine the width of each specific body region in the image Increase the height of the area, including:

    According to the width of the enlarged area of each specific body area in the image, and the ratio between the width and height of the enlarged area of each specific body area in the image, each specific The height of the enlarged area of the body area in the image.
23. The method according to claim 10, wherein the specific posture includes a specific body posture, the specific body area includes a head and shoulder area, and the determining each of the at least one specific body area The enlarged area of the body area in the image includes:

    Determining the height of the enlarged area of each specific body area in the image according to the height of the head and shoulders in each of the at least one specific body area;

    Determining the width of the enlarged area of each specific body area in the image according to the height of the enlarged area of each specific body area in the image;

    The enlarged area of each specific body area in the image is determined according to the width and the height.
24. 23. The method according to claim 23, wherein the at least one specific body area is determined to be in the image according to the head and shoulder height of each of the specific body areas. The height of the expanded area in, including:

    Determine the height of the enlarged area of each specific body area in the image according to the height of the head and shoulders in each specific body area in the at least one specific body area and the magnification factor of the head and shoulders height .
25. The method according to claim 23 or 24, wherein the determination of the height of each specific body area in the image according to the height of the enlarged area of each specific body area in the image Expand the width of the area, including:

    According to the height of the enlarged area of each specific body area in the image and the ratio between the height and the width of the enlarged area of each specific body area in the image, each specific body area is determined The width of the enlarged area of the body area in the image.
26. A method for controlling a handheld PTZ, characterized in that a photographing device is mounted on the handheld PTZ, and the method includes:

    Acquiring an image collected by the photographing device;

    Detecting a specific posture in the image;

    Controlling the parameters of the handheld pan/tilt and/or the photographing device according to the specific posture.
27. The method according to claim 26, wherein the parameters of the handheld PTZ include at least one of the following:

    The posture, rotation speed, and rotation direction of the handheld PTZ.
28. The method according to claim 26, wherein the parameters of the photographing device include at least one of the following:

    The shutter parameters of the shooting device, the lens focal length of the shooting device, and the follow focus parameters of the shooting device.
29. The method according to any one of claims 26-28, wherein the detecting a specific posture in the image comprises:

    According to the overall area of the image, a specific posture in the image is detected.
30. The method of claim 29, wherein the detecting a specific posture in the image according to the overall area of the image comprises:

    When the distance of the shooting device relative to the shooting object is less than or equal to the preset distance, the specific posture in the image is detected according to the overall area of the image.
31. The method of claim 29, wherein the detecting a specific posture in the image according to the overall area of the image comprises:

    When the photographing device uses a front camera to capture the image, it detects a specific posture in the image according to the overall area of the image.
32. The method according to any one of claims 26-28, wherein the detecting a specific posture in the image comprises:

    According to a partial area of the image, a specific posture in the image is detected.
33. The method according to claim 32, wherein the detecting a specific posture in the image according to a partial area of the image comprises:

    When the distance of the photographing device relative to the photographing object is greater than a preset distance, detecting a specific posture in the image according to a partial area of the image.
34. The method according to claim 32, wherein the detecting a specific posture in the image according to a partial area of the image comprises:

    When the photographing device uses a rear camera to capture the image, it detects a specific posture in the image according to a partial area of the image.
35. The method according to claim 33 or 34, wherein the partial area of the image includes at least one specific body area in the image;

    The detecting a specific posture in the image according to a partial area of the image includes:

    Determining an enlarged area of each of the at least one specific body area in the image;

    According to the enlarged area of each of the specific body regions in the image, a specific posture in the image is detected.
36. The method according to claim 35, wherein the detecting a specific posture in the image according to the enlarged area of each specific body region in the image comprises:

    The size of the enlarged area of each specific body area in the image is adjusted to a preset size to obtain the enlarged area of the preset size corresponding to each specific body area, and the preset size satisfies the detection The size of the input image of the model;

    The enlarged area of the preset size corresponding to each specific body area is input into the detection model, and the enlarged area of the preset size corresponding to each specific body area is detected by the detection model Of the specific posture.
37. The method according to claim 36, wherein the adjusting the size of the enlarged area of each of the specific body regions in the image to a preset size comprises:

    Performing cropping processing on the enlarged region of each specific body region in the image in the image, to obtain a cropped region corresponding to each of the enlarged regions;

    Adjusting the size of the cropping area corresponding to each of the enlarged areas to the preset size.
38. A control device for a handheld pan/tilt head, characterized in that a photographing device is mounted on the handheld pan/tilt head, and the control device includes a memory and a processor;

    The memory is used to store program codes;

    The processor calls the program code, and when the program code is executed, is used to perform the following operations:

    Acquiring an image collected by the photographing device;

    Detecting a specific posture in the image;

    Determining the following object in the image according to the specific posture;

    Control the handheld pan/tilt to follow the following object, and control the photographing device to photograph the following object during the following process.
39. The control device according to claim 38, wherein when the processor determines the following object in the image according to the specific posture, it is specifically configured to:

    Detecting at least one specific body area in the image;

    It is determined that an object in the specific body area matching the target object issuing the specific posture in the at least one specific body area is a follow object.
40. The control device according to claim 38, wherein when the processor determines the following object in the image according to the specific posture, it is specifically configured to:

    Detecting at least one specific body area in the image;

    It is determined that an object in the specific body area whose distance from the target object issuing the specific posture satisfies a first preset condition in the at least one specific body area is a follower object.
41. The control device according to any one of claims 38-40, wherein when the processor controls the handheld pan/tilt to follow the following object, it is specifically configured to:

    Control the handheld pan/tilt to follow the following object, so that the following object is located at a preset position in the shooting frame of the shooting device.
42. The control device according to any one of claims 38-40, wherein when the processor controls the handheld pan/tilt to follow the following object, it is specifically configured to:

    Control the handheld pan/tilt to follow the following object in a direction with a preset rotation angle.
43. The control device according to claim 41 or 42, wherein when the processor controls the photographing device to photograph the following object during the following process, it is specifically configured to:

    During the following process, the photographing device is controlled to photograph the following object with the position of the following object as the center.
44. The control device according to any one of claims 38-43, wherein the processor is further configured to:

    According to the specific posture, control the photographing device to photograph the following object after a preset period of time.
45. The control device according to any one of claims 38-44, wherein the processor is further configured to:

    Determining the target object emitting the specific posture in the image;

    Wherein, the specific posture issued by the target object is used to instruct the handheld pan/tilt to execute the following control instruction for the following object, and/or to instruct the photographing device to perform the following control instruction on the following object Describe shooting control commands for shooting.
46. The control device according to claim 45, wherein when the processor determines the target object emitting the specific posture in the image, it is specifically configured to:

    Detecting at least one specific body area in the image;

    Determining an enlarged area of each of the at least one specific body area in the image;

    According to the enlarged area of each specific body area in the image, the target object issuing the specific posture is determined.
47. The control device according to claim 46, wherein the processor is specifically configured to: when determining the target object issuing the specific posture according to the enlarged area of each specific body area in the image:

    Detecting whether each of the specific body regions has the specific posture in the enlarged area in the image;

    If the specific body area has the specific posture in the enlarged area in the image, it is determined that the object in the specific body area is the target object.
48. The control device according to claim 47, wherein the specific posture comprises a specific one-handed gesture, and the issuing position of the specific one-handed gesture is arbitrary;

    If the specific one-handed gesture exists in the enlarged area of the image in the specific body area, and the specific one-handed gesture matches the body in the specific body area, the processor determines The object in the specific body area is the target object.
49. The control device according to claim 48, wherein the target object is the following object.
50. The control device according to claim 48, wherein the processor is further configured to:

    If the specific one-handed gesture exists in the enlarged area of the image for the specific body area, and the specific one-handed gesture does not match the body in the specific body area, determine the specific body The object in the area is the follow object, and it is determined that the object in the specific body area corresponding to the body matching the specific one-hand gesture is the target object.
51. The control device according to claim 47, wherein the processor is further configured to:

    If a plurality of the specific body regions have the specific posture in the enlarged area in the image, determine an object in the specific body region that satisfies a second preset condition among the plurality of specific body regions as the target Object.
52. The control device according to claim 51, wherein the second preset condition comprises that the distance from the center of the image is less than a first preset threshold and/or the proportion of the screen in the image is greater than the first preset threshold. 2. The preset threshold;

    Wherein, the frame ratio is determined according to the size of the specific body area or the enlarged area of the specific body area in the image.
53. The control device according to any one of claims 47-52, wherein the processor detects whether each of the specific body regions has the specific posture in the enlarged area of the image, specifically using in:

    The size of the enlarged area of each specific body area in the image is adjusted to a preset size to obtain the enlarged area of the preset size corresponding to each specific body area, and the preset size satisfies the detection The size of the input image of the model;

    The enlarged area of the preset size corresponding to each specific body area is input into the detection model, and the enlarged area of the preset size corresponding to each specific body area is detected by the detection model Whether the specific posture exists.
54. The control device according to claim 53, wherein when the processor adjusts the size of the enlarged area of each specific body area in the image to a preset size, it is specifically configured to:

    Performing cropping processing on the enlarged region of each specific body region in the image in the image, to obtain a cropped region corresponding to each of the enlarged regions;

    Adjusting the size of the cropping area corresponding to each of the enlarged areas to the preset size.
55. The control device according to claim 54, wherein the size of the cropped area corresponding to the enlarged area is the same as the size of the enlarged area.
56. The control device according to claim 54, wherein a part of the cropped area corresponding to the enlarged area is outside the image.
57. The control device according to claim 47, wherein the specific posture includes a specific hand posture, the specific body area includes a head and shoulders area, and the processor determines each of the at least one specific body area. When the specific body area is an enlarged area in the image, it is specifically used for:

    Determining the width of the enlarged area of each specific body area in the image according to the width of the shoulder in each of the at least one specific body area;

    Determining the height of the enlarged area of each specific body area in the image according to the width of the enlarged area of each specific body area in the image;

    The enlarged area of each specific body area in the image is determined according to the width and the height.
58. The control device according to claim 57, wherein the processor determines that each of the at least one specific body area is in each of the specific body areas according to the shoulder width in each of the specific body areas. The width of the enlarged area in the image is specifically used for:

    Determine the width of the enlarged area of each specific body area in the image according to the width of the shoulder in each specific body area in the at least one specific body area and the expansion multiple of the width of the shoulder .
59. The control device according to claim 57 or 58, wherein the processor determines that each specific body area is in the image according to the width of the enlarged area of each specific body area in the image. The height of the enlarged area in the image is specifically used for:

    According to the width of the enlarged area of each specific body area in the image, and the ratio between the width and height of the enlarged area of each specific body area in the image, each specific body area is determined The height of the enlarged area of the body area in the image.
60. The control device according to claim 47, wherein the specific posture includes a specific body posture, the specific body area includes a head and shoulder area, and the processor determines each of the at least one specific body area When the specific body area is an enlarged area in the image, it is specifically used for:

    Determining the height of the enlarged area of each specific body area in the image according to the height of the head and shoulders in each of the at least one specific body area;

    Determine the width of the expanded area of each specific body area in the image according to the height of the expanded area of each specific body area in the image;

    The enlarged area of each specific body area in the image is determined according to the width and the height.
61. The control device according to claim 60, wherein the processor determines that each of the specific body regions is at the height of the head and shoulders in each of the at least one specific body region. The height of the expanded area in the image is specifically used for:

    Determine the height of the enlarged area of each specific body area in the image according to the height of the head and shoulders in each specific body area in the at least one specific body area and the expansion multiple of the head and shoulders height .
62. The control device according to claim 60 or 61, wherein the processor determines that each of the specific body areas is in the image according to the height of the enlarged area in the image. When the width of the enlarged area in the image, it is specifically used for:

    According to the height of the enlarged area of each specific body area in the image and the ratio between the height and the width of the enlarged area of each specific body area in the image, each specific body area is determined The width of the enlarged area of the body area in the image.
63. A control device for a handheld pan/tilt head, characterized in that a photographing device is mounted on the handheld pan/tilt head, and the control device includes a memory and a processor;

    The memory is used to store program codes;

    The processor calls the program code, and when the program code is executed, is used to perform the following operations:

    Acquiring an image collected by the photographing device;

    Detecting a specific posture in the image;

    Controlling the parameters of the handheld pan/tilt and/or the photographing device according to the specific posture.
64. The control device according to claim 63, wherein the parameters of the handheld PTZ include at least one of the following:

    The posture, rotation speed, and rotation direction of the handheld PTZ.
65. The control device according to claim 63, wherein the parameters of the photographing device include at least one of the following:

    The shutter parameters of the shooting device, the lens focal length of the shooting device, and the follow focus parameters of the shooting device.
66. The control device according to any one of claims 63-65, wherein when the processor detects a specific posture in the image, it is specifically configured to:

    According to the overall area of the image, a specific posture in the image is detected.
67. The control device according to claim 66, wherein the processor is specifically configured to: when detecting a specific posture in the image according to the overall area of the image:

    When the distance of the shooting device relative to the shooting object is less than or equal to the preset distance, the specific posture in the image is detected according to the overall area of the image.
68. The control device according to claim 66, wherein the processor is specifically configured to: when detecting a specific posture in the image based on the overall area of the image:

    When the photographing device uses a front camera to capture the image, it detects a specific posture in the image according to the overall area of the image.
69. The control device according to any one of claims 63-65, wherein when the processor detects a specific posture in the image, it is specifically configured to:

    According to a partial area of the image, a specific posture in the image is detected.
70. The control device according to claim 69, wherein the processor is specifically configured to: when detecting a specific posture in the image according to a partial area of the image:

    When the distance of the photographing device relative to the photographing object is greater than a preset distance, detecting a specific posture in the image according to a partial area of the image.
71. The control device according to claim 69, wherein the processor is specifically configured to: when detecting a specific posture in the image according to a partial area of the image:

    When the photographing device uses a rear camera to capture the image, it detects a specific posture in the image according to a partial area of the image.
72. The control device according to claim 70 or 71, wherein a partial area of the image includes at least one specific body area in the image;

    When the processor detects a specific posture in the image according to a partial area of the image, it is specifically configured to:

    Determining an enlarged area of each of the at least one specific body area in the image;

    According to the enlarged area of each of the specific body regions in the image, a specific posture in the image is detected.
73. The control device according to claim 72, wherein the processor is specifically configured to: when detecting a specific posture in the image according to the enlarged area of each specific body region in the image:

    The size of the enlarged area of each specific body area in the image is adjusted to a preset size to obtain the enlarged area of the preset size corresponding to each specific body area, and the preset size satisfies the detection The size of the input image of the model;

    Input the enlarged area of the preset size corresponding to each of the specific body regions into the detection model, and detect the enlarged area of the preset size corresponding to each of the specific body regions through the detection model Of the specific posture.
74. The control device according to claim 73, wherein when the processor adjusts the size of the enlarged area of each specific body area in the image to a preset size, it is specifically configured to:

    Performing cropping processing on the enlarged region of each specific body region in the image in the image, to obtain a cropped region corresponding to each of the enlarged regions;

    Adjusting the size of the cropping area corresponding to each of the enlarged areas to the preset size.
75. A handheld pan/tilt, characterized in that it comprises:

    A pan/tilt, on which photographing equipment is carried;

    A base, the base station is connected to the pan-tilt; and

    The control device according to any one of claims 38-74.
76. A computer-readable storage medium, characterized in that a computer program is stored thereon, and the computer program is executed by a processor to implement the method according to any one of claims 1-37.

Images