WO2020048509A1 - Inter-frame area mapping method and apparatus, and multi-camera observing system - Google Patents

Abstract

Embodiments of the present application provide an inter-frame area mapping method and apparatus, and a multi-camera observing system. The method comprises: determining a target area designated in a first frame to be subjected to area mapping; determining a target coordinate conversion relationship between a camera to which the first frame belongs and a camera to which a second frame to be subjected to area mapping belongs; and on the basis of the target area and the target coordinate conversion relationship, determining in the second frame an area corresponding to the target area. By applying the method provided by the embodiments of the present application, when a target area is given in a frame, an area corresponding to the target are can be effectively determined in another frame.

Description

Method and device for mapping regions between pictures and multi-camera observation system

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office on September 6, 2018, with the application number 201811038935.3 and the invention name "a method, device and multi-camera observation system for inter-screen region mapping". Citations are incorporated in this application.

Technical field

The present application relates to the technical field of intelligent video surveillance, and in particular, to a method, a device, and a multi-camera observation system for mapping regions between pictures.

Background technique

At present, with the continuous development of information technology, the application of multi-camera observation systems is increasing, for example, smart home, fire warning, road detection, and so on. The multi-camera observation system can shoot the same subject from multiple shooting angles. Specifically, the multi-camera observation system can include a dome camera. By changing the azimuth of the dome camera, the dome camera can be aligned with different shooting angles. One subject is taken to achieve the effect of using a single dome to shoot the same subject from multiple angles; in addition, a multi-camera observation system can also include multiple cameras, and each camera can face the same from different shooting angles The subject shoots.

In practical applications, it is often necessary to perform image processing on an area of a shooting object in a shooting frame, for example, covering the area or adding an AR tag to the area. Since the multi-camera observation system can shoot the same subject from multiple shooting angles, it is necessary to perform picture processing for a region corresponding to the subject on multiple shooting pictures. Furthermore, since the shooting angles corresponding to different shooting pictures are different, the coordinates of the areas corresponding to the same shooting subject in different shooting pictures are also different.

Therefore, for multiple shooting frames for the same subject collected by the multi-camera observation system from different shooting angles, when a target area is given in one shooting frame, how to effectively determine the corresponding target area in another shooting frame The region is an urgent problem.

Summary of the Invention

The purpose of the embodiments of the present invention is to provide a method, device and multi-camera observation system for region mapping between pictures, so that when a target region is given in one shooting frame, an area corresponding to the target region is effectively determined in another frame. the goal of. Specific technical solutions are as follows:

In a first aspect, an embodiment of the present application provides a method for mapping regions between pictures, where the method includes:

Determining the target area specified in the first picture of the area to be mapped;

Determining a target coordinate conversion relationship between a camera to which the first picture belongs and a camera to which the second picture to be mapped belongs;

Based on the target area and the target coordinate conversion relationship, an area corresponding to the target area is determined in the second picture.

In a second aspect, an embodiment of the present application adopts a device for mapping regions between pictures, where the device includes:

A first area determination module, configured to determine a target area specified in a first picture to be area-mapped;

A transformation relationship determining module, configured to determine a target coordinate transformation relationship between a camera to which the first picture belongs and a camera to which the second picture to be mapped belongs;

A second region determining module is configured to determine a region corresponding to the target region in the second screen based on the target region and the target coordinate conversion relationship.

According to a third aspect, an embodiment of the present invention provides that the system includes a controller and at least one camera, where the at least one camera includes a camera to which a first picture to be area mapped belongs and a camera to which a second picture to be area mapped belongs;

The camera to which the first picture belongs is used to collect the first picture to be mapped in the area to be mapped;

The camera to which the second picture belongs is used to collect the second picture to be mapped in the area to be mapped;

The controller is configured to determine a target area specified in the first picture; determine a target coordinate conversion relationship between a camera to which the first picture belongs and a camera to which the second picture belongs; based on the target area and the target The coordinate conversion relationship determines a region corresponding to the target region in the second screen.

According to a fourth aspect, an embodiment of the present invention provides an electronic device. The electronic device is a controller in a multi-camera observation system. The electronic device includes a processor, a communication interface, a memory, and a communication bus. The processor, Communication interface, the memory communicates with each other through the communication bus;

Memory for storing computer programs;

The processor is configured to implement the steps of any method for mapping between regions of a screen provided by the first aspect when the program stored in the memory is executed.

In a fifth aspect, an embodiment of the present invention provides a computer-readable storage medium, characterized in that a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the foregoing first aspect provides Steps of any inter-screen region mapping method.

It can be seen from the above that in the solution provided by the embodiment of the present invention, the target coordinate conversion relationship between the camera to which the first picture to be area mapped belongs and the camera to which the second picture to be area mapped belongs, and further, when the After the target area is determined in one picture, the area corresponding to the target area can be determined in the second picture to be area mapped according to the target coordinate conversion relationship and the target area, so that when the target area is given in one picture, , Effectively determine the area corresponding to the target area in another picture.

In addition, the solution provided by the embodiment of the present invention can be adapted to different types of cameras, and has a good adaptability. At the same time, the area corresponding to the target area is determined according to the target coordinate conversion relationship. The camera to which a picture belongs and the camera to which the second picture to be mapped belongs are determined. Therefore, the operation of determining the area corresponding to the target area is simple and does not require additional information.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly explain the embodiments of the present application and the technical solutions of the prior art, the following briefly introduces the drawings used in the embodiments and the prior art. Obviously, the drawings in the following description are only the present invention. Some embodiments of the application, for those of ordinary skill in the art, can obtain other drawings according to the drawings without paying creative labor.

FIG. 1 is a schematic diagram of field angle information of the ball machine;

FIG. 2 is a schematic diagram of a dome camera image changing with changes in the azimuth and magnification of the dome camera according to an embodiment of the present application; FIG.

FIG. 3 is a picture with distortion collected by a fisheye camera;

4 is a distorted picture collected by a gun with an excessively large field of view;

5 is a schematic flowchart of a method for mapping regions between pictures according to an embodiment of the present application;

6 is a schematic flowchart of a method for determining a first coordinate conversion relationship according to an embodiment of the present application;

7 is a schematic diagram of a dome camera;

FIG. 8 is a schematic structural diagram of a device for mapping between regions of an image provided by an embodiment of the present application; FIG.

9 is a schematic structural diagram of an electronic device according to an embodiment of the present application;

FIG. 10 is a schematic structural diagram of a multi-camera observation system according to an embodiment of the present application.

detailed description

In order to make the purpose, technical solution, and advantages of the present application clearer and clearer, the following describes the present application in detail with reference to the accompanying drawings and examples. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

Currently, for a plurality of pictures corresponding to different angles, which are collected by a multi-camera observation system, for a same subject, when a target area is given in one picture, how to effectively determine an area corresponding to the target area in another picture Is an urgent problem. In order to be able to realize that when a target area is given in one picture, an area corresponding to the target area is effectively determined in another picture, embodiments of the present application provide a method, a device, and a multi-camera observation system for mapping areas between pictures.

In order to facilitate understanding of the solutions in the embodiments of the present application, the following first briefly describes related concepts involved in the embodiments of the present application.

(1) Multi-camera observation system: An observation system with one or more cameras or devices that can perform two-dimensional imaging and reflect the target position of the shooting object. These cameras or devices may include: fisheye cameras, guns, Dome, radar, etc. Among them, the traditional gun-ball linkage, fish-ball linkage, multi-gun multi-ball linkage, and radar dome linkage can all be considered as a subclass of the multi-camera observation system, that is, all can be considered as a multi-camera observation system.

(2) Gun machine: a type of monitoring CCD (Charge Coupled Device) camera, which is characterized by stable imaging. Specifically, in the embodiment of the present application, after the gun is installed, the shooting angle is fixed, and further, the three-dimensional space corresponding to the captured image is fixed. It can be understood that, for a point with a fixed position in the three-dimensional space, the coordinates of the corresponding point in the picture collected by the same gun machine are fixed.

(3) Dome camera: a camera used to observe the details of the target. Specifically, in the embodiment of the present application, the dome camera refers to a dome camera having PTZ (pan / tilt / zoom) direction and zoom information, and may also be referred to as a PTZ camera. Among them, P refers to pan, which represents the azimuth angle in the horizontal direction; T refers to tilt, which represents the azimuth angle in the vertical direction, and determines the range of the field of view of the dome camera is Z, which is the zoom factor. Among them, P and T can be collectively referred to as the azimuth of the ball machine, which is expressed by the PT value. As shown in Figure 1, the PT value can be expressed as (P, T), where P refers to pan and represents the horizontal azimuth; T refers to tilt and represents the vertical azimuth. It can be understood that the larger the zoom and the larger the magnification, the more pixels the same target will occupy in the picture captured by the PTZ camera, but the smaller the overall field of view angle of the captured picture.

In the embodiment of the present application, the characteristic of the dome camera is that the azimuth angle can be changed. Generally, according to the type of the dome camera, the range of angle that the dome camera can rotate in the horizontal direction is (0,180) degrees or (0,360) degrees. The angle range that can be rotated in the straight direction is (-90,90) degrees. When the azimuth of the dome camera is changed, the shooting angle of the dome camera will change. It is understandable that for a fixed point in three-dimensional space, the images collected when the same dome camera is at different azimuth angles The coordinates of the corresponding points are different. Generally, the coordinates of the center point of the picture currently collected by the dome camera can be characterized by the current azimuth of the dome camera, that is, the coordinates of the center point of the picture collected by the dome camera can be expressed as the PT value when the dome camera collects the picture.

The azimuth and zoom factor of the dome camera can be collectively referred to as the field of view information of the dome camera. Obviously, when the field of view information of the dome camera is different, the shooting angle and the field of view of the screen are also different. Generally, when a dome is manufactured and shipped, the manufacturer of the dome will give a field of view information table of the dome, for example, as shown in Table 1, it is a field of view information table of a certain dome.

Table 1

ZOOM	1	1.1	1.2	1.3	...
PAN	56.78049	54.60501	51.61468	49.78598	...
TILT	33.81852	32.37984	30.43246	29.25658	...

Since the dome itself can be rotated and zoomed to change the azimuth and magnification of the dome camera, the same dome camera can shoot the same subject from different shooting angles, and according to the different magnification, the same subject is The size of the area occupied in different pictures is also different.

For example, as shown in FIG. 2, in FIG. 2a, the target area 201 is located on the right side of the screen captured by the dome camera, and it is marked; then the dome camera rotates to change its azimuth. In FIG. 2b, it is clear that the target The position of the area 201 has changed, and is located on the left side of the screen collected by the dome camera. Then, the dome camera is zoomed. In FIG. 2c, it is clear that the position and size of the target area 201 have changed.

(3) Fisheye camera: A fisheye camera is a camera with a fisheye lens. The so-called fisheye lens is a lens with a very short focal length and a viewing angle close to or greater than 180 degrees. As an extreme wide-angle lens, the fisheye lens has a very large deformation, so that except for the scene in the center of the picture, other scenes that should be horizontal or vertical have changed accordingly. This deformation can be Call it distortion. Specifically, as shown in FIG. 3, a picture acquired by a certain fisheye camera.

In order to establish an accurate coordinate conversion relationship between cameras, the images collected by the fisheye camera can be corrected for distortion, and the images collected by the fisheye camera after distortion correction can be regarded as the images collected by the gun.

Specifically, the stable Zhang Zhengyou calibration method can be used to correct the distortion of the picture collected by the fisheye camera. Specifically:

A. First collect several pictures with complete and clear calibration boards in the pictures;

B. Use the corresponding function of OpenCV (Open Source Computer Vision Library) to detect the corner points on the corresponding calibration board;

C. Determine the three-dimensional coordinates of the three-dimensional points corresponding to the corner points according to the size of the calibration plate;

D. According to the camera type, call the OpenCV corresponding function to obtain the camera's internal parameters and distortion parameters;

E. According to internal parameters and distortion parameters, refer to different camera types to obtain the coordinate mapping table of the distorted picture and the corrected picture.

Among them, the camera's internal parameters can be uniformly expressed by the following formula:

Among them, (cx, cy) is the reference point (usually the center point of the screen), fx, fy is the focal length of the camera, and its unit is pixel. When the picture is up-sampled or down-sampled, the parameters (fx, fy, cx, cy) will be scaled (multiplied or divided) by the same scale. It should be noted that the internal parameters of the camera do not depend on the view angle of the scene corresponding to the camera. When the focal length of the camera is fixed, the calculated internal parameters of the camera are also fixed and can be reused.

In many cases, when the field of view of the gun is too large, the picture captured by the gun will also be distorted, as shown in Figure 4. In this case, for the distorted pictures collected by the gun with a large viewing angle, the above-mentioned Zhang Zhengyou calibration method can also be used to correct the distortion of these pictures.

It should be noted that due to the characteristics of the fisheye camera's approximate circular imaging, the calculation method of the fisheye camera in the distortion correction process is different for a gun with a large field of view.

(4) The guns mentioned in the embodiments of the present application include not only cameras with a camera type of guns, but also fisheye cameras that correct the distortion of the captured images, as well as radar and thermal imaging cameras. A device that provides fixed two-dimensional imaging information.

The following describes an inter-screen region mapping method provided by an embodiment of the present application.

It should be noted that the method for mapping regions between pictures provided in the embodiments of the present application can be applied to a controller of a multi-camera observation system. Specifically, the controller may be various electronic devices such as a tablet computer, a notebook computer, and a desktop computer, and this embodiment is not specifically limited.

FIG. 5 is a schematic flowchart of an inter-screen region mapping method according to an embodiment of the present application. As shown in FIG. 5, the inter-screen region mapping method may include the following steps:

S501: Determine a target area specified in a first picture to be area-mapped;

It is understandable that the so-called inter-screen region mapping is that when a target region is given in one screen, the region corresponding to the target region is effectively determined in another screen, so that the screen content of the target region and the determined region can be directed to the same The subject. In this way, when it is necessary to perform image processing on the area of a certain subject in the picture, the area corresponding to the subject can be determined in different pictures collected from multiple shooting angles through the inter-screen area mapping method, and The determined screen area is processed.

In the embodiment of the present application, the different pictures collected from these multiple angles may be referred to as pictures to be area mapped, and the picture used to determine the target area may be referred to as the first picture to be area mapped.

It should be noted that, in the above step S501, the controller may determine the designated target area in the first screen to be mapped based on the target area determination instruction input by the user.

For example, the user can mark the target area on the first screen by dragging the mouse, and the controller can use the area surrounded by the path the mouse cursor passes as the target area; for example, if the first screen is displayed on the touch screen, then The user can also mark the target area directly on the touch screen with the hand, and the controller can use the area surrounded by the path that the user's finger travels on the touch screen as the target area; for example, the user can input several screens The coordinates of the pixel points, the controller may use the area surrounded by the line of the pixel points corresponding to these coordinates as the target area.

In addition, in step S501, the controller may further determine, as a target region, a region corresponding to a specific photographic object in the first screen according to a preset rule.

For example, when the subject of the first picture is a person, the controller may use the picture corresponding to the face of the subject as the target area in the first picture according to a preset rule.

Of course, in the embodiment of the present application, when the above step S501 is performed, in addition to the above description, the controller may also determine the target area specified in the first screen mapped by the area to be mapped in other ways. The manners of the target area specified in the mapped first picture all belong to the protection scope of the embodiment of the present application.

S502: Determine a target coordinate conversion relationship between a camera to which a first picture belongs and a camera to which a second picture to be mapped belongs.

Each frame to be mapped is obtained by a camera. In an embodiment of the present application, the controller may determine a target between the camera to which the first frame to be mapped belongs and the camera to which the second frame to be mapped belongs. Coordinate conversion relationship.

For the convenience of description, the camera to which the first picture to be area mapped belongs to the camera to which the first picture belongs, and the camera to which the second picture to be area map belongs to the camera to which the second picture belongs.

In actual applications, the camera to which the first picture belongs and the camera to which the second picture belongs may be different in different scenarios. Specifically, the camera to which the first picture belongs and the camera to which the second picture belongs may be one of the following situations:

In the first case, the camera to which the first picture belongs is a gun camera, and the camera to which the second picture belongs is a dome camera;

In the second case, the camera to which the first picture belongs is a dome camera, and the camera to which the second picture belongs is a gun machine;

In the third case, the camera to which the first picture belongs is the first dome camera, the camera to which the second picture belongs is the second dome camera, and the azimuth of the first dome camera is fixed;

In the fourth case, the camera to which the first picture belongs and the camera to which the second picture belongs are the same dome, and the first picture is a picture acquired when the azimuth of the dome is the initial azimuth;

In the fifth case, the camera to which the first picture belongs and the camera to which the second picture belongs are different guns.

Of course, in addition to the above-mentioned various situations, the camera to which the first picture belongs and the camera to which the second picture belongs may also be other situations, which are all reasonable.

It needs to be explained again that the guns mentioned in the above cases include not only cameras with a camera type of guns, but also fish-eye cameras with distortion correction on the captured images, as well as radar and thermal imaging cameras. And other equipment that can provide fixed two-dimensional imaging information.

In addition, the execution order of the above steps S501 and S502 may be to execute step S501 first, then step S502, or execute step S502 first, and then step S501. This is all reasonable.

S503: Determine a region corresponding to the target region in the second screen based on the target region and the target coordinate conversion relationship.

Wherein, in the embodiment of the present application, coordinate mapping between screens is implemented based on the target area and the target coordinate conversion relationship, and the target coordinate conversion relationship is the established coordinate relationship between the camera and the camera. Therefore, in the embodiment of the application, Before executing the above step S503, the controller may only need to determine the camera to which the first picture belongs, the camera to which the second picture belongs, and the first picture to be mapped to the area, or determine the camera to which the first picture belongs, the camera to which the second picture belongs, The first picture of the area mapping and the second picture of the area mapping are both reasonable.

After determining the target area marked in the first picture mapped to the area to be mapped and the target coordinate conversion relationship between the camera to which the first picture belongs and the camera to which the second picture belongs, the controller can then A region corresponding to the target region is determined in the mapped second picture.

It should be noted that, in the above steps S501-S503, the first screen is only used to indicate the screen used for determining the target region to be mapped when the controller completes the area mapping between screens once, and the second screen is only used to indicate When the controller completes the inter-screen area mapping, it needs to determine the screen of the area corresponding to the target area in the screen.

Optionally, in a specific implementation manner, after a picture of an area corresponding to a target area is determined in the second picture, the second picture can be used by the controller to determine the target when completing the next area mapping between pictures. The area-to-area-mapped picture, that is, the second picture can be used as the first picture when the controller completes the next area mapping between pictures, and then the picture of the area corresponding to the target area determined in the second picture can be used as When the controller completes the target area for the next area mapping between pictures, and so on, the area mapping from the first picture to the last picture to be area mapped can be completed.

Optionally, in another specific implementation manner, the controller may determine a plurality of groups of cameras to which the first picture belongs and a camera to which the second picture belongs, and when the foregoing step S502 is performed, the controller may determine the cameras to which the plurality of groups of first pictures belong And the target coordinate conversion relationship corresponding to the camera belonging to the first frame and the camera belonging to the second frame in each group of cameras belonging to the second frame, and further, for each group of target coordinate conversion relationships between the camera to which the first frame belongs and the camera to which the second frame belongs, The above step S503 is performed. That is, the controller may execute the above-mentioned steps S502-S503 multiple times to implement area mapping from the first picture in step S501 to the second pictures to be mapped on the plurality of areas to be mapped. That is, the controller may map the target area determined in the first picture to be area-mapped to a plurality of second pictures. The controller may execute the above steps S502-S503 in parallel for each group of cameras to which the first screen belongs and cameras to which the second screen belongs, or may sequentially perform sequentially for each group of cameras to which the first screen belongs and cameras to which the second screen belongs. The above steps S502-S503.

In the embodiment of the present application, the controller may perform the foregoing step S503 in various ways, so as to determine an area corresponding to the target area in the second picture to be area-mapped, and implement area mapping between the first picture and the second picture. For example, the controller may determine pixel points corresponding to each pixel point in the target area in the second picture, and these determined pixel points may form an area corresponding to the target area in the second picture.

In the embodiment of the present application, the so-called inter-screen area mapping can also be understood as: if the coordinates of a point in a target area are selected in a picture collected by one camera, it can be determined to correspond to the point in a picture collected by another camera The coordinates of the point. Therefore, optionally, in a specific implementation manner, the above step S503 may include the following steps A1-A2:

Step A1: selecting multiple feature points from the target area, where the multiple feature points are multiple pixel points capable of characterizing the target area;

After determining the target area, the controller may determine a plurality of pixels in the target area that can characterize the target area as feature points according to the shape characteristics of the target area.

For example, when the target area is a polygon, the controller may use multiple vertices of the target area as feature points. When the target area is circular, the controller may use the center of the target area and several pixels on the circumference as feature points. When the target area is a curved graphic, the controller may use the points on the edge curve of the target area with obvious changes in curvature as feature points.

Step A2: Determine a region corresponding to the target region in the second screen based on the multiple feature points and the target coordinate conversion relationship.

After determining multiple feature points, the controller can directly determine the area corresponding to the target area in the second picture to be mapped based on the multiple feature points and the target coordinate conversion relationship, thereby realizing the first picture and the second Area mapping between screens.

It can be seen from the above that in the solution provided by the embodiment of the present application, the target coordinate conversion relationship between the camera to which the first picture to be area mapped belongs and the camera to which the second picture to be area mapped belongs, and further, when the After the target area is determined in one picture, the area corresponding to the target area can be determined in the second picture to be area mapped according to the target coordinate conversion relationship and the target area, so that when the target area is given in one picture, , Effectively determine the area corresponding to the target area in another picture.

In addition, the solution provided by the embodiments of the present application can be adapted to different types of cameras, and has a good adaptability. At the same time, the area corresponding to the target area is determined according to the target coordinate conversion relationship. The camera to which a picture belongs and the camera to which the second picture to be mapped belongs are determined. Therefore, the operation of determining the area corresponding to the target area is simple and does not require additional information.

In the following, when the camera to which the first picture belongs and the camera to which the second picture belongs are different in actual application, the foregoing step S502 and step A2 will be specifically described according to the determined camera to which the first picture belongs and the camera to which the second picture belongs.

Embodiment 1 The camera to which the first picture belongs is a gun camera, and the camera to which the second picture belongs is a dome camera;

The above step S502, determining the target coordinate conversion relationship between the camera to which the first picture belongs and the camera to which the second picture belongs, may include the following steps B1-B3:

Step B1: Determine the coordinate conversion relationship between the gun and the dome camera as the first coordinate conversion relationship.

For the gun machine and the dome camera in this embodiment, the controller may determine a coordinate conversion relationship between the gun machine and the dome camera, and use the controller as a first coordinate transformation relationship.

Specifically, the following example illustrates the coordinate conversion relationship between the gun machine and the dome camera: Suppose that the picture collected by the gun machine includes a point A whose coordinates in the picture are (xa, ya). At this time, The azimuth of the dome camera is (Pa, Ta). Through the coordinate conversion relationship between the gun machine and the dome camera, the above coordinates (xa, ya) can be converted into the azimuth angle of the point A corresponding to the dome camera (Pa, Ta). ', Ta'), then rotate the dome camera according to the azimuth angle (Pa ', Ta') until the dome angle of the dome camera changes from (Pa, Ta) to (Pa ', Ta'). At this time, the dome camera collects The center point of the screen is the point A 'corresponding to the above point A.

Step B2: Determine the coordinate conversion relationship corresponding to the dome camera as the second coordinate conversion relationship.

The coordinate conversion relationship corresponding to the dome camera is the conversion relationship between the azimuth of the dome camera and the coordinates of the center point of the screen collected by the dome camera.

For the dome camera in this embodiment, the controller may determine the coordinate conversion relationship corresponding to the dome camera, and use it as the second coordinate conversion relationship.

Specifically, the following example illustrates the coordinate conversion relationship of the dome camera: Suppose that the picture collected by the dome camera includes a point B, which is at a non-center point of the picture, and its coordinates in the picture are (xb, yb). At this time, the azimuth of the dome is (Pb, Tb). Through the coordinate conversion relationship corresponding to the dome, the above coordinates (xb, yb) can be converted into the above point B corresponding to the azimuth of the dome. Angle (Pb ', Tb'), then rotate the dome camera according to the azimuth angle (Pb ', Tb') until the azimuth angle of the dome camera changes from (Pb, Tb) to (Pb ', Tb'), at this time, The center point of the picture collected by the dome camera is the point B ′ corresponding to the above point B.

Step B3: Determine the first coordinate conversion relationship and the second coordinate conversion relationship as the target coordinate conversion relationship between the camera to which the first picture belongs and the camera to which the second picture belongs.

After determining the first coordinate conversion relationship and the second coordinate conversion relationship, the controller may determine the first coordinate conversion relationship and the second coordinate conversion relationship as the target coordinate conversion of the camera to which the first picture belongs and the camera to which the second picture belongs. Relationship, thereby completing step S502 described above.

Furthermore, after determining the target coordinate conversion relationship between the camera to which the first picture belongs and the camera to which the second picture belongs, the controller may determine an area corresponding to the target area in the second picture to be area mapped based on the target coordinate conversion relationship. .

Optionally, in the case of the first embodiment, the camera to which the first picture belongs is a gun machine, and the camera to which the second picture belongs is a dome camera. In the above step A2, based on the conversion relationship between multiple feature points and target coordinates, The area corresponding to the target area in the determination may include the following steps C1-C4:

Step C1: Determine an azimuth corresponding to each feature point based on the first coordinate transformation relationship and the coordinates of the plurality of feature points in the first screen;

It can be understood that when the controller selects multiple feature points from the target area, the coordinates of the feature points in the first screen can be obtained. Furthermore, the controller can convert each feature point through the first coordinate transformation relationship determined above. The coordinates of each feature point in the first picture are converted into the azimuth angle corresponding to the feature point, that is, when the center point of the picture collected by the dome camera is the point corresponding to the feature point, the dome camera's azimuth angle.

For example, the coordinate of the feature point C in the first picture is (xc, yc). Based on the first coordinate transformation relationship, it can be determined that the azimuth corresponding to the feature point C is (Pc, Tc), and the feature point D is in the first picture. The internal coordinates are (xd, yd), and based on the first coordinate conversion relationship, it can be determined that the azimuth angle corresponding to the feature point D is (Pd, Td).

Step C2: selecting a first azimuth angle on which the target device is rotated from the determined azimuth angles, and rotating the target device according to the first azimuth angle to obtain a second picture to be area mapped; wherein the target device belongs to the second picture The camera is the above dome camera;

It should be noted that, in this embodiment, the second picture to be area mapped is determined after the controller executes the above step S502, that is, in this embodiment, the second picture to be area mapped is not The determination is based on any one of the azimuth angles corresponding to the multiple feature points determined by the controller in the above step C1.

The controller may select an azimuth angle from the azimuth angles corresponding to the plurality of feature points determined in the above step C1 as the azimuth angle on which the target device is rotated. Further, the selected azimuth angle may be used as the first azimuth angle, and further , Rotating the dome camera according to the first azimuth angle, and using the picture collected by the dome camera after the rotation as the second picture to be area mapped.

For example, the coordinate of the feature point C in the first picture is (xc, yc), and the azimuth angle corresponding to the feature point C determined by the controller in the above step C1 is (Pc, Tc), and the azimuth angle (Pc, Tc) as the first azimuth, and rotate the dome camera according to the azimuth (Pc, Tc) until the dome of the dome camera becomes (Pc, Tc). The picture collected by the dome camera at this time is used as the first Second picture.

It can be understood that the controller can obtain the above-mentioned first azimuth in any manner for selecting the azimuth. For example, according to the requirements of the shooting angle of the dome camera, the controller selects and presets the azimuths corresponding to the determined multiple feature points. It is reasonable to set the azimuth with the smallest azimuth error as the first azimuth. For example, it is also possible to randomly select an azimuth as the first azimuth among the determined azimuths corresponding to the plurality of feature points.

Step C3: Based on the second coordinate transformation relationship, determine the first target point corresponding to each feature point in the second picture, wherein the first target point corresponding to any feature point is: the corresponding azimuth angle and the feature point Corresponding pixels with the same azimuth;

After performing the above steps C1-C2, the controller can obtain the azimuth corresponding to each feature point and the azimuth corresponding to the dome camera corresponding to the second picture to be area mapped, according to the above description of the second coordinate conversion relationship, The controller may determine the coordinates of points corresponding to the feature points in the second screen based on the second coordinate conversion relationship. Furthermore, a point corresponding to each feature point can be determined in the second picture, and the determined point can be used as a first target point corresponding to the feature point. It can be understood that the azimuth angle corresponding to the first target point corresponding to any feature point is the same as the azimuth angle corresponding to the point determined by the controller in the above step C1.

For example, the coordinates of the feature point C in the first picture are (xc, yc), the azimuth corresponding to the feature point C determined by the controller in the above step C1 is (Pc, Tc), and the feature point D is in the first picture The internal coordinates are (xd, yd), and based on the first coordinate conversion relationship, it can be determined that the azimuth angle corresponding to the feature point D is (Pd, Td). Use the azimuth angle (Pc, Tc) as the first azimuth angle, and rotate the dome camera according to the azimuth angle (Pc, Tc) to obtain a second picture to be area mapped. Then according to the coordinate conversion relationship corresponding to the dome camera, the controller may determine the point D ′ corresponding to the feature point D in the second picture according to the azimuth angle (Pd, Td) corresponding to the feature point D in the second picture. The coordinates are (xd ', yd').

Step C4: Use the area determined on the second screen based on the plurality of first target points as the area corresponding to the target area.

After determining the first target points corresponding to the respective feature points in the second picture, the controller determines an area in the second picture based on the first target points, and further, may be based on the plurality of first targets. The area determined by the point is used as the area corresponding to the target area determined by the controller in step S501.

It should be noted that, as an implementation manner provided in the embodiment of the present application, when the camera to which the first picture belongs is a dome camera, and the camera to which the second picture belongs is a gun machine, in this case, the first embodiment may be adopted. Steps B1 to B3 provided in the step determine the target coordinate conversion relationship between the camera to which the first picture belongs and the camera to which the second picture belongs.

As another implementation manner provided in the embodiment of the present application, when the camera to which the first picture belongs is the first dome camera, the camera to which the second picture belongs is the second dome camera, and the azimuth of the first dome camera is fixed, In this case, an area corresponding to the target area may be determined in the second screen based on the multiple feature points and the target coordinate conversion relationship through steps C1-C4 provided in the first embodiment.

Embodiment 2 The camera to which the first picture belongs is a dome camera, and the camera to which the second picture belongs is a gun machine;

In this embodiment, the method of determining the target coordinate conversion relationship between the camera to which the first picture belongs and the camera to which the second picture belongs may be the same as steps B1-B3 provided in the first embodiment, and details are not described herein again.

Furthermore, after determining the target coordinate conversion relationship between the camera to which the first picture belongs and the camera to which the second picture belongs, the controller may determine an area corresponding to the target area in the second picture to be area mapped based on the target coordinate conversion relationship. .

Optionally, for the case where the camera to which the first picture belongs is a dome camera and the camera to which the second picture belongs is a gun machine in the second embodiment, the above step A2 is based on a plurality of feature points and target coordinate conversion relationships in the second picture. The region corresponding to the target region in the determination may include the following steps D1-D3:

Step D1: Determine the azimuth corresponding to each feature point based on the second coordinate transformation relationship and the coordinates of the plurality of feature points in the first screen.

Wherein, according to the description of the second coordinate conversion relationship in the first embodiment, in the second embodiment, the second coordinate conversion relationship is the conversion relationship between the azimuth of the dome camera and the coordinates of the center point of the screen collected by the dome camera. . In addition, when the controller selects multiple feature points from the target area, the controller can obtain the coordinates of the feature points in the first frame. Therefore, after the second coordinate transformation relationship is determined, the controller can determine the coordinates corresponding to each feature point based on the second coordinate transformation relationship and the coordinates of multiple feature points in the first screen based on the second coordinate transformation relationship. The azimuth angle refers to the azimuth angle of the dome camera for each feature point when the center point of the picture collected by the dome camera is the feature point.

For example, the coordinate of the feature point E in the first picture is (xe, ye). Based on the second coordinate conversion relationship, the controller can determine that the azimuth angle corresponding to the feature point E is (Pe, Te).

Step D2: Determine a second target point corresponding to each feature point in the second picture based on the first coordinate conversion relationship and the determined azimuth corresponding to each feature point.

It can be understood that the second picture is a picture collected by the dome camera.

According to the description of the first coordinate conversion relationship in the foregoing embodiment, in the second embodiment, the first coordinate conversion relationship is a coordinate conversion relationship between a gun machine and a ball machine, that is, for each feature point, the controller The azimuth angle corresponding to the feature point determined in the above step D1 may be converted into the coordinates of the point corresponding to the feature point in the second screen through the first coordinate conversion relationship. Furthermore, a point corresponding to each feature point can be determined in the second picture, and the determined point can be used as a second target point corresponding to the feature point. It can be understood that the azimuth angle corresponding to the second target point corresponding to any feature point is the same as the azimuth angle corresponding to the electrical determined by the controller in the above step D1.

For example, the coordinate of the feature point E in the first picture is (xe, ye). Based on the second coordinate conversion relationship, the azimuth corresponding to the feature point E determined by the controller is (Pe, Te). The azimuth in the second picture is (Pe ', Te'), based on the first coordinate conversion relationship, the controller can determine the feature in the second picture according to the azimuth (Pe, Te) corresponding to the feature point E The coordinates of the point E 'corresponding to the point E are (xe', ye ').

Step D3: Use the area determined on the second screen based on the plurality of second target points as the area corresponding to the target area.

After the second target points corresponding to each feature point are determined in the second picture, the controller determines an area in the second picture based on the second target points, and further, the controller can determine the area based on the plurality of second targets. The area determined by the point is used as the area corresponding to the target area determined by the controller in step S501.

Embodiment 3, the camera to which the first picture belongs is the first dome camera, the camera to which the second picture belongs is the second dome camera, and the azimuth of the first dome camera is fixed;

The above step S502, determining the target coordinate conversion relationship corresponding to the target camera, may include the following steps E1-E3:

Step E1: Determine a coordinate conversion relationship between the first dome camera and the second dome camera as the first coordinate transformation relationship;

For the first dome camera and the second dome camera included in the target camera, the controller may determine a coordinate conversion relationship between the first dome camera and the second dome camera, and use the controller as the first coordinate transformation relationship.

It should be noted that, since the azimuth of the first dome camera is fixed in the third embodiment, the first dome camera can be regarded as a gun machine in the third embodiment.

Specifically, the following example illustrates the coordinate conversion relationship between the first dome camera and the second dome camera: Suppose that the picture collected by the first dome camera includes a point F, and its coordinates in the picture are ( xf, yf). At this time, the azimuth of the dome camera is (Pf, Tf). Through the coordinate conversion relationship between the first dome camera and the second dome camera, the above coordinates (xf, yf) can be converted into the above F. The point corresponds to the azimuth angle (Pf ', Tf') of the dome camera, then the dome camera is rotated according to the azimuth angle (Pf ', Tf') until the azimuth angle of the dome camera changes from (Pf, Tf) to (Pf ', Tf '), at this time, the center point of the picture collected by the dome camera is the point F' corresponding to the above-mentioned F point.

Step E2: Determine the coordinate conversion relationship corresponding to the second dome camera as the second coordinate conversion relationship.

The coordinate conversion relationship corresponding to the second dome camera is: the conversion relationship between the azimuth of the second dome camera and the coordinates of the center point of the picture collected by the second dome camera.

In the above step E2, the method of determining the coordinate conversion relationship corresponding to the second dome camera and using it as the second coordinate conversion relationship may be the same as the above step B2, determining the coordinate conversion relationship corresponding to the dome camera and using it as the second The manner of the coordinate conversion relationship is the same, and is not repeated here.

Step E3: Determine the first coordinate conversion relationship and the second coordinate conversion relationship as the target coordinate conversion relationship between the camera to which the first picture belongs and the camera to which the second picture belongs.

After determining the first coordinate conversion relationship and the second coordinate conversion relationship, the controller may determine the first coordinate conversion relationship and the second coordinate conversion relationship as the target coordinate conversion of the camera to which the first picture belongs and the camera to which the second picture belongs. Relationship, thereby completing step S502 described above.

Furthermore, after determining the target coordinate conversion relationship between the camera to which the first picture belongs and the camera to which the second picture belongs, the controller may determine an area corresponding to the target area in the second picture to be area mapped based on the target coordinate conversion relationship. .

Optionally, for the third embodiment, the camera to which the first picture belongs is the first dome camera, the camera to which the second picture belongs is the second dome camera, and the azimuth of the first dome camera is fixed. The above step A2 is based on multiple The relationship between the feature points and the target coordinate conversion. The way to determine the area corresponding to the target area in the second picture can be the same as that for the first embodiment described above. The camera to which the first picture belongs is a gun and the camera to which the second picture belongs is a dome The steps C1-C4 provided in the situation are the same, and will not be repeated here.

It should be noted that, in the first embodiment, the camera to which the first picture belongs is a gun camera, and the camera to which the second picture belongs is a dome camera, and the target device in the above step C2 is a dome camera to which the second picture belongs; For the third embodiment described above, the camera to which the first picture belongs is the first dome camera, the camera to which the second picture belongs is the second dome camera, and the azimuth of the first dome camera is fixed. The target device in the above step C2 is The second ball machine to which the second screen belongs.

Embodiment 4: The camera to which the first picture belongs and the camera to which the second picture belongs are the same dome, and the first picture is a picture acquired when the azimuth of the dome is the initial azimuth;

The above step S502, determining the target coordinate conversion relationship between the camera to which the first picture belongs and the camera to which the second picture belongs, may include the following steps F1-F2:

Step F1: Determine the coordinate conversion relationship corresponding to the dome camera.

The coordinate conversion relationship corresponding to the dome camera is: the conversion relationship between the azimuth of the dome camera and the coordinates of the center point of the screen collected by the dome camera;

In the foregoing step F1, the manner in which the controller determines the coordinate conversion relationship corresponding to the dome camera may be the same as the manner in which the coordinate conversion relationship corresponding to the dome camera is determined in step B2, and details are not described herein again.

Step F2: Use the coordinate conversion relationship corresponding to the dome camera as the target coordinate conversion relationship between the camera to which the first picture belongs and the camera to which the second picture belongs.

After determining the coordinate conversion relationship corresponding to the dome camera, the controller may determine the coordinate conversion relationship corresponding to the dome camera as the target coordinate conversion relationship of the camera to which the first picture belongs and the camera to which the second picture belongs, thereby completing the above step S502.

Furthermore, after determining the target coordinate conversion relationship between the camera to which the first picture belongs and the camera to which the second picture belongs, the controller may determine an area corresponding to the target area in the second picture to be area mapped based on the target coordinate conversion relationship. .

Optionally, in the fourth embodiment described above, the camera to which the first picture belongs and the camera to which the second picture belongs are the same dome camera, and the first picture is a picture acquired when the azimuth of the dome camera is the initial azimuth angle. The above step A2, based on the multiple feature points and the target coordinate conversion relationship, to determine the area corresponding to the target area in the second screen, may include the following steps G1-G4:

Step G1: Determine the azimuth corresponding to each feature point based on the target coordinate conversion relationship and the coordinates of the plurality of feature points in the first screen;

Wherein, in the fourth embodiment, the target coordinate conversion relationship is the coordinate conversion relationship corresponding to the dome camera, that is, the transformation relationship between the azimuth angle of the dome camera and the coordinates of the center point of the screen collected by the dome camera. In addition, when the controller selects multiple feature points from the target area, the controller can obtain the coordinates of the feature points in the first frame. Further, after determining the target coordinate conversion relationship, based on the target coordinate conversion relationship, the controller can determine the azimuth corresponding to each feature point based on the target coordinate conversion relationship and the coordinates of multiple feature points in the first screen. That is, for each feature point, when the center point of the picture collected by the dome camera is the feature point, the azimuth angle of the dome camera.

For example, the coordinate of the feature point G in the first frame is (xg, yg). Based on the second coordinate conversion relationship, the controller can determine that the azimuth angle corresponding to the feature point G is (Pg, Tg), and the feature point H is at The coordinates in the first picture are (xh, yh). Based on the second coordinate conversion relationship, the controller can determine that the azimuth corresponding to the feature point H is (Ph, Th).

Step G2: Select a second azimuth angle on which the dome camera is based on the determined azimuth angle, and rotate the dome camera according to the second azimuth angle to obtain a second picture to be mapped on the area;

It should be noted that, in this embodiment, the second picture to be area mapped is determined after the controller executes the above step S502, that is, in this embodiment, the second picture to be area mapped is not The determination is based on any one of the azimuth angles corresponding to the multiple feature points determined by the controller in the above step G1.

The controller may select an azimuth angle from the azimuth angles corresponding to the plurality of feature points determined in the above step G1 as the azimuth angle on which the ball dome is based. Further, the selected azimuth angle may be used as the second azimuth angle, and further , Rotate the dome camera according to the second azimuth angle, and use the picture collected by the rotated dome camera as the second picture to be area mapped.

For example, the coordinate of the feature point G in the first screen is (xg, yg), and the azimuth angle corresponding to the feature point G determined by the controller in the above step G1 is (Pg, Tg), and the azimuth angle (Pg, Tg) as the first azimuth, and rotate the dome camera according to the azimuth (Pg, Tg) until the dome of the dome camera becomes (Pg, Tg). The picture collected by the dome camera at this time is used as the first Second picture.

It can be understood that the controller can obtain the above-mentioned first azimuth in any manner for selecting the azimuth. For example, according to the requirements of the shooting angle of the dome camera, the controller selects and presets the azimuths corresponding to the determined multiple feature points. It is reasonable to set the azimuth with the smallest azimuth error as the second azimuth. For example, one azimuth can be randomly selected as the second azimuth from the determined azimuths corresponding to the plurality of feature points.

Step G3: Based on the target coordinate conversion relationship, a third target point corresponding to each feature point is determined in the second screen. The third target point corresponding to any feature point is: the azimuth corresponding to the feature point Pixels with the same angle;

After the above steps G1-G2 are performed, the controller can obtain the azimuth corresponding to each feature point and the azimuth corresponding to the dome camera corresponding to the second screen to be area mapped, and then control according to the above description of the target coordinate conversion relationship The device may determine the coordinates of the points corresponding to each feature point in the second screen based on the target coordinate conversion relationship. Furthermore, a point corresponding to each feature point can be determined in the second picture, and the determined point can be used as a third target point corresponding to the feature point. It can be understood that the azimuth angle corresponding to the third target point corresponding to any feature point is the same as the azimuth angle corresponding to the point determined by the controller in the above step G1.

For example, the coordinates of the feature point G in the first picture are (xg, yg), the azimuth corresponding to the feature point G determined by the controller in the above step G1 is (Pg, Tg), and the feature point H is in the first picture The internal coordinates are (xh, yh), and based on the first coordinate conversion relationship, it can be determined that the azimuth angle corresponding to the feature point H is (Ph, Th). The above azimuth angle (Pg, Tg) is used as the first azimuth angle, and the dome camera is rotated according to the azimuth angle (Pg, Tg) to obtain a second picture to be area mapped. Then according to the coordinate conversion relationship corresponding to the dome camera, the controller may determine the corresponding point H ′ of the feature point H in the second picture according to the azimuth angle (Ph, Th) corresponding to the feature point H in the second picture. The coordinates are (xh ', yh').

Step G4: Use the area determined on the second screen based on the plurality of third target points as the area corresponding to the target area.

After the third target points corresponding to the feature points are determined in the second picture, the controller determines an area in the second screen based on the third target points, and further, the controller can determine the area based on the third targets. The area determined by the point is used as the area corresponding to the target area determined by the controller in step S501.

Embodiment 5: The cameras to which the first picture belongs and the camera to which the second picture belongs are different guns;

The above step S502, determining the target coordinate conversion relationship between the camera to which the first picture belongs and the camera to which the second picture belongs, may include the following steps H1-H2:

Step H1: Determine a coordinate conversion relationship between the different guns described above;

For different guns in this embodiment, the controller may determine a coordinate conversion relationship between the different guns, that is, determine a coordinate conversion relationship of the guns corresponding to the different guns.

Specifically, assuming that the first shot belongs to the first shot and the second shot belongs to the second shot, the following example can be used to explain the coordinate conversion relationship between different shots: Assume the first The picture includes a point I, whose coordinates in the first picture are (xi, yi), and the coordinates (xi, yi) can be converted into the first coordinate through the coordinate conversion relationship between different guns. The coordinates (xi ', yi') of the point I 'corresponding to the point I in the two pictures.

Step H2: Use the determined coordinate conversion relationship between different guns as the target coordinate conversion relationship between the camera to which the first picture belongs and the camera to which the second picture belongs.

After determining the coordinate conversion relationship between the different guns, the controller can determine the coordinate conversion relationship between the guns as the target coordinate conversion relationship of the camera to which the first picture belongs and the camera to which the second picture belongs, thereby completing The above step S502.

Furthermore, after determining the target coordinate conversion relationship between the camera to which the first picture belongs and the camera to which the second picture belongs, the controller may determine an area corresponding to the target area in the second picture to be area mapped based on the target coordinate conversion relationship. .

Optionally, for the case where the camera to which the first picture belongs and the camera to which the second picture belongs are different guns in the fifth embodiment, the above step A2 is based on a plurality of feature points and target coordinate conversion relationships in the second picture. Determining the area corresponding to the target area may include the following steps I1-I2:

Step I1: Determine a fourth target point corresponding to each feature point in the second picture based on the target coordinate conversion relationship and the coordinates of the multiple feature points in the first picture;

Among them, in the fifth embodiment, the target coordinate conversion relationship is a coordinate conversion relationship between different guns. In addition, when the controller selects multiple feature points from the target area, the controller can obtain the coordinates of the feature points in the first frame. Therefore, after determining the target coordinate conversion relationship, the controller can determine the coordinates of the points corresponding to each feature point in the second screen based on the target coordinate conversion relationship and the coordinates of the plurality of feature points in the first screen. Furthermore, a point corresponding to each feature point can be determined in the second screen, and the determined point can be used as a fourth target point corresponding to the feature point.

Step I2: Use the area determined on the second screen based on the fourth target points as the area corresponding to the target area.

After the third target point corresponding to each feature point is determined in the second screen, the controller determines an area in the second screen based on the fourth target points, and further, the controller can determine the area based on the fourth targets. The area determined by the point is used as the area corresponding to the target area determined by the controller in step S501.

It should be noted that, in the specific implementation manners of the first embodiment to the fifth embodiment, multiple coordinate conversion relationships are involved. The multiple coordinate conversion relationships mentioned above will be described below.

FIG. 6 is a schematic flowchart of a manner for determining a coordinate conversion relationship between the gun machine and the ball machine according to an implementation manner provided in an embodiment of the present application.

Among them, considering the simplicity of the determination method of the first coordinate conversion relationship, a planar homography matrix mapping method is adopted in the embodiment of the present application, so that the three-dimensional space of the above-mentioned gun machine and the above-mentioned ball machine may not be considered. Position, and points in three-dimensional space, but can directly establish a mathematical model to establish the coordinate conversion relationship between the gun machine and the ball machine.

Specifically, as shown in FIG. 6, the manner of determining the coordinate conversion relationship between the gun machine and the ball machine may include the following steps 601-604:

S601: Determine the coordinates corresponding to the preset m reference points in the frame collected by the gun machine, and obtain m first coordinates.

The reference point is: a point in the three-dimensional space that can be captured by the camera to which the first picture belongs and the camera to which the second picture belongs, m≥4;

The controller can obtain m reference points in the three-dimensional space that can be captured by the camera to which the first picture belongs and the camera to which the second picture belongs. In order to ensure that the homography matrix can be calculated, m≥4 must be guaranteed. For example, the reference point may be the four vertices of a photo frame that can be captured by the camera to which the first picture belongs and the camera to which the second picture belongs. To the eight vertices of a rectangular box, etc., this is all reasonable.

It should be noted that the controller can obtain the m reference points in various ways. For example, after receiving the reference point information sent by the user, the user can select m reference points in the three-dimensional space that can be captured by the camera to which the first screen belongs and the camera to which the second screen belongs, and then pass the information of these reference points through related instructions Send to the controller;

For another example, the controller may select m reference points in the three-dimensional space that can be captured by the camera to which the first picture belongs and the camera to which the second picture belongs according to a preset rule. For example, when the first picture belongs to the camera and the second picture When the three-dimensional space captured by the camera to which the picture belongs includes a certain thing or a certain type of specified object, the controller may select m reference points on the object.

For example, if the three-dimensional space captured by the camera to which the first picture belongs and the camera to which the second picture belongs is a room with a fire extinguisher, the controller may select m reference points on the fire extinguisher.

It should be noted that the controller can also obtain m reference points in other ways, which is not specifically limited in this embodiment of the present application.

After obtaining the m reference points in the three-dimensional space that can be taken by the camera to which the first picture belongs and the camera to which the second picture belongs, the controller can determine the corresponding reference points of the above m reference points in the picture collected by the gun. And use it as the first coordinate, that is, the controller can obtain m first coordinates.

It should be noted that, in the above step S601, the controller may obtain the above-mentioned m first coordinates in multiple manners, which is not specifically limited in this embodiment of the present application.

It can be understood that the picture collected by the aforementioned gun is a two-dimensional plane picture, and the points in the picture have two-dimensional plane coordinates, which can be represented by (x, y).

S602: For each reference point, determine the azimuth angle when the dome camera uses the reference point as the center of the screen, and obtain m azimuth angles.

Correspondingly, after obtaining m reference points in the three-dimensional space that can be captured by the camera to which the first picture belongs and the camera to which the second picture belongs, for each reference point, by turning the dome camera, the controller can determine The azimuth of the second camera when the center point of the picture collected by the dome camera is the point corresponding to the reference point.

That is, after obtaining m reference points in the three-dimensional space that can be captured by the camera to which the first picture belongs and the camera to which the second picture belongs, for each reference point, the controller can determine that the second camera uses the reference point as the center of the picture To obtain m azimuth angles, each azimuth angle is composed of a horizontal azimuth angle P and a vertical azimuth angle T, expressed as (P, T).

It should be noted that the execution sequence of the above steps S601 and S602 may be that the controller first executes the above step S601 and then executes the above step S602, or the controller first executes the above device S602 and then the above step S601, or The controller executes the above steps S601 and S602 simultaneously.

S603: Convert m azimuths into m second coordinates;

Obviously, in step S601, each first coordinate obtained by the controller is a two-dimensional plane coordinate, and in step S602, each azimuth angle obtained by the controller obviously cannot be used as a two-dimensional plane coordinate. Therefore, The controller needs to convert each azimuth obtained in step S602 into a two-dimensional plane coordinate corresponding to the azimuth.

Specifically, the controller can obtain a reference height value h in advance, so that the above azimuth angle can be converted to a two-dimensional plane coordinate by the following formula.

x _i = h / tan (T _i ) * sin (P _i )

y _i = h / tan (T _i ) * cos (P _i )

Among them, T _i is the vertical azimuth of the i-th azimuth, P _i is the horizontal azimuth of the i-th azimuth, and x _i is the x value in the two-dimensional plane coordinates corresponding to the calculated i-th azimuth, y ⁱ is the y value in the two-dimensional plane coordinate corresponding to the calculated ith azimuth angle, i ≦ m.

Specifically, the reference height h may take a value of 1.

S604: Calculate the homography matrix based on the m first coordinates and the m second coordinates, and use the homography matrix as the coordinate conversion relationship between the gun machine and the ball machine.

After obtaining the above m first coordinates and m second coordinates, the controller can calculate a homography matrix based on the m first coordinates and m second coordinates, and use the homography matrix as a gun machine and Coordinate conversion relationship between dome cameras.

It should be noted that in the calculation process, in order to facilitate the distinction between the first and second coordinates, the m first coordinates are expressed as (x _i , y _i ), and the m second coordinates are expressed as (u _i , v _i ), i≤m.

Optionally, in a specific implementation manner, based on m first coordinates and m second coordinates in step S604, the homography matrix can be calculated by the following formula (1) to formula (6), where:

D = H × S (1)

among them,

Among them, H is the homography matrix to be calculated.

Furthermore, through the above formula (1), the following formula (2) can be obtained:

In order to obtain the homography matrix, it is necessary to solve the equation set shown in the following formula (3) transformed from the above formula (2), so that each element in the homography matrix H can be obtained by calculating the value of each element in h And further obtain the homography matrix H:

h = (H ₁₁ , H ₁₂ , H ₁₃ , H ₂₁ , H ₂₂ , H ₂₃ , H ₃₁ , H ₃₂ , H ₃₃ ) ^Τ (3)

Based on the above m first coordinates and m second coordinates, construct two vectors a _{x, u} and a _{y, v} as shown below, where

a _{x, u} = (-x _i , -y _i , -1,0,0,0, u _i x _i , u _i y _i , u _i ) ^Τ

a _{y, v} = (0,0,0, -x _i , -y _i , -1, v _i x _i , v _i y _i , v _i ) ^Τ

Furthermore, by combining the vectors a _{x, u} and the vectors a _{y, v} , a new matrix can be obtained as shown in formula (4):

In this way, the following formula (5) can be solved to obtain the values of each element in h, and then the values of each element in the homography matrix H, and then the homography matrix H:

Ah = 0 (5)

Wherein, when solving the above formula (5), SVD (Singular Value Decomposition) can be performed on the above matrix A. The specific SVD process is as follows:

From the above formula (6), the right singular value and left singular vector of A can be obtained. Σ, V are in a corresponding relationship, and the values in Σ are sorted from large to small. The right singular vector corresponding to the minimum value in Σ is h. The approximate solution of H is the approximate solution of h, as shown in Equation 7 below.

[U, Σ, V] = svd (A) (6)

h = V [[min (∑)] ,:] (7)

In this way, the value of each element in h can be obtained. At this time, h is a 1 * 9 vector, and the value of each element in h corresponds to the value of one element in the homography matrix H. The calculated h is readjusted by the number of rows, columns, and dimensions to obtain the homography matrix H.

It should be noted that according to the description of the third embodiment, it can be understood that when the camera to which the first picture belongs is the first dome camera, the camera to which the second picture belongs is the second dome camera, and the azimuth of the first dome camera is fixed , The first dome camera can be regarded as a gun machine, so that the method for determining the coordinate conversion relationship between the first dome camera and the second dome camera involved in the above third embodiment can be determined with the aforementioned dome camera and dome camera. The coordinate transformation relationship is established in the same way, so it will not be described again.

The following describes the determination method of the coordinate conversion relationship corresponding to the ball machine.

Specifically, the manner of determining the coordinate conversion relationship corresponding to the dome camera described above can be described according to the schematic diagram of the dome camera imaging in FIG. 7.

Among them, as shown in FIG. 7, the uv plane is the imaging plane of the dome camera and is tangent to the lens of the dome camera at point C, and the point C is the center point of the uv plane. The range can be approximated to the imaging plane, and further, the point corresponding to point C in the picture collected by the dome camera is the center point of the picture collected by the dome camera. It should be noted that during the process of collecting pictures by the dome camera, the imaging plane uv plane of the dome camera does not really exist, but is a hypothetical plane used to determine the coordinate conversion relationship corresponding to the dome camera.

Specifically, the manner of determining the coordinate conversion relationship corresponding to the ball machine may include:

Step J1: Obtain the coordinates of the points on the dome camera's imaging point preset in the imaging plane of the dome camera as the first calibration coordinates.

As shown in FIG. 7, the point P can be used as a preset calibration point in the uv plane, and then the controller can obtain the coordinates of the point corresponding to the point P in the screen collected by the dome camera, and place the coordinate at The first calibration coordinates are specified. Specifically, the first calibration coordinates can be expressed as (xp, yp).

It should be noted that in the above step J1, the controller can obtain a preset calibration point in various ways. For example, the user selects a point in the imaging plane of the dome camera as a preset calibration point, and selects the selected calibration point. The information of the calibration point is sent to the controller and the like; further, the controller can obtain the coordinates of the point corresponding to the calibration point in the screen collected by the dome camera in various ways. In the embodiment of the present application, the specific implementation manner of the above step J1 is not limited.

Step J2: Determine the optical axis length of the dome camera based on the resolution, magnification, and azimuth when the dome camera collects the picture for determining the first calibration coordinate.

The optical axis of the dome camera is the line connecting the origin of the camera coordinate system of the dome camera and the center point of the imaging plane. For the convenience of description, the resolution, magnification, and azimuth when the dome camera collects the picture for determining the first calibration coordinate may be simply referred to as the current resolution of the dome camera, the current magnification of the dome camera, and the current azimuth of the dome camera.

Specifically, as shown in FIG. 7, the straight segment OC is the optical axis of the dome camera as shown in FIG. 7. The length of the OC can be calculated by the following formula, that is, the optical axis length of the dome camera in FIG. 7:

R = width / 2.0 / tan ( P _current / 2) / Z _Current

Wherein, R is the length of the optical axis of the dome, width of the resolution in the dome of the current resolution in the width direction, P is the _current horizontal azimuth dome current azimuth, Z is _currently dome current ratio.

Step J3: Based on the first calibration coordinates and the current resolution of the dome camera, determine the coordinates of the calibration point in the imaging plane of the dome camera as the second calibration coordinates.

The above-mentioned second calibration coordinates can be expressed as (U, V). Specifically, the above-mentioned second calibration coordinates can be calculated by the following formula:

U = xp-width / 2.0

V = yp-height / 2.0

Among them, width is the resolution in the width direction in the current resolution of the dome camera, and height is the resolution in the height direction in the current resolution of the dome camera.

Step J4: Determine the coordinates of the calibration point in the camera coordinate system of the dome camera as the third calibration coordinates based on the second calibration coordinates and the optical axis length of the dome camera.

Specifically, the third calibration coordinate may be expressed as (x0, y0, z0). Specifically, the third calibration coordinate may be calculated by the following formula:

x0 = U

y0 = R * cosT _current- V * sinT _current

z0 = R * sinT _{current current} + V * cosT

Wherein T is _currently azimuth dome current azimuth angle in the vertical direction.

Step J5: Based on the third calibration coordinates and the current azimuth angle of the dome camera, determine an angular offset of the calibration point in the horizontal direction and an angular deviation in the vertical direction with respect to the optical axis of the dome camera.

The angular offset in the horizontal direction is the horizontal azimuth in the azimuth corresponding to the calibration point, which can be expressed as P_new, and the angular offset in the vertical direction is the vertical azimuth in the azimuth corresponding to the calibration. The angle can be expressed as T_new, so that the azimuth corresponding to the calibration point can be expressed as (P_new, T_new). Specifically, the azimuth corresponding to the calibration points can be calculated by the following formula:

P_new = atan (x0 / y0) + P _current ;

T_new = atan (z0 / (sqrt (x0 * x0 + y0 * y0))) + T _Current

Obviously, the mathematical relationship between the coordinates of the corresponding points in the picture collected by the dome camera and the azimuth angles corresponding to the calibration points can be established by explaining the above steps J1-Step J5 and each formula. Further, since the azimuth corresponding to the calibration point is the azimuth of the dome camera when the center point of the picture collected by the dome camera is the point corresponding to the calibration point.

Therefore, given the coordinates of any pixel point in the picture collected by the dome camera and the azimuth angle when the dome camera collected the picture, the coordinate conversion relationship corresponding to the dome camera can be established through the above steps J1-J5 to obtain When the center point of the picture collected by the dome camera is the point corresponding to the pixel point, the azimuth angle of the dome camera. Furthermore, it can be understood that when the center point of the picture collected by the dome camera is a point corresponding to a certain pixel point, the azimuth angle of the dome camera and the azimuth angle of the dome camera's current captured picture can also be calculated. The coordinates of the point in the picture currently captured by the dome camera.

It should be noted that, in the embodiment of the present application, the coordinate conversion relationship corresponding to the second dome camera mentioned in the third embodiment is the azimuth of the second dome camera and the center point of the screen collected by the second dome camera. Therefore, the coordinate conversion relationship corresponding to the second dome camera and the coordinate conversion relationship corresponding to the above dome camera are both the coordinate transformation relationship between the azimuth of the dome camera and the center point of the screen collected by the dome camera. Coordinate mapping between different pictures collected when the dome camera is at different azimuth angles can be realized.

Therefore, the determination method of the coordinate conversion relationship corresponding to the second dome camera mentioned in the third embodiment may be the same as the determination method of the coordinate conversion relationship corresponding to the above dome camera, and details are not described herein again.

The following describes the determination manner of the coordinate conversion relationship between different guns provided in the foregoing embodiments of the present application. In the same manner as the above-mentioned first coordinate conversion relationship, a planar homography matrix mapping method is also used in the embodiments of the present application to determine the coordinate conversion relationship between the guns, so that different guns can be ignored. The position in the three-dimensional space and the point in the three-dimensional space can be established by directly establishing a mathematical model to establish the coordinate conversion relationship between different guns.

Specifically, assuming that the different guns include the first gun and the second gun, the method for determining the coordinate conversion relationship between the different guns may include:

Step K1: Determine the corresponding coordinates of the preset n target points in the picture collected by the first gun machine, obtain n third coordinates, and determine the corresponding of the preset n target points in the picture collected by the second gun machine. Coordinates to get n fourth coordinates, where n≥4;

It should be noted that, in step K1, the target point is a reference point with the same property as the reference point in step S601. That is, the controller obtains the reference points in the three-dimensional space that can be captured by the first and second guns in advance, and in order to ensure that the homography matrix can be calculated, n≥4 must be guaranteed. The manner in which the controller obtains the n target points may be the same as the manner in which the controller obtains m reference points in step S601, and details are not described herein again.

After obtaining the n target points in the three-dimensional space that can be captured by the first and second guns, the controller can determine the positions of each of the n target points in the picture collected by the first gun. The corresponding coordinates are used as the third coordinates, and the coordinates corresponding to each of the above n target points are determined in the picture collected by the second gun machine, and the preset n target points are determined to be collected by the second gun machine The corresponding coordinates in the picture of the image as the fourth coordinate. Therefore, the controller can obtain n third coordinates and n fourth coordinates.

It should be noted that the manner in which the controller obtains n third coordinates and n fourth coordinates in the above step K1 may be the same as the manner in which the controller obtains m first coordinates in the above step S601, and is not repeated here. To repeat.

Step K2: Calculate the target homography matrix based on the n third coordinates and n fourth coordinates, and use the target homography matrix as the coordinate conversion relationship between the guns.

After obtaining the n third coordinates and n fourth coordinates, the controller can calculate a target homography matrix based on the n third coordinates and n fourth coordinates, and use the target homography matrix as Coordinate conversion relationship between guns.

It should be noted that the controller executes the above step K2, calculates the target homography matrix based on the n third coordinates and the n fourth coordinates, and uses the target homography matrix as a method of coordinate conversion relationship between the guns. The method is the same as that when the controller executes the above step S604 and calculates the homography matrix based on the m first coordinates and the m second coordinates, and uses the homography matrix as the first coordinate conversion relationship, which is not described herein again.

Further, on the basis of the above-mentioned area mapping between screens, the embodiments of the present application can further perform screen processing on the regions determined in different screens to be mapped, for example, adding AR tags to these regions for labeling, or Privacy masking is performed on these areas, or focused display is performed on these areas. Of course, other screen processing may also be performed, which is not specifically limited in this embodiment of the present application.

Specifically, the so-called adding an AR tag for labeling may refer to designating a relevant area on a screen captured by one camera, and displaying a region corresponding to the relevant area on a screen captured by another camera.

The so-called privacy masking can refer to designating a region in a frame captured by a camera and masking the region, and then, in other frames, the region corresponding to the region is also masked. Generally, since the shooting angle of the dome camera can be changed, the method of mapping between regions of the picture provided in the embodiment of the present application can ensure that during the rotation of the dome camera, the relevant area in the picture collected by the dome camera at any shooting angle is performed. Shelter.

Corresponding to the method for mapping regions between screens provided in the embodiments of the present application, the embodiment of this application also provides a device for mapping regions between screens.

FIG. 8 is a schematic structural diagram of an inter-screen region mapping device according to an embodiment of the present application. As shown in FIG. 8, the inter-screen region mapping device may include the following modules:

A first area determination module 810, configured to determine a target area specified in a first picture to be area-mapped;

A conversion relationship determining module 820, configured to determine a target coordinate conversion relationship between a camera to which a first picture belongs and a camera to which a second picture to be mapped belongs;

The second region determining module 830 is configured to determine a region corresponding to the target region in the second picture based on the target region and the target coordinate conversion relationship.

It can be seen from the above that in the solution provided by the embodiment of the present application, the target coordinate conversion relationship between the camera to which the first picture to be area mapped belongs and the camera to which the second picture to be area mapped belongs, and further, when the After the target area is determined in one picture, the area corresponding to the target area can be determined in the second picture to be area mapped according to the target coordinate conversion relationship and the target area, so that when the target area is given in one picture, , Effectively determine the area corresponding to the target area in another picture. In addition, the solution provided by the embodiments of the present application can be adapted to different types of cameras, and has a good adaptability. At the same time, the area corresponding to the target area is determined according to the target coordinate conversion relationship. The camera to which a picture belongs and the camera to which the second picture to be mapped belongs are determined. Therefore, the operation of determining the area corresponding to the target area is simple and does not require additional information.

Optionally, in a specific implementation manner, the foregoing second area determination module 830 may include:

Feature point selection sub-module, for selecting multiple feature points from the target area, where the multiple feature points are multiple pixel points capable of characterizing the target area;

The second region determining submodule is configured to determine a region corresponding to the target region in the second picture based on a plurality of feature points and a target coordinate conversion relationship.

Optionally, in a specific implementation manner, the camera to which the first picture belongs is a dome camera, and the camera to which the second picture belongs is a dome camera; or, the camera to which the first picture belongs is a dome camera, and the camera to which the second picture belongs is a dome camera; The above conversion relationship determination module 820 may include:

A first relationship determining submodule, configured to determine a coordinate conversion relationship between a gun machine and a dome camera, as a first coordinate conversion relationship;

A second relationship determination sub-module is used to determine a coordinate conversion relationship corresponding to the dome camera as a second coordinate conversion relationship, wherein the coordinate conversion relationship corresponding to the dome camera is: the azimuth angle of the dome camera and the center point of the picture collected by the dome camera The transformation relationship of the coordinates;

The first target relationship determining submodule is configured to determine the first coordinate conversion relationship and the second coordinate conversion relationship as a target coordinate conversion relationship between a camera to which the first picture belongs and a camera to which the second picture to be mapped belongs.

Optionally, in a specific implementation manner, the camera to which the first picture belongs is a first dome camera, the camera to which the second picture belongs is a second dome camera, and the azimuth of the first dome camera is fixed; then the above-mentioned conversion relationship determination module 820 may include:

A third relationship determining submodule, configured to determine a coordinate conversion relationship between the first dome camera and the second dome camera, as a first coordinate transformation relationship;

A fourth relationship determining sub-module is used to determine a coordinate transformation relationship corresponding to the second dome camera as a second coordinate transformation relationship, wherein the coordinate transformation relationship corresponding to the second dome camera is: the azimuth of the second dome camera and the second dome camera The transformation relationship of the coordinates of the center point of the picture collected by the dome camera;

A second target relationship determining sub-module is configured to determine the first coordinate conversion relationship and the second coordinate conversion relationship as a target coordinate conversion relationship between a camera to which the first picture belongs and a camera to which the second picture to be mapped belongs.

Optionally, in a specific implementation manner, the camera to which the first picture belongs is a gun camera, and the camera to which the second picture belongs is a dome camera; or, the camera to which the first picture belongs is a first dome camera, and the camera to which the second picture belongs is a second camera. Dome camera, the azimuth of the first dome camera is fixed; then the above-mentioned second area determination submodule may include:

A first azimuth determining unit, configured to determine an azimuth corresponding to each feature point based on a first coordinate transformation relationship and coordinates of a plurality of feature points in a first picture;

A first picture obtaining unit, configured to select a first azimuth on which the rotation target device is based from the determined azimuth, and rotate the target device according to the first azimuth to obtain a second picture; wherein when the camera to which the first picture belongs belongs Is a dome camera, when the camera to which the second picture belongs is a dome camera, the target device is a dome camera; when the camera to which the first picture belongs is a dome camera, the camera to which the second picture belongs is a second dome camera, and the azimuth of the first dome camera When fixed, the target device is the second dome camera;

A first target point determining unit, configured to determine a first target point corresponding to each feature point in the second picture based on a second coordinate transformation relationship, wherein the first target point corresponding to any feature point is: Pixels with the same azimuth as the azimuth corresponding to the feature point;

The first region determining unit is configured to use a region determined based on a plurality of first target points in the second picture as a region corresponding to the target region.

Optionally, in a specific implementation manner, the camera to which the first picture belongs is a dome camera, and the camera to which the second picture belongs is a gun machine; then the above-mentioned second area determination submodule may include:

A second azimuth angle determining unit, configured to determine an azimuth angle corresponding to each feature point based on the second coordinate transformation relationship and the coordinates of multiple feature points in the first picture;

A second target point determining unit, configured to determine a second target point corresponding to each feature point in the second picture based on the first coordinate conversion relationship and the determined azimuth corresponding to each feature point;

The second region determining unit is configured to use, as the region corresponding to the target region, a region determined based on the plurality of second target points in the second picture.

Optionally, a specific implementation manner is that the inter-screen area mapping device may further include a coordinate conversion relationship determining module for determining a coordinate conversion relationship between a gun and a dome camera, and the coordinate conversion relationship determining module may include:

The first coordinate determination submodule is used to determine the preset coordinates corresponding to the m reference points in the picture collected by the gun machine to obtain m first coordinates, where the reference points are: the camera to which the first picture belongs and the second picture Points in the three-dimensional space that can be captured by the camera, m≥4;

The azimuth determination sub-module is used to determine, for each reference point, the azimuth when the dome camera uses the reference point as the center of the screen, and obtain m azimuth angles;

A second coordinate determination submodule, configured to convert m azimuth angles into m second coordinates;

The coordinate conversion relationship determination sub-module is configured to calculate a homography matrix based on m first coordinates and m second coordinates, and use the homography matrix as a first coordinate transformation relationship.

Optionally, in a specific implementation manner, the camera to which the first picture belongs and the camera to which the second picture belongs are the same dome camera, and the first picture is a picture acquired when the azimuth of the dome camera is the initial azimuth; The determination module 820 may include:

A fifth relationship determining sub-module is used to determine a coordinate conversion relationship corresponding to the dome camera, wherein the coordinate conversion relationship corresponding to the dome camera is: a transformation relationship between the azimuth angle of the dome camera and the coordinates of the center point of the screen collected by the dome camera;

A third target relationship determining submodule is configured to use the coordinate conversion relationship corresponding to the dome camera as the target coordinate conversion relationship of the camera to which the first picture belongs and the camera to which the second picture to be mapped belongs.

Furthermore, in this implementation manner, the above-mentioned second area determination submodule may include:

A third-party position angle determining unit, configured to determine an azimuth angle corresponding to each feature point based on a target coordinate conversion relationship and coordinates of multiple feature points in the first screen;

A second picture obtaining unit, configured to select, from the determined azimuth angles, a second azimuth angle on which the dome camera is based, and rotate the dome camera according to the second azimuth angle to obtain a second picture;

A third coordinate point determining unit, configured to determine a third target point corresponding to each feature point in the second picture based on the target coordinate conversion relationship, and the third target point corresponding to any feature point is: the corresponding azimuth angle and The feature points correspond to pixels with the same azimuth;

The third region determining unit is configured to use, as the region corresponding to the target region, the region determined based on the plurality of third target points in the second picture.

Optionally, in a specific implementation manner, the camera to which the first picture belongs and the camera to which the second picture belongs are different guns; the conversion relationship determining module 820 may include:

The sixth relationship determining sub-module is used to determine the coordinate conversion relationship between different guns;

A fourth target relationship determination submodule is configured to use the determined coordinate conversion relationship between different guns as a target coordinate conversion relationship between a camera to which the first picture belongs and a camera to which the second picture to be mapped belongs.

Furthermore, in this implementation manner, the above-mentioned second area determination submodule may include:

A fourth target point determining unit, configured to determine a fourth target point corresponding to each feature point in the second picture based on the target coordinate conversion relationship and the coordinates of the multiple feature points in the first picture;

The fourth area determining unit is configured to use, as the area corresponding to the target area, the area determined based on the plurality of fourth target points in the second picture.

Corresponding to the foregoing method for mapping regions between pictures provided by the embodiment of the present application, the embodiment of the present application further provides a multi-camera observation system.

FIG. 10 is a schematic structural diagram of a multi-camera observation system according to an embodiment of the present application. As shown in FIG. 10, the system may include a controller 110 and at least one camera 120, where the at least one camera 120 may include a region to be mapped. The camera to which the first picture belongs and the camera to which the second picture to be mapped belongs;

The camera to which the first picture belongs can be used to collect the first picture to be mapped in the area to be mapped;

The camera to which the second picture belongs can be used to collect the second picture to be mapped in the area to be mapped;

The controller 110 may be configured to: determine a target area specified in the first frame; determine a target coordinate conversion relationship between a camera to which the first frame belongs and a camera to which the second frame belongs; based on the target area and the target coordinate conversion relationship, in the second frame The area corresponding to the target area is determined in the screen.

Optionally, in a specific implementation manner, the foregoing controller 110 may also be used for:

Select multiple feature points from the target area; determine the area corresponding to the target area in the second screen based on the multiple feature points and the target coordinate conversion relationship; where the multiple feature points are multiple pixel points capable of characterizing the target area .

Optionally, in a specific implementation manner, the at least one camera 120 may include a gun machine and a dome camera, and the camera to which the first picture belongs is a gun machine, and the camera to which the second picture belongs is a dome camera; or A camera 120 may include a shooter and a dome camera, and the camera to which the first picture belongs is a dome camera, and the camera to which the second picture belongs is a gun machine;

Then, in this implementation manner, the controller 110 may be configured to: determine a coordinate conversion relationship between a gun and a dome camera as a first coordinate conversion relationship; determine a coordinate conversion relationship corresponding to the dome camera as a second coordinate conversion relationship Determining the first coordinate conversion relationship and the second coordinate conversion relationship as the target coordinate conversion relationship between the camera to which the first screen belongs and the camera to which the second screen to be mapped belongs;

The coordinate conversion relationship corresponding to the dome camera is the conversion relationship between the azimuth of the dome camera and the coordinates of the center point of the screen collected by the dome camera.

Optionally, in a specific implementation manner, the at least one camera 120 may include a first dome camera and a second dome camera, and the camera to which the first picture belongs is the first dome camera, and the camera to which the second picture belongs is the second dome camera. Machine, the azimuth of the first ball machine is fixed;

Then, in this implementation manner, the controller 110 may be configured to: determine a coordinate conversion relationship between the first dome camera and a second dome camera as the first coordinate transformation relationship; determine a coordinate transformation relationship corresponding to the second dome camera, As the second coordinate conversion relationship; determining the first coordinate conversion relationship and the second coordinate conversion relationship as the target coordinate conversion relationship of the camera to which the first screen belongs and the camera to which the second screen to be mapped belongs;

The coordinate conversion relationship corresponding to the second dome camera is: the conversion relationship between the azimuth of the second dome camera and the coordinates of the center point of the picture collected by the second dome camera.

Optionally, in a specific implementation manner, the at least one camera 120 may include a gun machine and a dome camera, and the camera to which the first picture belongs is a gun machine, and the camera to which the second picture belongs is a dome camera; or A camera 120 may include a first dome camera and a second dome camera, and the camera to which the first picture belongs is the first dome camera, the camera to which the second picture belongs is the second dome camera, and the azimuth of the first dome camera is fixed;

In this implementation manner, the controller 110 is configured to: determine an azimuth angle corresponding to each feature point based on the first coordinate transformation relationship and the coordinates of multiple feature points in the first screen; and select from the determined azimuth angles Rotate the first azimuth on which the target device is based, and rotate the target device according to the first azimuth to obtain a second picture.

Wherein, when the camera to which the first picture belongs is a dome camera, and the camera to which the second picture belongs is a dome camera, the target device is a dome camera; when the camera to which the first picture belongs is a first dome camera, and the camera to which the second picture belongs is a second dome camera. When the azimuth of the first dome camera is fixed, the target device is the second dome camera; based on the second coordinate transformation relationship, the first target point corresponding to each feature point is determined in the second picture, where any feature point is The corresponding first target point is: a pixel point whose corresponding azimuth angle is the same as the azimuth angle corresponding to the feature point; and an area determined based on the plurality of first target points in the second picture is used as the area corresponding to the target area.

Optionally, in a specific implementation manner, the at least one camera 120 may include a gun machine and a dome camera, and the camera to which the first picture belongs is a dome camera, and the camera to which the second picture belongs is a gun machine;

In this implementation manner, the controller 110 may be configured to determine an azimuth corresponding to each feature point based on the second coordinate transformation relationship and the coordinates of multiple feature points in the first screen; based on the first coordinate transformation relationship and The determined azimuth corresponding to each feature point determines the second target point corresponding to each feature point in the second picture; and the area determined based on the plurality of second target points in the second picture is used as the target area corresponding Area.

Optionally, in a specific implementation manner, the at least one camera 120 may include a dome camera, and the camera to which the first picture belongs and the camera to which the second picture belongs are dome cameras, and the first picture is the azimuth of the dome camera. The picture collected at the initial azimuth;

In this implementation manner, the controller 110 may be configured to: determine the coordinate conversion relationship corresponding to the dome camera; use the coordinate conversion relationship corresponding to the dome camera as the target of the camera to which the first screen belongs and the camera to which the second screen to be mapped belongs. Coordinate conversion relationship;

In this implementation manner, after the controller 110 determines the target coordinate conversion relationship between the camera to which the first screen belongs and the camera to which the second screen to be mapped belongs, it can also be used to: based on the target coordinate conversion relationship and multiple feature points in the The coordinates in the first frame determine the azimuth angle corresponding to each feature point; select the second azimuth angle on which the dome camera is based on the determined azimuth angle, and rotate the dome camera according to the second azimuth angle to obtain the second picture; Based on the target coordinate conversion relationship, a third target point corresponding to each feature point is determined in the second picture; and an area determined based on the plurality of third target points in the second picture is used as a region corresponding to the target area.

Among them, the coordinate conversion relationship corresponding to the dome camera is: the conversion relationship between the dome camera's azimuth angle and the coordinates of the center point of the screen collected by the dome camera; the third target point corresponding to any feature point is: the corresponding azimuth Pixel points with the same azimuth angle as the feature points.

Optionally, in a specific implementation manner, the at least one camera 120 may include: different guns, and the cameras to which the first picture belongs and the cameras to which the second picture belongs are different guns;

In this implementation manner, the controller 110 may be configured to: determine the coordinate conversion relationship between different guns; and use the determined coordinate conversion relationship between the different guns as the camera and the target to which the first screen belongs. The target coordinate conversion relationship of the camera to which the second picture of the area map belongs;

In this implementation manner, after the controller 110 determines the target coordinate conversion relationship between the camera to which the first screen belongs and the camera to which the second screen belongs to be mapped, it can also be used to: based on the target coordinate conversion relationship and multiple feature points in the The coordinates in the first picture determine the fourth target point corresponding to each feature point in the second picture; and use the area determined based on the plurality of fourth target points in the second picture as the area corresponding to the target area.

It should be noted that, for a detailed description of each step performed by the controller 110, reference may be made to corresponding content of the foregoing method embodiment, and details are not described herein.

An embodiment of the present application further provides an electronic device. The electronic device is a controller in a multi-camera observation system. As shown in FIG. 9, the electronic device includes a processor 901, a communication interface 902, a memory 903, and a communication bus 904. Among them, the processor 901, the communication interface 902, and the memory 903 complete communication with each other through the communication bus 904.

The memory 903 is configured to store a computer program;

The processor 901 is configured to, when executing a program stored in the memory 903, implement any of the method steps in the method for mapping regions between screens provided by the embodiments of the present application.

The communication bus mentioned by the controller may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, and the like. For ease of representation, the figure only uses a thick line to represent, but it does not mean that there is only one bus or one type of bus.

The communication interface is used for communication between the controller and other devices.

The memory may include random access memory (Random Access Memory, RAM), and may also include non-volatile memory (Non-Volatile Memory, NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located far from the foregoing processor.

The aforementioned processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc .; it may also be a digital signal processor (Digital Signal Processing, DSP), special integration Circuit (Application Specific Integrated Circuit, ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

An embodiment of the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the method for mapping regions between screens provided by the embodiments of the present application is implemented. In any of the method steps.

It should be noted that in this article, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that these entities or operations There is any such actual relationship or order among them. Moreover, the terms "including," "including," or any other variation thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or device that includes a series of elements includes not only those elements but also those that are not explicitly listed Or other elements inherent to such a process, method, article, or device. Without more restrictions, the elements defined by the sentence "including a ..." do not exclude the existence of other identical elements in the process, method, article, or equipment including the elements.

Each embodiment in this specification is described in a related manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. In particular, the device embodiment, the system embodiment, the controller embodiment, and the computer-readable storage medium embodiment are basically similar to the method embodiment, so the description is relatively simple. For the related parts, refer to the method embodiment Just explain.

The above are only preferred embodiments of this application, and are not intended to limit this application. Any modification, equivalent replacement, or improvement made within the spirit and principles of this application shall be included in this application Within the scope of protection.

Claims (20)

1. A method for mapping regions between pictures is characterized in that the method includes:

   Determining the target area specified in the first picture of the area to be mapped;

   Determining a target coordinate conversion relationship between a camera to which the first picture belongs and a camera to which the second picture to be mapped belongs;

   Based on the target area and the target coordinate conversion relationship, an area corresponding to the target area is determined in the second picture.
2. The method according to claim 1, wherein the step of determining an area corresponding to the target area in the second screen based on the target area and the target coordinate conversion relationship comprises:

   Selecting multiple feature points from the target area, where the multiple feature points are multiple pixel points capable of characterizing the target area;

   Based on the plurality of feature points and the target coordinate conversion relationship, a region corresponding to the target region is determined in the second picture.
3. The method according to claim 2, wherein the camera to which the first picture belongs is a gun machine, and the camera to which the second picture belongs is a dome camera; or, the camera to which the first picture belongs is a dome camera, and The camera to which the second picture belongs is a gun;

   The step of determining a target coordinate conversion relationship between a camera to which the first picture belongs and a camera to which the second picture to be mapped belongs to includes:

   Determining a coordinate conversion relationship between the gun machine and the ball machine as a first coordinate conversion relationship;

   Determine the coordinate conversion relationship corresponding to the dome camera as the second coordinate conversion relationship, wherein the coordinate conversion relationship corresponding to the dome camera is: the azimuth of the dome camera and the center point of the picture collected by the dome camera Coordinate conversion relationship;

   The first coordinate conversion relationship and the second coordinate conversion relationship are determined as a target coordinate conversion relationship between a camera to which the first picture belongs and a camera to which the second picture to be mapped belongs.
4. The method according to claim 2, wherein the camera to which the first picture belongs is a first dome camera, the camera to which the second picture belongs is a second dome camera, and the azimuth of the first dome camera is fixed;

   The step of determining a target coordinate conversion relationship between a camera to which the first picture belongs and a camera to which the second picture to be mapped belongs includes:

   Determining a coordinate transformation relationship between the first dome camera and the second dome camera as a first coordinate transformation relationship;

   Determining a coordinate conversion relationship corresponding to the second ball machine as a second coordinate conversion relationship, wherein the coordinate conversion relationship corresponding to the second ball machine is: an azimuth of the second ball machine and the second ball The conversion relationship of the coordinates of the center point of the screen collected by the camera;

   The first coordinate conversion relationship and the second coordinate conversion relationship are determined as a target coordinate conversion relationship between a camera to which the first picture belongs and a camera to which the second picture to be mapped belongs.
5. The method according to claim 3 or 4, wherein the camera to which the first picture belongs is a gun machine, and the camera to which the second picture belongs is a dome camera; or the camera to which the first picture belongs is a first ball Camera, the camera to which the second picture belongs is a second dome camera, and the azimuth of the first dome camera is fixed;

   The step of determining an area corresponding to the target area in the second screen based on the plurality of feature points and the target coordinate conversion relationship includes:

   Determining an azimuth corresponding to each feature point based on the first coordinate conversion relationship and the coordinates of the plurality of feature points in the first screen;

   Selecting the first azimuth angle on which the target device is rotated from the determined azimuth angle, and rotating the target device according to the first azimuth angle to obtain the second picture; wherein when the camera to which the first picture belongs belongs Is a dome camera, when the camera to which the second picture belongs is a dome camera, the target device is the dome camera; when the camera to which the first picture belongs is a first dome camera, and the camera to which the second picture belongs is a second camera A dome camera, when the azimuth of the first dome camera is fixed, the target device is the second dome camera;

   Based on the second coordinate conversion relationship, a first target point corresponding to each feature point is determined in the second picture, wherein the first target point corresponding to any feature point is: the corresponding azimuth angle and the feature Pixels corresponding to the same azimuth angle;

   The area determined in the second picture based on the plurality of first target points is used as an area corresponding to the target area.
6. The method according to claim 3, wherein the camera to which the first picture belongs is a dome camera, and the camera to which the second picture belongs is a gun machine;

   The step of determining an area corresponding to the target area in the second screen based on the plurality of feature points and the target coordinate conversion relationship includes:

   Determining an azimuth corresponding to each feature point based on the second coordinate conversion relationship and the coordinates of the plurality of feature points in the first screen;

   Determining a second target point corresponding to each feature point in the second picture based on the first coordinate transformation relationship and the determined azimuth angle corresponding to each feature point;

   The area determined in the second picture based on the plurality of second target points is used as an area corresponding to the target area.
7. The method according to claim 3, wherein the coordinate conversion relationship between the gun machine and the ball machine is determined in the following ways, and the modes include:

   Determine the preset coordinates corresponding to the m reference points in the picture collected by the gun machine, and obtain m first coordinates, where the reference points are: the camera to which the first picture belongs and the second picture to which the first picture belongs Points in the three-dimensional space that can be captured by the camera, m≥4;

   For each reference point, determine the azimuth angle when the dome camera uses the reference point as the center of the screen, and obtain m azimuth angles;

   Converting the m azimuth angles into m second coordinates;

   Based on the m first coordinates and the m second coordinates, a homography matrix is calculated, and the homography matrix is used as a coordinate conversion relationship between the gun machine and the ball machine.
8. The method according to claim 2, wherein the camera to which the first picture belongs and the camera to which the second picture belongs are the same dome camera, and the azimuth of the first picture is the dome camera as an initial Picture collected at azimuth;

   The step of determining a target coordinate conversion relationship between a camera to which the first picture belongs and a camera to which the second picture to be mapped belongs to includes:

   Determining a coordinate conversion relationship corresponding to the dome camera, wherein the coordinate conversion relationship corresponding to the dome camera is: a transformation relationship between an azimuth angle of the dome camera and coordinates of a center point of a screen collected by the dome camera;

   The coordinate conversion relationship corresponding to the dome camera is used as the target coordinate conversion relationship corresponding to the camera to which the first picture belongs and the camera to which the second picture to be mapped belongs.
9. The method according to claim 8, wherein the step of determining a region corresponding to the target region in the second screen based on the plurality of feature points and the target coordinate conversion relationship includes: :

   Determining an azimuth corresponding to each feature point based on the target coordinate conversion relationship and the coordinates of the plurality of feature points in the first picture;

   Selecting a second azimuth angle on which the dome camera is rotated from the determined azimuth angle, and rotating the dome camera according to the second azimuth angle to obtain the second picture;

   Based on the target coordinate conversion relationship, a third target point corresponding to each feature point is determined in the second picture, and the third target point corresponding to any feature point is: a corresponding azimuth angle corresponding to the feature point Pixels with the same azimuth;

   The area determined in the second picture based on the plurality of third target points is used as an area corresponding to the target area.
10. The method according to claim 2, wherein the camera to which the first picture belongs and the camera to which the second picture belongs are different guns;

    The step of determining a target coordinate conversion relationship between a camera to which the first picture belongs and a camera to which the second picture to be mapped belongs to includes:

    Determining a coordinate conversion relationship between the different guns;

    The determined coordinate conversion relationship between different guns is used as a target coordinate conversion relationship between the camera to which the first picture belongs and the camera to which the second picture to be mapped belongs.
11. The method according to claim 10, wherein the step of determining an area corresponding to the target area in the second screen based on the plurality of feature points and the target coordinate conversion relationship comprises: :

    Determining a fourth target point corresponding to each feature point in the second picture based on the target coordinate conversion relationship and coordinates of the plurality of feature points in the first picture;

    And use an area determined in the second picture based on the plurality of fourth target points as an area corresponding to the target area.
12. A device for mapping regions between pictures is characterized in that the device includes:

    A first area determination module, configured to determine a target area specified in a first picture to be area-mapped;

    A transformation relationship determining module, configured to determine a target coordinate transformation relationship between a camera to which the first picture belongs and a camera to which the second picture to be mapped belongs;

    A second region determining module is configured to determine a region corresponding to the target region in the second screen based on the target region and the target coordinate conversion relationship.
13. A multi-camera observation system, characterized in that the system includes a controller and at least one camera, wherein the at least one camera includes a camera to which a first picture to be area mapped belongs and a camera to which a second picture to be area mapped belongs. ;

    The camera to which the first picture belongs is used to collect the first picture to be mapped in the area to be mapped;

    The camera to which the second picture belongs is used to collect the second picture to be mapped in the area to be mapped;

    The controller is configured to determine a target area specified in the first picture; determine a target coordinate conversion relationship between a camera to which the first picture belongs and a camera to which the second picture belongs; based on the target area and the target The coordinate conversion relationship determines a region corresponding to the target region in the second screen.
14. The system according to claim 13, wherein the controller is further configured to:

    Selecting multiple feature points from the target area, where the multiple feature points are multiple pixel points capable of characterizing the target area;

    Based on the plurality of feature points and the target coordinate conversion relationship, a region corresponding to the target region is determined in the second picture.
15. The system according to claim 14, wherein the at least one camera includes a gun machine and a dome camera, and the camera to which the first picture belongs is a gun machine and the camera to which the second picture belongs is a dome Or, the at least one camera includes a gun camera and a dome camera, and the camera to which the first picture belongs is a dome camera, and the camera to which the second picture belongs is a gun machine;

    The controller is configured to determine a coordinate transformation relationship between the gun machine and the dome camera as a first coordinate transformation relationship; determine a coordinate transformation relationship corresponding to the dome camera as a second coordinate transformation relationship; The first coordinate conversion relationship and the second coordinate conversion relationship are determined as a target coordinate conversion relationship between a camera to which the first picture belongs and a camera to which the second picture to be mapped belongs.

    The coordinate conversion relationship corresponding to the dome camera is a conversion relationship between the azimuth angle of the dome camera and the coordinates of the center point of the screen collected by the dome camera.
16. The system according to claim 14, wherein the at least one camera includes a first dome camera and a second dome camera, and the camera to which the first picture belongs is the first dome camera, and the second picture belongs to The camera is a second dome camera, and the azimuth of the first dome camera is fixed;

    The controller is configured to determine a coordinate transformation relationship between the first dome camera and the second dome camera as a first coordinate transformation relationship; determine a coordinate transformation relationship corresponding to the second dome camera as a second A coordinate conversion relationship, determining the first coordinate conversion relationship and the second coordinate conversion relationship as a target coordinate conversion relationship between a camera to which the first picture belongs and a camera to which the second picture to be mapped belongs;

    The coordinate conversion relationship corresponding to the second dome camera is a conversion relationship between the azimuth of the second dome camera and the coordinates of the center point of the screen collected by the second dome camera.
17. The system according to claim 15 or 16, wherein the at least one camera includes a gun machine and a dome camera, and the camera to which the first picture belongs is a gun machine, and the camera to which the second picture belongs is Dome camera; or, the at least one camera includes a first dome camera and a second dome camera, and the camera to which the first picture belongs is a first dome camera, and the camera to which the second picture belongs is a second dome camera, so The first azimuth angle is fixed;

    The controller is configured to determine an azimuth corresponding to each feature point based on the first coordinate conversion relationship and the coordinates of the plurality of feature points in the first screen; and select a rotation target from the determined azimuth angle. A first azimuth on which the device is based, and rotating the target device according to the first azimuth to obtain the second picture; and determining each feature point in the second picture based on the second coordinate conversion relationship A corresponding first target point; and an area determined in the second picture based on the plurality of first target points as an area corresponding to the target area;

    Wherein, when the camera to which the first picture belongs is a gun camera and the camera to which the second picture belongs is a dome camera, the target device is the dome camera; when the camera to which the first picture belongs is a first dome camera, The camera to which the second picture belongs is a second dome camera, and when the azimuth of the first dome camera is fixed, the target device is the second dome camera; the first target point corresponding to any feature point is: Pixel points corresponding to the same azimuth angle as the feature point.
18. The system according to claim 15, wherein the at least one camera comprises a gun machine and a dome camera, and the camera to which the first picture belongs is a dome camera, and the camera to which the second picture belongs is a gun machine. ;

    The controller is configured to determine an azimuth angle corresponding to each feature point based on the second coordinate conversion relationship and the coordinates of the plurality of feature points in the first screen; based on the first coordinate conversion relationship and all positions Determine the azimuth corresponding to each feature point, determine a second target point corresponding to each feature point in the second picture; and determine an area determined in the second picture based on the plurality of second target points As the area corresponding to the target area.
19. The system according to claim 14, wherein the at least one camera comprises: a dome camera, and the camera to which the first picture belongs and the camera to which the second picture belongs is the dome camera, and the The first picture is a picture collected when the azimuth of the dome camera is the initial azimuth;

    The controller is configured to determine the coordinate transformation relationship corresponding to the dome camera, and use the coordinate transformation relationship corresponding to the dome camera as the target coordinate transformation relationship of the camera to which the first picture belongs and the camera to which the second picture to be mapped belongs. ;

    The controller is further configured to determine an azimuth angle corresponding to each feature point based on the target coordinate conversion relationship and the coordinates of the plurality of feature points in the first screen; and select rotation from the determined azimuth angle. A second azimuth angle on which the dome camera is based, and rotating the dome camera according to the second azimuth angle to obtain the second picture; and based on the target coordinate conversion relationship, determine each in the second picture A third target point corresponding to the feature point; and an area determined in the second picture based on the plurality of third target points as an area corresponding to the target area;

    The coordinate conversion relationship corresponding to the dome camera is: the conversion relationship between the azimuth angle of the dome camera and the coordinates of the center point of the screen collected by the dome camera; the third target point corresponding to any feature point is: Pixel points corresponding to the same azimuth angle as the feature point.
20. The system according to claim 14, wherein the at least one camera comprises: different guns, and the camera to which the first picture belongs and the camera to which the second picture belongs are different guns;

    The controller is configured to determine a coordinate conversion relationship between the different guns; and use the determined coordinate conversion relationship between the different guns as a second mapping between the camera to which the first screen belongs and the area to be mapped. The target coordinate conversion relationship of the camera to which the picture belongs;

    The controller is further configured to determine a fourth target point corresponding to each feature point in the second picture based on the target coordinate conversion relationship and coordinates of the plurality of feature points in the first picture; And use an area determined in the second picture based on the plurality of fourth target points as an area corresponding to the target area.

Images