CN110517298B

CN110517298B - Track matching method and device

Info

Publication number: CN110517298B
Application number: CN201910799610.5A
Authority: CN
Inventors: 张成月; 亢乐; 包英泽
Original assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Current assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Priority date: 2019-08-27
Filing date: 2019-08-27
Publication date: 2022-10-21
Anticipated expiration: 2039-08-27
Also published as: CN110517298A

Abstract

The application discloses a track matching method and device, and relates to the technical field of image processing. The specific implementation scheme is as follows: determining a first reference point in an image acquired by a camera according to the constructed reference surface; acquiring a corresponding relation of each first reference point in the first camera and the second camera with relevance; according to the corresponding relation, mapping the character movement track acquired by the first camera to the image acquired by the second camera to form a first movement track; matching the first moving track with a second moving track of a person in an image acquired by a second camera; and judging whether the images acquired by the first camera and the second camera have the same target person or not according to the matching result of the movement track. According to the method and the device, the character moving track in the image collected by the first camera can be accurately mapped to the image collected by the second camera according to the corresponding relation of the first reference points, so that the character identification and tracking of the cross-camera can be accurately realized.

Description

Track matching method and device

Technical Field

The present application relates to the field of computer technology, and more particularly, to the field of image processing technology.

Background

In order to realize the movement track tracking and behavior analysis of the target person in the scene, the movement tracks of the target person in different cameras need to be determined. However, due to the fact that the arrangement positions of the cameras are different and the object occlusion exists in the acquisition scene, the acquired target person movement track error is large, and the accuracy of person cross-camera track matching is reduced.

Disclosure of Invention

The embodiment of the application provides a track matching method and device, and aims to solve one or more technical problems in the prior art.

In a first aspect, an embodiment of the present application provides a track matching method, including:

determining a plurality of first reference points in an image acquired by a camera according to the constructed reference surface;

acquiring corresponding relation of each first reference point in images acquired by a first camera and a second camera with relevance;

according to the corresponding relation, mapping the movement track of the person in the image acquired by the first camera to the image acquired by the second camera to form a first movement track;

matching the first moving track with a second moving track of a person in an image acquired by a second camera;

and identifying whether the images acquired by the first camera and the second camera have the same target person or not according to the matching result of the moving track.

According to the embodiment, the movement track of the person in the image acquired by the first camera can be accurately mapped to the image acquired by the second camera according to the corresponding relation of the first reference points, so that the person identification and tracking across the cameras can be accurately realized.

In one embodiment, before the step of mapping the movement track of the person in the image captured by the first camera to the image captured by the second camera according to the corresponding relationship to form the first movement track, the method further includes:

determining a plurality of second reference points in the image acquired by the camera by utilizing an interpolation algorithm based on the plurality of first reference points;

and acquiring the corresponding relation of each second reference point in the images acquired by the first camera and the second camera.

In this embodiment, each second reference point obtained by the interpolation algorithm can play a role in enriching reference points in the images acquired by the cameras, and further, the correspondence between the images acquired by the two cameras can be more accurately represented by the first reference point and the second reference point.

In one embodiment, the method further comprises:

and under the condition that the same target person exists in the images acquired by the first camera and the second camera, combining the moving track of the target person in the image acquired by the first camera with the moving track of the target person in the image acquired by the second camera to obtain the continuous moving track of the target person.

In this embodiment, by combining the movement tracks of the same target person in the first camera and the second camera, the continuous movement track of the target person when the target person moves across the cameras can be accurately obtained.

In one embodiment, the method further comprises:

calculating the average height of people according to the information of people in the image acquired by each camera;

and constructing a reference surface in the image acquired by each camera according to the average height of the person.

Because the image acquired by the camera is a plane image, a reference surface is constructed based on the average height of people, so that the reference point determined on the reference surface has higher reference value, the corresponding relation between the first camera and the second camera can be more accurately determined, and the error of the corresponding relation between the acquired images is reduced.

In one embodiment, matching the first movement track with a second movement track of a person in an image captured by a second camera comprises:

obtaining candidate moving tracks, wherein the candidate moving tracks comprise second moving tracks, the distance between the second moving tracks and the first moving tracks meets the threshold requirement;

judging whether the first moving track and the candidate moving track have the same track section within preset time or not;

and under the condition that the same track segment exists in the preset time, determining that the candidate moving track and the first moving track are the same track.

According to the embodiment, the second movement track corresponding to the first movement track can be more accurately determined through the distance between the first movement track and each second movement track and the track section.

In one embodiment, acquiring the correspondence between the first reference points in the images acquired by the first camera and the second camera with the relevance includes:

and acquiring the corresponding relation of each first reference point in the images acquired by the first camera and the second camera with the overlapped acquisition regions.

In the embodiment, the first camera and the second camera with the overlapped acquisition regions are used as the cameras with relevance, so that the corresponding relation of each first reference point in the images acquired by different cameras can be acquired more accurately.

In one embodiment, mapping the movement track of the person in the image captured by the first camera to the image captured by the second camera according to the corresponding relationship comprises:

and mapping the movement track of the person in the image acquired by the first camera to the image acquired by the second camera by utilizing homography transformation or perspective transformation according to the corresponding relation.

According to the embodiment, the movement track of the person in the image acquired by the first camera can be more accurately mapped to the image acquired by the second camera in a homography transformation or perspective transformation mode.

In one embodiment, the collection direction of the camera is towards the ground and perpendicular to the ground.

The acquisition direction of the camera of this embodiment is owing to the perpendicular to ground is disposed, consequently can make the acquisition scope maximize of camera, realizes utilizing as few as possible camera to accomplish the image acquisition of whole scene.

In a second aspect, an embodiment of the present application provides a track matching apparatus, including:

the first determining module is used for determining a plurality of first reference points in the image acquired by the camera according to the constructed reference surface;

the first acquisition module is used for acquiring the corresponding relation of each first reference point in the images acquired by the first camera and the second camera with relevance;

the mapping module is used for mapping the movement track of the person in the image acquired by the first camera to the image acquired by the second camera according to the corresponding relation to form a first movement track;

the matching module is used for matching the first movement track with a second movement track of a person in an image acquired by the second camera;

and the identification module is used for judging whether the images acquired by the first camera and the second camera have the same target person or not according to the matching result of the moving track.

In one embodiment, further comprising:

the second determination module is used for determining a plurality of second reference points in the image acquired by the camera by utilizing an interpolation algorithm based on the plurality of first reference points;

and the second acquisition module is used for acquiring the corresponding relation of each second reference point in the images acquired by the first camera and the second camera.

In one embodiment, the method further comprises:

and the third acquisition module is used for combining the moving track of the target person in the image acquired by the first camera with the moving track of the target person in the image acquired by the second camera to acquire the continuous moving track of the target person under the condition that the same target person exists in the images acquired by the first camera and the second camera.

In one embodiment, the matching module comprises:

the first obtaining sub-module is used for determining candidate moving tracks, and the candidate moving tracks comprise second moving tracks, the distance between the second moving tracks and the first moving tracks meets the requirement of a threshold value;

the judgment sub-module is used for judging whether the first moving track and the candidate moving track have the same track section within preset time;

and the determining submodule is used for judging that the candidate moving track and the first moving track are the same track under the condition that the same track segment exists in the preset time.

In one embodiment, the first obtaining module comprises:

and the second acquisition submodule is used for acquiring the corresponding relation of each first reference point in the images acquired by the first camera and the second camera with the overlapped acquisition regions.

In one embodiment, the mapping module includes:

and the mapping submodule is used for mapping the movement track of the person in the image acquired by the first camera to the image acquired by the second camera by utilizing homography transformation or perspective transformation according to the corresponding relation.

In a third aspect, an embodiment of the present application provides an electronic device, where functions of the electronic device may be implemented by hardware, or may be implemented by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above-described functions.

In one possible design, the electronic device includes a processor and a memory, the memory is used for storing a program for supporting the electronic device to execute the trajectory matching method, and the processor is configured to execute the program stored in the memory. The electronic device may also include a communication interface for communicating with other devices or a communication network.

In a fourth aspect, embodiments of the present application provide a non-transitory computer-readable storage medium storing computer instructions for storing an electronic device and computer software instructions for the electronic device, which include a program for executing the trajectory matching method.

One embodiment in the above application has the following advantages or benefits: according to the embodiment, the movement track of the person in the image collected by the first camera can be accurately mapped to the image collected by the second camera according to the corresponding relation of the first reference points, so that the person identification and tracking across the cameras are accurately realized, and the arrangement number of the cameras is reduced.

Other effects of the above-described alternative will be described below with reference to specific embodiments.

Drawings

The drawings are included to provide a better understanding of the present solution and are not to be considered limiting of the present application. Wherein:

FIG. 1 is a flow chart of a trajectory matching method according to a first embodiment of the present application;

FIG. 2 is a flow chart of another trajectory matching method according to a first embodiment of the present application;

FIG. 3 is a flow chart of another trajectory matching method according to the first embodiment of the present application;

FIG. 4 is a flow chart of another trajectory matching method according to the first embodiment of the present application;

FIG. 5 is a flowchart of step S400 of a trajectory matching method according to a first embodiment of the present application;

FIG. 6 is a flow chart of another trajectory matching method according to the first embodiment of the present application;

FIG. 7 is a flow chart of another trajectory matching method according to the first embodiment of the present application;

fig. 8A is a diagram of a track matching scenario in which the embodiment of the present application may be implemented.

Fig. 8B is a diagram of a trajectory matching scenario in which an embodiment of the present application may be implemented.

Fig. 9A is a diagram of a trajectory matching scenario in which an embodiment of the present application may be implemented.

Fig. 9B is a diagram of a track matching scenario in which an embodiment of the present application may be implemented.

Fig. 10A is a diagram of a trajectory matching scenario in which an embodiment of the present application may be implemented.

Fig. 10B is a diagram of a track matching scenario in which the embodiment of the present application may be implemented.

FIG. 11 is a block diagram of a trajectory matching device according to a second embodiment of the present application;

FIG. 12 is a block diagram of another trajectory matching device according to the second embodiment of the present application;

FIG. 13 is a block diagram of another trajectory matching device according to a second embodiment of the present application;

fig. 14 is a block diagram showing the configuration of a matching block of a trajectory matching device according to a second embodiment of the present application;

FIG. 15 is a block diagram of a first obtaining module of a track matching device according to a second embodiment of the present application;

FIG. 16 is a block diagram of a mapping module of a track matching device according to a second embodiment of the present application;

fig. 17 is a block diagram of an electronic device for implementing the trajectory matching method according to the embodiment of the present application.

Detailed Description

The following description of the exemplary embodiments of the present application, taken in conjunction with the accompanying drawings, includes various details of the embodiments of the application to assist in understanding, which are to be considered exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

According to a first embodiment of the present application, there is provided a trajectory matching method, as shown in fig. 1, the method including:

s100: and determining a plurality of first reference points in the image acquired by the camera according to the constructed reference surface.

The reference surface may include a virtual reference surface in the image captured by the camera, or may include a real reference surface formed based on a certain object in the image captured by the camera. The first reference point may be disposed on the reference surface, or may be disposed at a predetermined distance from the reference surface.

In one example, whether a reference plane needs to be constructed on each camera in the scene and the first reference point is determined may be adjusted as needed. If only a few of the cameras need to be used, the reference surface and the first reference point can be set on only the cameras that need to be used. And if all the cameras need to be used, setting reference surfaces and first reference points on all the cameras.

In one example, the first reference point may be disposed on what is covered by the reference surface. The first reference point may also be arranged on something at a distance from the reference surface. The object is an environmental object which can be shown in an image acquired by the camera.

S200: and acquiring the corresponding relation of each first reference point in the images acquired by the first camera and the second camera with relevance. By acquiring the corresponding relation of the first reference points in the two cameras with relevance, the position of the object in the image acquired by the first camera in the image acquired by the second camera can be determined.

In one example, the correspondence between the first reference points in the images acquired by the first camera and the second camera may be understood as: when the first reference point in the image collected by the first camera is mapped to the image collected by the second camera, the first reference point is the same as the first reference point in the image collected by the second camera. The corresponding relationship between the first reference points in the images acquired by the first camera and the second camera can be further understood as follows: when the first reference point in the image collected by the first camera is mapped to the image collected by the second camera, the first reference point is in relative position relation with the first reference point in the image collected by the second camera.

In one example, the first reference point having a corresponding relationship in the first camera and the second camera may be determined by means of image recognition.

S300: and mapping the movement track of the person in the image acquired by the first camera to the image acquired by the second camera according to the corresponding relation to form a first movement track. The movement track of the person can be formed according to the position change of the same person in the multi-frame images collected by the first camera.

In one example, a trajectory of movement of at least one person in an image captured by a first camera is projected into an image captured by a second camera.

In another example, a movement trajectory of a target person in an image captured by a first camera is mapped into an image captured by a second camera, the target person including a person whose movement trajectory passes through the overlapping region of the first and second camera captures. The acquisition overlap region can be understood as the same region in the acquired environment that the first camera and the second camera can shoot.

S400: and matching the first movement track with a second movement track of the person in the image acquired by the second camera. The second moving track of the person can be formed according to the position change of the same person in the multi-frame images collected by the second camera.

In one example, at least a portion of the first movement path and the second movement path are generated at the same time to ensure that the first movement path and the second movement path have a matching value.

In one example, although people in the images captured by the first camera and the second camera are changed in real time, things in the captured environment are relatively fixed, so that the first movement track and the second movement track are displayed on the image of any frame captured by the second camera, and the real movement routes of the people in the captured environment corresponding to the first movement track and the second movement track can be represented.

S500: and identifying whether the images acquired by the first camera and the second camera have the same target person or not according to the matching result of the movement track.

According to the embodiment, the movement track of the person in the image collected by the first camera can be accurately mapped to the image collected by the second camera according to the corresponding relation of the first reference points, so that the person identification and tracking across the cameras are accurately realized, and the arrangement number of the cameras is reduced.

It should be noted that in the same acquisition scene, every two cameras with relevance can be used as the first camera and the second camera. For example, when the cameras a, B, and C are related cameras (each camera has a collection overlap area with two other cameras), the cameras a and B may be a first camera and a second camera, and the cameras B and C may be the first camera and the second camera, and the cameras a and C may also be the first camera and the second camera.

In one embodiment, as shown in fig. 2, before the step of mapping the movement track of the person in the image captured by the first camera to the image captured by the second camera according to the corresponding relationship to form the first movement track, the method further includes:

and S600, determining a plurality of second reference points in the image acquired by the camera by using an interpolation algorithm based on the plurality of first reference points.

S700, acquiring the corresponding relation of each second reference point in the images acquired by the first camera and the second camera.

In this embodiment, each second reference point obtained by the interpolation algorithm can play a role in enriching reference points in the images acquired by the cameras, so as to further realize that the correspondence between the images acquired by the two cameras is more accurately represented by the first reference point and the second reference point.

In one example, the interpolation process includes generating a number of intermediate points (second reference points) in a proportional relationship between the first reference points. For example, the collected image of the first camera has first reference points a and B, and the collected image of the second camera has first reference points a 'and B' corresponding to the first reference points a and B. And respectively taking 1/3 and 2/3 positions of the connection line of the reference points A and B, and setting second reference points E and F. And similarly, respectively taking 1/3 and 2/3 positions of the connecting line of the reference points A 'and B', and setting second reference points E 'and F'. Since the first reference points a, B and the first reference points a ', B' have a corresponding relationship, the second reference points E, F and the second reference points E ', F' set in proportion also have a corresponding relationship.

In another example, a number of new second reference points may also be generated in a proportional relationship between the second reference points.

In one example, the first reference points may be set near the periphery of the image captured by the camera, so that more areas in the image captured by the camera can be covered by the interposed second reference points. In this way, when the movement track in the image collected by the first camera is mapped into the image collected by the second camera, the mapping error of the movement track is reduced.

In one embodiment, as shown in fig. 3, the method further includes:

s800: and under the condition that the same target person exists in the images acquired by the first camera and the second camera, combining the moving track of the target person in the image acquired by the first camera with the moving track of the target person in the image acquired by the second camera to obtain the continuous moving track of the target person.

In one embodiment, as shown in fig. 4, the trajectory matching method further includes:

s900: and calculating the average height of the people according to the people information in the images acquired by each camera.

S1000: and constructing a reference surface in the image acquired by each camera according to the average height of the people.

The personal information may include height information of a person in the captured image. Because the average height of human bodies is not greatly different under normal conditions, the reference surface constructed by taking the average height as the reference has higher universality. The error between the first reference points in the collected image of the relevance camera can be effectively reduced.

Because the image acquired by the camera is a plane image, the reference surface is constructed based on the average height of the person, so that the correlation between the reference point determined on the reference surface and the movement track of the person can be better realized, and the corresponding relation between the images acquired by the first camera and the second camera can be more accurate.

In one example, the average height of the human in one of the cameras can be used as a reference, and a reference surface can be constructed in the image acquired by each camera.

In one embodiment, as shown in fig. 5, matching the first movement track with the second movement track of the person in the image captured by the second camera includes:

s410: and acquiring candidate movement tracks, wherein the candidate movement tracks comprise second movement tracks of which the distances from the first movement tracks meet the threshold requirement.

In one example, the first camera and the second camera are cameras disposed at different locations that capture different regions. If the two persons have the same target person and have the same acquisition overlapping area, even if mapping errors exist, the first movement track and the second movement track are close to each other when the same person passes through the same area of the acquisition environment. Therefore, through the distance between the first moving track and the second moving track, the irrelevant moving track can be eliminated partially.

S420: and judging whether the first moving track and the candidate moving track have the same track section within preset time.

S430: and under the condition that the same track segment exists in the preset time, determining that the candidate moving track and the first moving track are the same track. The movement trajectory is formed by the person walking through different positions at different times. Because the collected environments corresponding to the first camera and the second camera are not completely the same, the moving tracks of the same target person collected by the first camera and the second camera are not completely the same. However, if the first movement trajectory and the second movement trajectory are substantially the same in a certain period of time, it may be determined that the first movement trajectory and the second movement trajectory are the same trajectory.

In one embodiment, as shown in fig. 6, acquiring a correspondence relationship between first reference points in images acquired by a first camera and a second camera having a relevance includes:

s210: and acquiring the corresponding relation of each first reference point in the images acquired by the first camera and the second camera with the acquisition overlapping areas. The acquisition overlap region can be understood as the same region in the acquired environment that the first camera and the second camera can shoot.

In the embodiment, the first camera and the second camera with the acquisition overlapping regions are used as the cameras with relevance, so that the corresponding relation of each first reference point in the images acquired by different cameras can be acquired more accurately.

In one example, the first camera and the second camera have a 10% -30% overlap area.

In one embodiment, as shown in fig. 7, mapping the movement track of the person in the image captured by the first camera to the image captured by the second camera according to the corresponding relationship includes:

s310: and mapping the movement track of the person in the image acquired by the first camera to the image acquired by the second camera by utilizing homography transformation or perspective transformation according to the corresponding relation.

In one example, the movement trajectory of the person in the image captured by the first camera is mapped into the image captured by the second camera by calling a mapping function cv2.FindHomography (homography transform) of opencv (Open Source Computer Vision Library). Or, by calling a mapping function cv2.GetPerspectiveTransform (perspective transformation) of opencv, the movement track of the person in the image acquired by the first camera is mapped to the image acquired by the second camera.

The homography transformation is the mapping of points in one image into another image by a homography matrix. It is also understood to mean projection from one plane to another. The homography is a reversible mapping of points and lines that occurs on a projection plane. The homography transformation matrix is a 3x3 matrix. This transform can be arbitrarily multiplied by a non-zero constant without changing the transform itself.

A perspective transformation is a mapping of a picture onto a new view plane, also called a projective map, which is a two-dimensional (X, Y) to three-dimensional (X, Y, Z) and then to another two-dimensional (X ', Y') (X ', Y') space. It provides greater flexibility in mapping one quadrilateral area to another quadrilateral area (not necessarily a parallelogram). It enables linear transformation and translation.

In one embodiment, the collection direction of the camera is towards the ground and perpendicular to the ground. The acquisition direction of the camera of the embodiment is perpendicular to the ground, so that the acquisition range of the camera can be maximized.

In one embodiment, the images captured by the camera are pre-processed by a de-distortion technique prior to construction of the reference plane.

In one embodiment, if the cameras a and B have a relationship and the cameras B and C also have a relationship, the movement trajectory a of the person in the image captured by the camera a matches the corresponding movement trajectory B in the image captured by the camera B, and the movement trajectory B of the person in the image captured by the camera B matches the corresponding movement trajectory C in the image captured by the camera C, it can be determined that the movement trajectory a of the person in the image captured by the camera a and the movement trajectory C of the person in the image captured by the camera C also correspond to each other. Therefore, the same target person can be further confirmed to be acquired by the cameras A, B and C. And combining the moving tracks a, B and C to obtain the continuous moving tracks of the target person when the target person approaches the cameras A, B and C.

In an application example, as shown in fig. 8 to 10, the methods of the embodiments described above are applied to a supermarket collection scene. Fig. 8A is an image captured by a first camera, and fig. 8B is an image captured by a second camera. Fig. 9A is an image captured by a first camera, and fig. 9B is an image captured by a second camera. Fig. 10A is an image captured by a first camera, and fig. 10B is an image captured by a second camera.

And constructing a reference surface according to the average height of each person in the first camera and the second camera. The reference surface is the fourth layer edge of the shelf.

As shown in fig. 8A and 8B, reference points a, B, C, D, and E are set on the edge of the fourth layer of the rack in the image captured by the first camera based on the reference plane. Reference points A ', B ', C ', D ' and E ' are arranged on the edge of the fourth layer of the shelf in the image acquired by the second camera. The reference points A and A ' have a corresponding relationship, the reference points B and B ' have a corresponding relationship, and the reference points C and C ' have a corresponding relationship.

As shown in fig. 9A and 9B, based on the reference points a, B, and C in the image captured by the first camera, reference points a, B, and C are added to the image captured by the first camera by using an interpolation algorithm. Based on the reference points A ', B' and C 'in the image collected by the second camera, the reference points a', B 'and C' are added in the image collected by the first camera by utilizing an interpolation algorithm.

As shown in fig. 10A and 10B, the movement locus 1 of the person Z, the movement locus 2 of the person Y, and the movement locus 3 of the person X are specified in the image captured by the first camera. The movement track 1 'of the person Z', the movement track 2 'of the person Y' and the movement track 3 'of the person X' are determined in the image captured by the second camera.

And mapping the moving track 1 of the person Z, the moving track 2 of the person Y and the moving track 3 of the person X into the image collected by the second camera. And is matched with the moving track 1 'of the person Z', the moving track 2 'of the person Y' and the moving track 3 'of the person X' in the image captured by the second camera.

And according to the matching result, confirming that the person Z in the image acquired by the first camera is the same person as the person Z' in the image acquired by the second camera. The person Y in the image collected by the first camera and the person Y' in the image collected by the second camera are the same person.

According to a second embodiment of the present application, an embodiment of the present application provides a trajectory matching device 100, as shown in fig. 11, the device includes:

the first determining module 10 is configured to determine a plurality of first reference points in an image acquired by the camera according to the constructed reference plane.

The first obtaining module 20 is configured to obtain a corresponding relationship between first reference points in images acquired by the first camera and the second camera, where the first reference points have a relevance.

And the mapping module 30 is configured to map the movement track of the person in the image acquired by the first camera to the image acquired by the second camera according to the corresponding relationship, so as to form a first movement track.

And the matching module 40 is used for matching the first movement track with a second movement track of a person in the image acquired by the second camera.

And the identification module 50 is configured to determine whether the images acquired by the first camera and the second camera have the same target person according to the matching result of the movement track.

In one embodiment, as shown in fig. 12, the method further includes:

a second determining module 60, configured to determine a plurality of second reference points in the image captured by the camera by using an interpolation algorithm based on the plurality of first reference points.

The second obtaining module 70 is configured to obtain a corresponding relationship between second reference points in the images acquired by the first camera and the second camera.

In one embodiment, as shown in fig. 13, the method further includes:

and a third obtaining module 80, configured to, in a case where the same target person exists in the images captured by the first camera and the second camera, combine a moving track of the target person in the image captured by the first camera with a moving track of the target person in the image captured by the second camera to obtain a continuous moving track of the target person.

In one embodiment, as shown in fig. 14, the matching module 40 includes:

the first obtaining sub-module 41 is configured to determine candidate movement tracks, where the candidate movement tracks include a second movement track whose distance from the first movement track meets a threshold requirement.

And the judging submodule 42 is configured to judge whether the first moving trajectory and the candidate moving trajectory have the same trajectory segment within a preset time.

The determining submodule 43 is configured to determine that the candidate moving trajectory is the same as the first moving trajectory when the same trajectory segment exists within a preset time.

In one embodiment, as shown in fig. 15, the first obtaining module 20 includes:

and the second obtaining submodule 21 is configured to obtain a corresponding relationship between the first reference points in the images collected by the first camera and the second camera, where the first reference points have a collection overlapping area.

In one embodiment, as shown in FIG. 16, the mapping module 30 includes:

and the mapping submodule 31 is configured to map the movement track of the person in the image acquired by the first camera to the image acquired by the second camera by using homography transformation or perspective transformation according to the corresponding relationship.

According to an embodiment of the present application, an electronic device and a readable storage medium are also provided.

Fig. 17 is a block diagram of an electronic device according to the trajectory matching method of the embodiment of the present application. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. Electronic devices may also represent various forms of mobile devices, such as personal digital processors, cellular telephones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present application that are described and/or claimed herein.

As shown in fig. 17, the electronic apparatus includes: one or more processors 901, memory 902, and interfaces for connecting the various components, including high-speed interfaces and low-speed interfaces. The various components are interconnected using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions for execution within the electronic device, including instructions stored in or on the memory to display Graphical information for a Graphical User Interface (GUI) on an external input/output device, such as a display device coupled to the Interface. In other embodiments, multiple processors and/or multiple buses may be used, along with multiple memories and multiple memories, as desired. Also, multiple electronic devices may be connected, with each device providing some of the necessary operations (e.g., as an array of servers, a group of blade servers, or a multi-processor system). Fig. 17 illustrates an example of a processor 901.

Memory 902 is a non-transitory computer readable storage medium as provided herein. Wherein the memory stores instructions executable by at least one processor to cause the at least one processor to perform the trajectory matching method provided herein. The non-transitory computer readable storage medium of the present application stores computer instructions for causing a computer to perform the method of trajectory matching provided herein.

The memory 902, which is a non-transitory computer readable storage medium, may be used to store non-transitory software programs, non-transitory computer executable programs, and modules, such as program instructions/modules corresponding to the method of trajectory matching in the embodiments of the present application. The processor 901 executes various functional applications of the server and data processing, i.e., a method of trajectory matching in the above-described method embodiments, by executing non-transitory software programs, instructions, and modules stored in the memory 902.

The memory 902 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to use of the track-matched electronic device, and the like. Further, the memory 902 may include high speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory 902 may optionally include memory located remotely from the processor 901, which may be connected to the trace-matching electronic device over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The electronic device of the method of trajectory matching may further include: an input device 903 and an output device 904. The processor 901, the memory 902, the input device 903, and the output device 904 may be connected by a bus or other means, and fig. 17 illustrates an example of connection by a bus.

The input device 903 may receive input numeric or character information and generate key signal inputs related to user settings and function controls of the electronic apparatus matched to the trajectory, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, a pointing stick, one or more mouse buttons, a track ball, a joystick, or other input devices. The output devices 904 may include a display device, auxiliary lighting devices (e.g., LEDs), and tactile feedback devices (e.g., vibrating motors), among others. The Display device may include, but is not limited to, a Liquid Crystal Display (LCD), a Light Emitting Diode (LED) Display, and a plasma Display. In some implementations, the display device can be a touch screen.

Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, application Specific Integrated Circuits (ASICs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.

These computer programs (also known as programs, software applications, or code) include machine instructions for a programmable processor, and may be implemented using high-level procedural and/or object-oriented programming languages, and/or assembly/machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and/or device (e.g., magnetic discs, optical disks, memory, programmable Logic Devices (PLDs)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and/or data to a programmable processor.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (Cathode Ray Tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the internet.

The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

The application has the following advantages: 1. the acquisition direction of the camera of the embodiment is perpendicular to the ground, so that the acquisition range of the camera can be maximized, and the arrangement number of the cameras is reduced. 2. According to the method and the device, the mapping of the movement track is realized by using the reference point and the reference plane, so that the relevance after the movement track is mapped is effectively enhanced. Therefore, the matching of the person track, the human body positioning, the track tracking and the track combination can be completed under the condition that the cameras are not required to be densely arranged. The burden of data calculation, processing and synchronous uploading is greatly reduced. 3. According to the method and the device, the character moving track in the image collected by the first camera can be accurately mapped to the image collected by the second camera according to the corresponding relation of the first reference points, so that the character identification and tracking of the cross-camera can be accurately realized, and the arrangement number of the cameras is reduced.

It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present application may be executed in parallel, sequentially, or in different orders, and the present invention is not limited thereto as long as the desired results of the technical solutions disclosed in the present application can be achieved.

The above-described embodiments should not be construed as limiting the scope of the present application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made, depending on design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A trajectory matching method, comprising:

acquiring the corresponding relation of each first reference point in the images acquired by the first camera and the second camera with relevance;

matching the first moving track with a second moving track of a person in the image acquired by the second camera;

identifying whether the images acquired by the first camera and the second camera have the same target person or not according to the matching result of the movement track;

under the condition that the same target person exists in the images acquired by the first camera and the second camera, combining the moving track of the target person in the image acquired by the first camera with the moving track of the target person in the image acquired by the second camera to obtain a continuous moving track of the target person;

calculating the average height of people according to the information of people in the images acquired by each camera;

and constructing a reference surface in the image acquired by each camera according to the average height of the people.

2. The method of claim 1, wherein before mapping the movement track of the person in the image captured by the first camera to the image captured by the second camera according to the correspondence relationship to form a first movement track, the method further comprises:

3. The method of claim 1 or 2, wherein matching the first movement trajectory with a second movement trajectory of a person in an image captured by the second camera comprises:

acquiring candidate moving tracks, wherein the candidate moving tracks comprise second moving tracks, the distance between which and the first moving tracks meets the requirement of a threshold value;

and under the condition that the same track segment exists in the preset time, determining that the candidate movement track and the first movement track are the same track.

4. The method according to claim 1 or 2, wherein obtaining the corresponding relationship of each first reference point in the images acquired by the first camera and the second camera with the correlation comprises:

5. The method according to claim 1 or 2, wherein mapping the movement track of the person in the image captured by the first camera to the image captured by the second camera according to the correspondence comprises:

6. The method according to claim 1 or 2, wherein the collection direction of the camera is towards the ground and perpendicular to the ground.

7. A trajectory matching device, comprising:

the matching module is used for matching the first moving track with a second moving track of a person in the image acquired by the second camera;

the identification module is used for identifying whether the images acquired by the first camera and the second camera have the same target person or not according to the matching result of the movement track;

a third obtaining module, configured to, in a case that the same target person exists in the images acquired by the first camera and the second camera, combine a moving track of the target person in the image acquired by the first camera with a moving track of the target person in the image acquired by the second camera to obtain a continuous moving track of the target person;

the first obtaining submodule is used for determining candidate movement tracks, and the candidate movement tracks comprise second movement tracks, the distance between the second movement tracks and the first movement tracks meets the requirement of a threshold value;

8. The apparatus of claim 7, further comprising:

9. The apparatus of claim 7 or 8, wherein the first obtaining module comprises:

10. The apparatus of claim 7 or 8, wherein the mapping module comprises:

11. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein, the first and the second end of the pipe are connected with each other,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-6.

12. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-6.