WO2023273154A1

WO2023273154A1 - Image processing method and apparatus, and device, medium and program

Info

Publication number: WO2023273154A1
Application number: PCT/CN2021/134883
Authority: WO
Inventors: 干刚; 谭志颖
Original assignee: 西安商汤智能科技有限公司
Priority date: 2021-06-29
Filing date: 2021-12-01
Publication date: 2023-01-05
Also published as: CN113643320A

Abstract

Disclosed in the embodiments of the present application are an image processing method and apparatus, and a device, a medium and a program. The method is executed by an electronic device, and the method comprises: acquiring a first three-dimensional model of a target scene and at least one piece of first trajectory point data of a target object in the target scene; and displaying the trajectory of the target object in the first three-dimensional model according to the at least one piece of first trajectory point data. In this way, the trajectory of a target object in a target scene can be visually presented.

Description

Image processing method, device, equipment, medium and program

Cross References to Related Applications

This patent application requires that the Chinese patent application number 202110728309.2 submitted on June 29, 2021, the applicant is Xi'an Shangtang Intelligent Technology Co., Ltd., and the application name is "image processing method and device, electronic equipment and computer readable storage medium" priority rights, the entirety of which is incorporated into this application by reference.

technical field

The present application relates to the technical field of image processing, and in particular to an image processing method, device, equipment, medium and program.

Background technique

In the field of security, the trajectory of the target object is usually displayed, and relevant personnel can analyze the whereabouts of the target object and other information by observing the trajectory of the target object. Therefore, how to improve the display effect of the trajectory of the target object is of great significance.

Contents of the invention

Embodiments of the present application provide an image processing method, device, device, medium, and program.

An embodiment of the present application provides an image processing method, the method is executed by an electronic device, and the method includes:

Acquiring a first three-dimensional model of a target scene and at least one first trajectory point data of a target object in the target scene;

A trajectory of the target object is displayed in the first three-dimensional model according to the at least one first trajectory point data.

In this way, by displaying the trajectory of the target object in the first three-dimensional model according to at least one first trajectory point data of the target object in the target scene, the trajectory of the target object in the target scene can be displayed more intuitively.

In some embodiments of the present application, the displaying the trajectory of the target object in the first three-dimensional model according to the at least one first trajectory point data includes: according to the at least one first trajectory point data Determining at least one first trajectory point of the target object; determining a first three-dimensional area containing the at least one first trajectory point from the first three-dimensional model; displaying in the first three-dimensional model in a predetermined display manner the first three-dimensional area, and display the track of the target object in the first three-dimensional area. In this way, the display effect of the track of the target object can be improved.

In some embodiments of the present application, the determining the first three-dimensional area containing the at least one first trajectory point from the first three-dimensional model includes: obtaining the size of the first three-dimensional area and the first three-dimensional area The shape of a three-dimensional area; determining the centroid of the at least one first trajectory point; using the centroid as the center of the first three-dimensional area, and according to the size of the first three-dimensional area and the first three-dimensional area A shape determines the first three-dimensional region from the first three-dimensional model. In this way, the first three-dimensional area can contain more trajectory points.

In some embodiments of the present application, the at least one first track point includes a second track point, a third track point, a fourth track point, at least one fifth track point, the timestamp of the second track point, The timestamp of the third trajectory point, the timestamp of the fourth trajectory point, and the timestamp of the at least one fifth trajectory point decrease in sequence; the determination of the centroid of the at least one first trajectory point includes : When the third trajectory point is located in the first region to be confirmed, determine the centroid of the first region to be confirmed as the centroid of the at least one first trajectory point, the first region to be confirmed is a polygon area, the vertices of the first area to be confirmed are the second track point, the fourth track point, and the at least one fifth track point; if the third track point is located outside the first area to be confirmed In this case, the centroid of the second area to be confirmed is determined as the centroid of the at least one first track point, the first area to be confirmed and the second area to be confirmed are both polygonal areas, and the first area to be confirmed is The vertices of the second track point, the fourth track point, and the at least one fifth track point, the vertices of the second area to be confirmed are the second track point, the third track point, The fourth track point, the at least one fifth track point. In this way, the accuracy of the centroid of at least one first trajectory point can be improved.

In some embodiments of the present application, the determining the at least one first track point of the target object according to the at least one first track point data includes: determining the maximum time stamp in the at least one first track point data The n pieces of track point data are at least one valid track point data; the n is a positive integer; and at least one first track point of the target object is determined according to the at least one valid track point data. In this way, the amount of data processing can be reduced and the processing speed can be increased.

In some embodiments of the present application, when the number of the first track point data is greater than 1, the acquiring at least one first track point data of the target object in the target scene includes: acquiring the At least two second track point data and a distance threshold of the target object in the target scene; determine the distance between adjacent track point data sets, and the adjacent track point data sets include two of the second track point data sets with adjacent time stamps. Two track point data; when the distance is less than the distance threshold, remove the old track point data from the at least two second track point data to obtain the at least one first track point data, the The old track point data is the second track point data with the smallest time stamp in the adjacent track point data set; or the at least two second track point data are used as the one or more first track point data. In this way, the amount of data processing can be reduced.

In some embodiments of the present application, the trajectory of the target object includes a first trajectory and a second trajectory, the first trajectory is a trajectory between the trajectory point with the mth largest timestamp and the trajectory point with the largest timestamp, The second track is a track other than the first track in the track of the target object, and the display mode of the first track is different from the display mode of the second track; the m is equal to or greater than A positive integer of 2. In this way, the first trajectory and the second trajectory are displayed in different ways to distinguish the first trajectory from the second trajectory, and then the moving direction of the target object can be determined.

In some embodiments of the present application, the first three-dimensional model includes a first floor and a second floor, the height of the first floor is different from the height of the second floor, and the trajectory of the target object is in the On the first floor, the method further includes: displaying the second floor in the first three-dimensional model when it is determined that the target object appears on the second floor. In this way, the display effect of the first floor can be improved.

In some embodiments of the present application, the method of determining that the target object appears before the second floor further includes: acquiring a first image containing the target object; determining that the target object appears On the second floor, comprising: determining that the second image contains the target object by comparing the first image with a second image, the second image being collected by a camera on the second floor . In this way, the appearance position of the target object can be determined efficiently.

For the effect description of the following devices, electronic equipment, etc., please refer to the description of the above-mentioned image processing method.

An embodiment of the present application provides an image processing device, the device comprising:

An acquisition unit configured to acquire a first three-dimensional model of a target scene and at least one first trajectory point data of a target object in the target scene;

A processing unit configured to display the trajectory of the target object in the first three-dimensional model according to the at least one first trajectory point data.

In some embodiments of the present application, the processing unit includes: a first processing subunit configured to determine at least one first track point of the target object according to the at least one first track point data; the second processing A subunit configured to determine a first three-dimensional region containing the at least one first trajectory point from the first three-dimensional model; a third processing subunit configured to display in a predetermined display manner in the first three-dimensional model the first three-dimensional area, and display the track of the target object in the first three-dimensional area.

In some embodiments of the present application, the second processing subunit is further configured to obtain the size of the first three-dimensional area and the shape of the first three-dimensional area; determine the centroid of the at least one first trajectory point ; taking the centroid as the center of the first three-dimensional area, and determining the first three-dimensional area from the first three-dimensional model according to the size of the first three-dimensional area and the shape of the first three-dimensional area.

In some embodiments of the present application, the at least one first track point includes a second track point, a third track point, a fourth track point, at least one fifth track point, the timestamp of the second track point, The timestamp of the third trajectory point, the timestamp of the fourth trajectory point, and the timestamp of the at least one fifth trajectory point decrease in sequence; the second processing subunit is further configured to When the three track points are located in the first area to be confirmed, the centroid of the first area to be confirmed is determined as the centroid of the at least one upper first track point, the first area to be confirmed is a polygonal area, the The vertices of the first area to be confirmed are the second track point, the fourth track point, and the at least one fifth track point; when the third track point is located outside the first area to be confirmed, determine The centroid of the second area to be confirmed is used as the centroid of the at least one first trajectory point, the first area to be confirmed and the second area to be confirmed are both polygonal areas, and the vertex of the first area to be confirmed is the The second track point, the fourth track point, and the at least one fifth track point, the apex of the second area to be confirmed is the second track point, the third track point, the fourth track point trajectory points, the at least one fifth trajectory point.

In some embodiments of the present application, the first processing subunit is further configured to determine that the n track point data with the largest timestamp in the at least one first track point data are at least one valid track point data; n is a positive integer; at least one first track point of the target object is determined according to the at least one valid track point data.

In some embodiments of the present application, when the number of the first trajectory point data is greater than 1, the acquisition unit includes: a first acquisition subunit configured to acquire the location of the target object in the target scene At least two second track point data and a distance threshold within the distance; determine the distance of the adjacent track point data set, the adjacent track point data set includes the two second track point data with adjacent time stamps; in the When the distance is less than the distance threshold, the old track point data is removed from the at least two second track point data to obtain the at least one first track point data, and the old track point data is the The second track point data with the smallest time stamp in the adjacent track point data sets; or using the at least two second track point data as the one or more first track point data.

In some embodiments of the present application, the trajectory of the target object includes a first trajectory and a second trajectory, the first trajectory is a trajectory between the trajectory point with the mth largest timestamp and the trajectory point with the largest timestamp, The second track is a track other than the first track in the track of the target object, and the display mode of the first track is different from the display mode of the second track; the m is equal to or greater than A positive integer of 2.

In some embodiments of the present application, the first three-dimensional model includes a first floor and a second floor, the height of the first floor is different from the height of the second floor, and the trajectory of the target object is in the On the first floor, the device further includes: a display unit configured to display the second floor in the first three-dimensional model when it is determined that the target object appears on the second floor.

In some embodiments of the present application, the determining that the target object appears before the second floor, the acquiring unit is configured to acquire a first image containing the target object; the processing unit is further configured In order to determine that the second image includes the target object by comparing the first image with the second image, the second image is collected by a camera on the second floor.

An embodiment of the present application provides an electronic device, including: a processor and a memory, the memory is used to store computer program codes, the computer program codes include computer instructions, and when the processor executes the computer instructions , the electronic device executes the image processing method described in any one of the above embodiments.

An embodiment of the present application provides another electronic device, including: a processor, a sending device, an input device, an output device, and a memory, the memory is used to store computer program codes, and the computer program codes include computer instructions. When the processor executes the computer instructions, the electronic device executes the image processing method described in any one of the above embodiments.

An embodiment of the present application provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, the computer program includes program instructions, and when the program instructions are executed by a processor, the The processor executes the image processing method described in any one of the above embodiments.

An embodiment of the present application provides a computer program product, the computer program product includes a computer program or an instruction, and when the computer program or instruction is run on an electronic device, the electronic device is made to execute any of the above-mentioned embodiments The image processing method described.

Embodiments of the present application at least provide an image processing method, device, device, medium, and program. First, acquire a first three-dimensional model of a target scene and at least one first trajectory point data of a target object in the target scene; then, A trajectory of the target object is displayed in the first three-dimensional model according to the at least one first trajectory point data. In this way, the trajectory of the target object in the target scene can be intuitively displayed

It should be understood that the above general description and the following detailed description are only exemplary and explanatory, rather than limiting the embodiment of the application.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiment of the present application or the background art, the following will describe the drawings that need to be used in the embodiment of the present application or the background art.

The accompanying drawings here are incorporated into the specification and constitute a part of the specification. These drawings show embodiments consistent with the application, and are used together with the description to describe the technical solution of the application.

FIG. 1 is a schematic flow diagram of an image processing method provided in an embodiment of the present application;

FIG. 2 shows a schematic diagram of a system architecture to which the image processing method of the embodiment of the present application can be applied;

FIG. 3 is a schematic diagram of a track point area of a target object provided in an embodiment of the present application;

FIG. 4 is a schematic diagram of a track point area of a target object provided in an embodiment of the present application;

FIG. 5 is a schematic diagram of a track point area of a target object provided in an embodiment of the present application;

FIG. 6 is a schematic diagram of a trajectory between trajectory points provided by an embodiment of the present application;

FIG. 7 is a schematic diagram of another trajectory between trajectory points provided by the embodiment of the present application;

FIG. 8 is another schematic diagram of a trajectory between trajectory points provided by the embodiment of the present application;

FIG. 9 is another schematic diagram of a trajectory between trajectory points provided by the embodiment of the present application;

FIG. 10 is a schematic diagram showing the first floor in the first three-dimensional model provided by the embodiment of the present application;

Fig. 11 is a schematic diagram showing the second floor in the first three-dimensional model provided by the embodiment of the present application;

FIG. 12 is a schematic structural diagram of an image processing device provided by an embodiment of the present application;

FIG. 13 is a schematic diagram of a hardware structure of an image processing device provided by an embodiment of the present application.

detailed description

In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiment of the application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiment of the application. Obviously, the described embodiment is only It is a part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the scope of protection of this application.

The terms "first", "second" and the like in the specification and claims of the present application and the above drawings are used to distinguish different objects, rather than to describe a specific order. Furthermore, the terms "include" and "have", as well as any variations thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but optionally also includes unlisted steps or units, or optionally further includes For other steps or units inherent in these processes, methods, products or devices.

It should be understood that in this application, "at least one (item)" means one or more, "multiple" means two or more, and "at least two (items)" means two or three And three or more, "and/or", is used to describe the association relationship of associated objects, indicating that there can be three types of relationships, for example, "A and/or B" can mean: only A exists, only B exists, and A exists at the same time and B, where A and B can be singular or plural. The character "/" can indicate that the contextual objects are an "or" relationship, which refers to any combination of these items, including any combination of single items (items) or plural items (items). For example, at least one item (piece) of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c ", where a, b, c can be single or multiple. The character "/" can also represent a division sign in mathematical operations, for example, a/b=a divided by b; 6/3=2. "At least one of the following" or similar expressions.

Reference herein to an "embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The occurrences of this phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is understood explicitly and implicitly by those skilled in the art that the embodiments described herein can be combined with other embodiments.

In related technologies, the more intuitive display of structured information in the physical world in the digital world is of great value and significance to the actual business work of customers, and the multi-faceted information of personnel, ground and objects in buildings can be carried out in the 3D (3Dimension, 3D) digital world. Digital twin; Among them, the target trajectory is the most concerned aspect. Based on the analysis and extraction of the video information of the camera inside the building, combined with the internal road network information, the trajectory of the personnel target can be acquired and displayed in real time, combined with the 3D building model The rendering of the floor achieves the effect of dismantling the floor.

The execution subject of the embodiment of the present application is an image processing device, wherein the image processing device may be any electronic device capable of executing the technical solutions disclosed in the method embodiments of the present application. In some embodiments of the present application, the image processing device may be one of the following: a mobile phone, a computer, a tablet computer, and a wearable smart device.

It should be understood that the method embodiments of the present application may also be implemented in a manner in which a processor executes computer program codes. Embodiments of the present application are described below with reference to the drawings in the embodiments of the present application. Please refer to FIG. 1 . FIG. 1 is a schematic flowchart of an image processing method provided in an embodiment of the present application.

101. Acquire a first three-dimensional model of a target scene and at least one first track point data of a target object in the target scene.

In this embodiment of the present application, the target scene may be any scene. For example, the target scene is inside a building; for another example, the target scene is an underground parking lot; for another example, the target scene is a campus; for another example, the target scene is a scene in a shopping mall.

In the embodiment of the present application, the three-dimensional model (including the first three-dimensional model and the second three-dimensional model to be mentioned below) can be a computer-aided design (Computer Aided Design, CAD) three-dimensional model, and the three-dimensional model can also be a three-dimensional convex hull, A 3D model can also be a 3D point cloud.

In this embodiment of the present application, the target object may be any object. In some embodiments of the present application, the target object includes one of the following: people, vehicles, and robots.

In this embodiment of the present application, the number of the first track point data may be one or greater than one. The first track point data includes a location and a time stamp. The position of the first track point data is a position in the coordinate system of the first three-dimensional model, that is, the position of the first track point data is a three-dimensional coordinate.

For example, at least one first track point data of the target object includes first track point data a. If the first track point data a includes point A and time stamp t1. According to the first track point data, it can be determined that the target object appears at point A at t1.

In some embodiments of the present application, the image processing device uses the 3D model of the target scene input by the user through the input component as the first 3D model. The above-mentioned input components include: a keyboard, a mouse, a touch screen, a touch pad and an audio input device.

In some embodiments of the present application, the image processing apparatus receives the 3D model of the target scene sent by the terminal as the first 3D model. The above-mentioned terminal can be any of the following: mobile phone, computer, tablet computer, server.

In some embodiments of the present application, the image processing device includes lidar. The image processing device scans the target scene through the laser radar, and obtains a 3D model of the target scene as a first 3D model.

In some embodiments of the present application, the image processing device receives the track point data input by the user through the input component, so as to obtain at least one first track point data of the target object in the target scene.

In some embodiments of the present application, the image processing apparatus receives the track point data sent by the terminal to obtain at least one first track point data of the target object in the target scene.

In some embodiments of the present application, the image processing device acquires at least one track point data to be converted and the coordinate conversion relationship of the target object in the global positioning system coordinate system; wherein, the coordinate conversion relationship is the coordinate system of the global positioning system and the first The conversion relationship between the coordinate systems of the 3D model. The image processing device converts at least one trajectory point data to be converted of the target object in the global positioning system coordinate system into at least one first trajectory point data of the target object in the first three-dimensional model according to the coordinate transformation relationship, and obtains the target object in the first three-dimensional model. At least one first track point data in the target scene.

It should be understood that, in this embodiment of the present application, acquiring the first 3D model of the target scene and acquiring at least one first trajectory point data of the target object in the target scene may be performed separately, or may be performed simultaneously.

102. Display the trajectory of the target object in the first three-dimensional model according to at least one first trajectory point data.

The image processing device sequentially connects the first trajectory point data of the target object in the target scene according to the size of the time stamp, obtains the trajectory of the target object in the first 3D model, and displays the trajectory in the first 3D model.

For example, the first track point data of the target object includes: track point data a, track point data b, and track point data c; wherein, track point data a includes position A and timestamp t1, and track point data b includes position B and time Stamp t2, track point data c includes position C and time stamp t3.

If t1 is earlier than t2, and t2 is earlier than t3, then the trajectory of the target object in the first three-dimensional model is that the target object appears at position A at t1, appears at position B at t2, and appears at position C at t3.

In the embodiment of the present application, the image processing device displays the trajectory of the target object in the first three-dimensional model based on at least one first trajectory point data of the target object in the target scene, which can more intuitively display the trajectory of the target object in the target scene .

FIG. 2 shows a schematic diagram of a system architecture to which the image processing method of the embodiment of the present application can be applied; as shown in FIG. 2 , the system architecture includes: an acquisition terminal 201 , a network 202 and a determination terminal 203 . In order to support an exemplary application, the acquisition terminal 201 and the determination terminal 203 establish a communication connection through the network 202, and the acquisition terminal 201 reports at least one of the first three-dimensional model of the target scene and the target object in the target scene to the determination terminal 203 through the network 202. The first track point data, determining that the terminal 203 responds to the first three-dimensional model of the target scene and at least one first track point data of the target object in the target scene, and according to the at least one first track point data, in the first three-dimensional model Displays the trajectory of the target object. Finally, the determination terminal 203 uploads the track of the target object to the network 202 and sends it to the acquisition terminal 201 through the network 202 .

As an example, the acquisition terminal 201 may include an image acquisition device, and the determination terminal 203 may include a visual processing device or a remote server capable of processing visual information. The network 202 may be connected in a wired or wireless manner. Wherein, when it is determined that the terminal 203 is a visual processing device, the acquisition terminal 201 can communicate with the visual processing device through a wired connection, such as performing data communication through a bus; Data exchange between the network and the remote server.

Or, in some scenarios, the acquisition terminal 201 may be a vision processing device with a video capture module, or a host with a camera. At this time, the image processing method of the embodiment of the present disclosure may be executed by the acquisition terminal 201 , and the above-mentioned system architecture may not include the network 202 and the determination terminal 203 .

In some embodiments of the present application, the image processing device may perform the following steps during the execution of step 102:

Step 1. Determine at least one first track point of the target object according to at least one first track point data.

In the embodiment of the present application, the track point is a point where the target object has appeared in the target scene, and the track point has a time stamp. The image processing device may determine the position of the track point according to the position in the first track point data, and determine the time stamp of the track point according to the time stamp in the first track point data. For example, the first track point data a includes point A and time stamp t1. At this time, the track point determined according to the first track point data a is point A, and the time stamp of the track point is t1.

The image processing device can obtain a first trajectory point according to a first trajectory point data, and correspondingly, can obtain at least one first trajectory point according to at least one first trajectory point data.

Step 2. Determine a first three-dimensional area including at least one first trajectory point from the first three-dimensional model.

Since the area of the target scene may be large, in order to better display the track of the target object in the target scene, the area where the target object has appeared may be emphatically displayed. Therefore, before the track of the target object is displayed in the first 3D model, the area where the target object has appeared can be determined from the first 3D model.

In some embodiments of the present application, the image processing device takes the area containing at least one first trajectory point of the target object as the area where the target object has appeared, and determines the at least one first trajectory containing the target object from the first three-dimensional model The area of points to get the first three-dimensional area.

Step 3: Displaying the first three-dimensional area in the first three-dimensional model according to a predetermined display manner, and displaying the trajectory of the target object in the first three-dimensional area.

In the embodiment of the present application, the predetermined display mode is a display mode emphatically displaying the first three-dimensional region. Highlighting the first three-dimensional area in the first three-dimensional model means that the display mode of the first three-dimensional area is different from that of the non-first three-dimensional area; wherein, the non-first three-dimensional area includes the first three-dimensional area in the first three-dimensional model except the first three-dimensional area outside the area. In some embodiments of the present application, the predetermined display manner includes one or more of the following: color highlighting, highlighting, and hovering.

In some embodiments of the present application, the predetermined display manner includes color highlighting. The image processing device converts the non-first three-dimensional area into a grayscale image, and retains the color of the first three-dimensional area, so as to highlight the first three-dimensional area. The display effect of highlighting the first three-dimensional area in the first three-dimensional model is that the first three-dimensional area is a color image, and the non-first three-dimensional area is a black and white image.

In some embodiments of the present application, the predetermined display manner includes highlight display. The image processing device highlights the first three-dimensional area, so as to realize highlighting the first three-dimensional area.

In some embodiments of the present application, the predetermined display manner includes floating display. The first three-dimensional model includes a head-up display (Head Up Display, HUD) layer, and the image processing device determines a display area corresponding to the first three-dimensional area from the HUD layer as a floating display area. The image processing device displays the first three-dimensional area in the floating display area.

The image processing device displays the track of the target object in the first three-dimensional area while highlighting the first three-dimensional area, thereby improving the display effect of the track of the target object.

In some embodiments of the present application, the image processing device may implement follow-up display of the target object by performing Step 1 to Step 3 .

For example, the target object is Zhang San, and the target scene is a shopping mall. The image processing device can highlight Zhang San's trajectory in the mall according to Zhang San's real-time position in the mall by performing steps 1 to 3, so as to achieve the effect of following and displaying Zhang San's trajectory in the mall.

In some embodiments of the present application, the image processing device performs the following steps during step 2:

Step 4. Obtain the size of the first three-dimensional area and the shape of the first three-dimensional area.

In the embodiment of the present application, the size of the first three-dimensional region is used to determine the area of the first three-dimensional region.

For example, the shape of the first three-dimensional area is a rectangle, and the size of the first three-dimensional area is a length of 100 pixel units and a width of 50 pixel units. At this time, the first three-dimensional area is a rectangle with a length of 100 pixel units and a width of 50 pixel units. the area surrounded by.

For another example, the shape of the first three-dimensional area is a circle, and the size of the first three-dimensional area is a radius of 50 pixel units. At this time, the first three-dimensional area is an area surrounded by a circle with a radius of 50 pixel units.

For another example, the shape of the first three-dimensional area is an isosceles trapezoid, the size of the first three-dimensional area is 50 pixel units for the upper base, 80 pixel units for the lower base, and 30 pixel units for the height. The area is an area surrounded by an isosceles trapezoid with an upper base of 50 pixel units, a lower base of 80 pixel units, and a height of 30 pixel units.

In some embodiments of the present application, the image processing device uses the size of the first three-dimensional area input by the user through the input component as the size of the first three-dimensional area.

In some embodiments of the present application, the image processing apparatus receives the size of the first three-dimensional area sent by the terminal as the size of the first three-dimensional area.

In some embodiments of the present application, the image processing device uses the shape of the first three-dimensional area input by the user through the input component as the shape of the first three-dimensional area.

In some embodiments of the present application, the image processing apparatus receives the shape of the first three-dimensional area sent by the terminal as the shape of the first three-dimensional area.

Step 5. Determine the centroid of at least one first trajectory point of the target object.

Since the distribution of the track points of the target object may not be uniform, determining the centroid of the track points of the target object may determine a concentration area of the track points of the target object.

For example, the track point area of the target object shown in FIG. 3 is the same as the track point area of the target object shown in FIG. 4 , and the number of track points in FIG. 3 is the same as that in FIG. 4 . However, since the track point distribution in FIG. 3 is different from that in FIG. 4 , the positions of the centroids of the track points in FIG. 3 and FIG. 4 are different.

In the track area of the target object shown in FIG. 3 , the track points of the target object are distributed unevenly, but the centroid is in the concentration area of the track points, that is, the center of mass of the track points of the target object can determine the concentration area of the track points of the target object.

In the embodiment of the present application, the image processing device determines the centroid of the trajectory point of the target object, and determines the coordinates of the centroid of the trajectory point of the target object in the coordinate system of the first three-dimensional model.

In some embodiments of the present application, the image processing device determines a polygon surrounding the track points of the target object according to the track points of the target object, wherein vertices of the polygon are all track points of the target object. The image processing device determines the centroid of the polygon as the centroid of the track point of the target object.

Step 6. Taking the centroid as the center of the first three-dimensional area, and determining the first three-dimensional area from the first three-dimensional model according to the size of the first three-dimensional area and the shape of the first three-dimensional area.

In this embodiment of the present application, the center of the first three-dimensional area may be the geometric center of the first three-dimensional area. Since the area of the first three-dimensional region is determined by the size of the first three-dimensional region, if the size of the first three-dimensional region is limited, the area of the first three-dimensional region is also limited. When the area of the first three-dimensional region is limited, the image processing device may use the center of mass as the center of the first three-dimensional region, so that the first three-dimensional region may contain more trajectory points.

For example, in the track point area of the target object shown in Figure 5, the centroid of the track point is in the concentrated area of the track point of the target object, and the image processing device uses the centroid as the center of the first three-dimensional area, so that the first three-dimensional area The concentration area of the track points is closer to the target object, so that the first three-dimensional area contains more track points.

In some embodiments of the present application, it is assumed that the shape of the first three-dimensional region is a rectangle. In Fig. 5, if the centroid is used as the center of the first three-dimensional area, the centroid area can be determined according to the area of the first three-dimensional area, and the centroid area now includes seven trajectory points (i.e., trajectory point A, trajectory point B, and trajectory point C , track point D, track point E, track point F, track point G). If the center of the track point area of the target object is taken as the center of the first three-dimensional area, the center area can be determined according to the area of the first three-dimensional area, and the center area includes one track point (ie track point F).

That is to say, when the area of the first three-dimensional area is determined, the center of mass is taken as the center of the first three-dimensional area, the first three-dimensional area contains 7 track points, and the center of the track point area of the target object is taken as the first three-dimensional area The center of the region, the first three-dimensional region contains 1 trajectory point. Obviously, taking the centroid as the center of the first three-dimensional area can make the first three-dimensional area contain more trajectory points.

After determining the center of the first three-dimensional area, the image processing device may determine the first three-dimensional area from the first three-dimensional model according to the center, the size of the first three-dimensional area, and the shape of the first three-dimensional area.

The image processing device further uses the first three-dimensional area as the first three-dimensional area, so that the first three-dimensional area includes more track points of the target object. In this way, the image processing device displays the track of the target object while displaying the first three-dimensional area, which can improve the display effect.

In some embodiments of the present application, at least one first track point includes a second track point, a third track point, a fourth track point, and at least one fifth track point.

In this implementation manner, the time stamp of the second track point, the time stamp of the third track point, the time stamp of the fourth track point, and the time stamp of at least one upper fifth track point decrease in sequence.

For example, at least one first track point includes: track point a, track point b, track point c, and track point d; wherein, the time stamp of track point a is earlier than the time stamp of track point b, and the time stamp of track point b is earlier The time stamp of track point d is earlier than the time stamp of track point c. At this time, track point d is the second track point, track point c is the third track point, track point b is the fourth track point, and track point a is the fifth track point, that is, the number of fifth track points is one.

For example, at least one first trajectory point includes: trajectory point a, trajectory point b, trajectory point c, trajectory point d, and trajectory point e; wherein, the timestamp of trajectory point a is earlier than the timestamp of trajectory point b, and trajectory point b The timestamp of the trajectory point is earlier than the timestamp of the trajectory point c, the timestamp of the trajectory point d is earlier than the timestamp of the trajectory point c, and the timestamp of the trajectory point e is earlier than the timestamp of the trajectory point d. At this time, track point e is the second track point, track point d is the third track point, track point c is the fourth track point, track point a and track point b are the fifth track point, that is, the fifth track point The quantity is 2.

In this embodiment, the image processing device performs the following steps during step 5:

Step 7. When the third trajectory point is located in the first region to be confirmed, determine the centroid of the first region to be confirmed as the centroid of at least one first trajectory point. The first region to be confirmed is a polygonal region, and the first region to be confirmed is a polygonal region. The vertices of the region are the second track point, the fourth track point, and at least one fifth track point.

Since the at least one first trajectory point is discontinuous in the time dimension, there is an error between the trajectory obtained according to the at least one first trajectory point and the actual trajectory of the target object. For example, as shown in Figure 6, suppose the real trajectory of the target object moving from trajectory point A to trajectory point B is curve AB, but the image processing device determines the position of trajectory point A and trajectory point B, according to the trajectory The trajectory obtained by point A and trajectory point B is line segment AB.

Therefore, in order to make at least one first track point area determined based on at least one first track point contain more actual tracks of the target object and improve the display accuracy of the track of the target object, the area containing at least one first track point can be The convex hull serves as at least one first trajectory point region.

For example, in Fig. 7, track point A, track point B, track point C, track point D and track point E are at least one first track point, wherein the time stamp of track point A is earlier than the time of track point B The time stamp of track point B is earlier than the time stamp of track point C, the time stamp of track point C is earlier than the time stamp of track point D, and the time stamp of track point D is earlier than the time stamp of track point E. The curve CD is the real trajectory of the target object moving from the trajectory point C to the trajectory point D.

If the track points are connected in ascending order of the time stamps of the track points, the obtained track point area is a polygon ABCDE. At this time, the curve CD is located outside the track point area, and if the centroid of the track point area is used as the centroid of at least one first track point, the accuracy of the centroid of at least one first track point will obviously be reduced.

If the convex hull including at least one first trajectory point is determined according to the at least one first trajectory point, the trajectory point area ABCE as shown in FIG. 8 can be obtained. At this time, the track point area includes the curve CD, and if the centroid of the track point area is used as the centroid of the at least one first track point, the accuracy of the centroid of the at least one first track point can be improved.

In the embodiment of the present application, the third track point is located in the first area to be confirmed, which means that the first area to be confirmed is a convex hull area including at least one first track point. At this time, the image processing device uses the first area to be confirmed as the whereabouts area of the target object in the first three-dimensional model, which can improve the accuracy of the track area, and further improve the display accuracy of the track of the target object. The image processing device further uses the centroid of the first region to be confirmed as the centroid of the at least one first trajectory point, which can improve the accuracy of the centroid of the at least one first trajectory point.

For example, in FIG. 8 , track point A and track point B are the fifth track point, track point C is the fourth track point, track point D is the third track point, and track point E is the second track point. The first area to be confirmed is the area enclosed by the polygon ABCE. Since the trajectory point D is located within the polygon ABCE, the image processing device takes the centroid of the polygon ABCE as the centroid of at least one first trajectory point.

For another example, in FIG. 9 , track point A is the fifth track point, track point B is the fourth track point, track point C is the third track point, and track point D is the second track point. The first area to be confirmed is the area enclosed by the triangle ABD. Since the trajectory point C is located in the triangle ABD, the image processing device takes the centroid of the triangle ABD as the centroid of at least one first trajectory point.

Step 8. When the third trajectory point is located outside the first region to be confirmed, determine the centroid of the second region to be confirmed as the centroid of at least one first trajectory point, the first region to be confirmed and the second region to be confirmed are both In a polygonal area, the vertices of the first area to be confirmed are the second track point, the fourth track point, and at least one fifth track point, and the vertices of the second area to be confirmed are the second track point, the third track point, and the fourth track point , at least one fifth trajectory point.

The third track point is located outside the first area to be confirmed, indicating that some track points are located outside the first area to be confirmed. At this time, if the first area to be confirmed is used as the track area of the target object in the first three-dimensional model, there will be a large error . Therefore, in order to reduce the error, the image processing device takes the second area to be confirmed as the trajectory area of the target object in the first three-dimensional model, and then uses the centroid of the second area to be confirmed as the centroid of at least one first trajectory point, which can improve at least one The accuracy of the centroid of the first trajectory point.

It should be understood that the second track point, the third track point, the fourth track point, and at least one fifth track point are all examples, wherein the second track point refers to the track point with the largest timestamp among the at least one first track point, The third track point refers to the track point with the second largest time stamp among at least one first track point, the fourth track point refers to the track point with the third largest time stamp among at least one first track point, and at least one fifth track point represents Refers to at least one track point of the first track point whose time stamp is smaller than the time stamp of the fourth track point.

In practical applications, if the trajectory of the target object is in the state of real-time update, then if the connection mode between the trajectory point with the smallest timestamp and the trajectory point with the third largest timestamp has been determined, according to this implementation mode It is determined that the track point with the largest time stamp is connected to the track point with the second largest time stamp, or the track point with the largest time stamp is connected with the track point with the third largest time stamp, so as to determine the track area of the target object.

For example, at t1 moment, at least one first track point includes track point A, track point B, track point C and track point D shown in Figure 9, that is, at t1 moment, track point A is the fifth track point, track point B is the fourth track point, track point C is the third track point, and track point D is the second track point. At time t1, at least one first track point area is the area surrounded by the triangle ABD in FIG. 9 .

After t2 seconds, at time t2, at least one first track point is added with track point E in FIG. 8 . At this time, track point A and track point B are both the fifth track point, track point C is the fourth track point, track point D is the third track point, and track point E is the second track point. The image processing device first determines whether the track point E is connected to the track point D or the track point C, and when it is determined that the track point E is connected to the track point C, the image processing device removes the connection between the track point C and the track point D. line, and then determine at least one first trajectory point area as the area enclosed by the polygon ABCE.

In some embodiments of the present application, the image processing device may also perform the following steps during step 1:

Step 9: Determine the n track point data with the largest time stamp among the at least one first track point data as at least one valid track point data.

Since there may be a large amount of first track point data, the processing of all the first track point data by the image processing device will bring about a large amount of data processing. Considering that the track point data of the target object has timeliness, the image processing device selects n track point data with the largest time stamp from at least one first track point data as at least one valid track point data. Therefore, while ensuring the validity of the track point data of the target object (ie, the first track point data), the quantity of the track point data of the target object is reduced, thereby reducing the amount of data processing brought by processing the track point data of the target object .

In the embodiment of the present application, n is a positive integer, and the value of n can be set according to actual requirements. For example, if the user wants to reduce the amount of data processing and increase the processing speed, the value of n can be set to a small value, such as setting the value of n to 80; the user wants to display more track point data of the target object, and the value of n can be set to a large value, such as The value of n is 200.

The image processing device determines the n track point data with the largest timestamp in at least one track point data, that is, determines the n first track point data closest to the target object, and takes the n first track point data as valid track point data, and obtains At least one valid track point data.

For example, assuming n=3, at least one first track point data includes track point data 1, track point data 2, track point data 3, track point data 4, and track point data 5, wherein the timestamp of track point data 1 is t1, the time stamp of track point data 2 is t2, the time stamp of track point data 3 is t3, the time stamp of track point data 4 is t4, and the time stamp of track point data 5 is t5. If t1 is earlier than t2, t2 is earlier than t3, t3 is earlier than t4, and t4 is earlier than t5, the valid track point data includes track point data 1, track point data 2 and track point data 3.

Step 10. Determine at least one first track point of the target object according to at least one valid track point data.

The image processing device can determine a valid track point according to the position and time stamp in the valid track point data. The image processing device can determine at least one valid track point according to the position and time stamp in the at least one valid track point data. After obtaining at least one valid trajectory point, the image processing device uses the at least one valid trajectory point as at least one first trajectory point.

For example, at least one valid track point data includes track point data 1, track point data 2 and track point data 3, track point data 1 includes position 1 and time stamp t1, track point data 2 includes position 2 and time stamp t2, track point Data 3 includes position 3 and time stamp t3. The image processing device can obtain an effective trajectory point 1 according to the trajectory point data 1; wherein, the effective trajectory point 1 is that the target object appears at position 1 at t1. The image processing device can obtain the effective trajectory point 2 according to the trajectory point data 2, wherein the effective trajectory point 2 is that the target object appears at the position 2 at t2. The image processing device can obtain the effective trajectory point 3 according to the trajectory point data 3, wherein the effective trajectory point 3 is that the target object appears at the position 3 at t3.

By performing step 9 and step 10, the image processing device can reduce the amount of data processing and increase the processing speed.

In some embodiments of the present application, the trajectory of the target object includes a first trajectory and a second trajectory, the first trajectory is the trajectory between the trajectory point with the mth largest timestamp and the trajectory point with the largest timestamp, and the second trajectory is Among the tracks of the target object except the first track, the display mode of the first track is different from the display mode of the second track.

In the embodiment of the present application, the first trajectory is a trajectory between the trajectory point with the mth largest timestamp and the trajectory point with the largest timestamp, that is, the first trajectory includes the m trajectory points with the largest timestamp in the trajectory of the target object.

In some embodiments of the present application, m is a positive integer equal to or greater than 2. For example, assuming m=2, the trajectory of the target object includes trajectory point 1, trajectory point 2, trajectory point 3, trajectory point 4, and trajectory point 5; wherein, the timestamp of trajectory point 1 is t1, and the timestamp of trajectory point 2 is t2, the time stamp of track point 3 is t3, the time stamp of track point 4 is t4, and the time stamp of track point 5 is t5. If t1 is earlier than t2, t2 is earlier than t3, t3 is earlier than t4, and t4 is earlier than t5, then the first track includes track point 1 and track point 2.

Since the trajectory does not have direction information, in order to facilitate the user to distinguish the moving direction of the target object, the image processing device distinguishes the first trajectory from the second trajectory by displaying the first trajectory and the second trajectory in different ways.

In some embodiments of the present application, the display mode includes one of the following: color, whether the trajectory line is virtual or solid, and whether the trajectory line carries an arrow.

In some embodiments of the present application, if the display manner includes color, the image processing device displays the first trajectory in blue, and displays the second trajectory in red. In this way, the user can distinguish the first track from the second track by color, and then can determine that the moving direction of the target object is from the second track to the first track.

In some embodiments of the present application, when the display mode includes virtual and solid trajectory lines, the image processing device displays the first trajectory as a solid line and displays the second trajectory as a dotted line. In this way, the user distinguishes the first trajectory from the second trajectory through the virtual reality of the trajectory lines, and then can determine the moving direction of the target object.

In some embodiments of the present application, when the display method includes whether the trajectory line carries an arrow, the image processing device displays the first trajectory with a line carrying an arrow, and the arrow points to the trajectory point with the largest time stamp, and displays the first trajectory as a line without an arrow. The lines show the second locus. In this way, the user distinguishes the first trajectory from the second trajectory according to whether the trajectory line carries an arrow, and then can determine the moving direction of the target object.

In some embodiments of the present application, the first three-dimensional model includes a first floor and a second floor, and the height of the first floor is different from that of the second floor. The trajectory of the target object is within the first floor.

In the embodiment of the present application, the first floor and the second floor are any two floors in the first three-dimensional model, and the heights of the first floor and the second floor are different, that is, the first floor and the second floor are two different floors .

In the case that the trajectory of the target object is on the first floor, the image processing device displays the trajectory of the target object in the first three-dimensional model, that is, the trajectory of the target object is displayed on the first floor, that is, the image processing device displays the trajectory of the target object During trajectory, the first floor is displayed in the first 3D model. For example, the image processing device highlights the first three-dimensional area in the first floor, and displays the track of the target object in the first three-dimensional area.

In some embodiments of the present application, the image processing device displays the first floor in the first three-dimensional model by hiding floors in the first three-dimensional model except the first floor.

For example, the first three-dimensional model has five floors in total, and the first floor is the fourth floor in the first three-dimensional model. The image processing device may hide the fifth floor in the first three-dimensional model, and then display the first floor while displaying the shape of the first three-dimensional model. Since the first floor is the fourth floor, the first floor, the second floor and the third floor will be covered when the first floor is displayed, so that when the first 3D model is displayed, the first floor, the second floor and the third floor The interference caused by the third floor to the display of the first floor, thereby improving the display effect of the first floor. In some embodiments of the present application, the display effect can be seen in FIG. 10 .

In some embodiments of the present application, the image processing device also performs the following steps:

Step 11, in a case where it is determined that the target object appears on the second floor, display the second floor in the first three-dimensional model.

In order to track and display the trajectory of the target object, the image processing device should display the second floor in the first three-dimensional model in case the target object appears on the second floor. In some embodiments of the present application, the image processing device switches the displayed content in the first three-dimensional model from the first floor to the second floor when it is determined that the target object appears on the second floor.

For example, as shown in Figure 10, the image processing device displays the first floor in the first three-dimensional model, and the image processing device switches the display content of the first three-dimensional model from the first floor to the second floor to display the content shown in Figure 11 .

In some embodiments of the present application, the image processing device can improve the display effect of the trajectory of the target object by switching the display content in the first three-dimensional model from the first floor to the second floor.

In some embodiments of the present application, the image processing device determines that the target object appears on the second floor when an instruction that the target object appears on the second floor is detected.

For example, the user may input an instruction that the target object appears on the second floor to the image processing device through the input component, and the image processing device determines that the target object appears on the second floor when the instruction is detected.

In some embodiments of the present application, the image processing device determines that the target object appears on the second floor when it is determined that the image captured by the camera on the second floor contains the target object.

In some embodiments of the present application, the image processing device obtains the predicted trajectory of the target object according to the trajectory of the target object and the moving direction of the target object. The image processing device determines that the target object appears on the second floor when it is determined according to the predicted trajectory that the target object appears on the second floor.

For example, the image processing device determines that the target object appears on the second floor at time t2 according to the predicted trajectory, then the image processing device determines that the target object appears on the second floor at time t2, and then displays the second floor in the first three-dimensional model at time t2 .

In some embodiments of the present application, before the image processing device determines that the target object appears on the second floor, the following steps are further performed:

Step 12. Acquire the first image containing the target object.

In this embodiment of the present application, the target object may be any object. In some embodiments of the present application, the target object includes one of the following: a human body, a human face, and a vehicle.

In some embodiments of the present application, the image processing apparatus acquires the first image containing the target object by receiving the user's input of the image containing the target object through the input component.

In some embodiments of the present application, the image processing apparatus receives the image containing the target object sent by the terminal to obtain a first image containing the target object.

In some embodiments of the present application, the image processing device is equipped with a camera. The image processing device obtains a first image including the target object by shooting the target object using the camera.

In some embodiments of the present application, there is a communication connection between the image processing device and the camera. The camera captures the target object to obtain an image containing the target object, and the image processing device obtains the first image containing the target object from the camera through the communication connection.

After step 12 is executed, the image processing device determines that the target object appears on the second floor by performing the following steps:

Step 13. Determine that the second image contains the target object by comparing the first image with the second image, and the second image is collected by the camera on the second floor.

In the embodiment of the present application, comparing the first image with the second image refers to comparing the similarity between the first image and the second image to determine whether the second image contains the target object.

In some embodiments of the present application, when the target object is a human face, the image processing device can determine whether the second image contains the target object by comparing the face of the first image with the second image;

In some embodiments of the present application, when the target object is a human body, the image processing device can determine whether the second image contains the target object by comparing the first image with the second image for the human body;

In some embodiments of the present application, when the target object is a vehicle, the image processing device can determine whether the second image contains the target object by comparing the first image with the second image for the vehicle.

Since the second image is an image collected by the camera on the second floor, if it is determined that the second image contains the target object, it is determined that the target object appears on the second floor.

Based on the technical solution provided by the embodiment of the present application, the embodiment of the present application also provides an application scenario.

In order to improve social security management capabilities and maintain a good social security environment, cameras are installed in more and more places. When relevant personnel need to find a target person, they can determine the trajectory of the target person based on the images collected by the surveillance camera. In order to facilitate the user to view the trajectory of the target person, the trajectory of the target person can be displayed in the scene. Based on the technical solutions provided by the embodiments of the present application, the trajectory of the target person can be displayed in the three-dimensional model of the building.

For example, the relevant personnel want to track Zhang San's track in Building A, and the relevant personnel input Zhang San's face image to the server (ie, the image processing device). The server further compares Zhang San's face image with the image captured by the camera in Building A in real time to determine whether Zhang San appears in Building A. The server determines Zhang San's track point data in Building A according to the face comparison result. For example, the image processing device determines that image a contains Zhang San through the face comparison result, and image a is collected by camera b at time t1, then the image processing device can obtain a track point data according to image a, and the track point data includes Zhang San 3 appears at the position captured by camera b, and the time of appearance is t1.

The image processing device processes Zhang San's trajectory point data in Building A and the 3D model of Building A according to the technical solution provided by the embodiment of the present application, so as to display Zhang San's trajectory in the 3D model of Building A.

In some embodiments of the present application, in order to obtain Zhang San's trajectory in real time, the image processing device may highlight the first three-dimensional area containing Zhang San's trajectory points in Building A based on the technical solution provided by the embodiment of the present application, and The trajectory of Zhang San is displayed in the first three-dimensional area.

In some embodiments of the present application, when it is determined that Zhang San moves from floor c to floor d of building A, the image processing device switches the floor displayed in the 3D model of building A from floor c to floor d.

In some embodiments of the present application, when the number of first track point data is greater than 1, the image processing device acquires at least one first track point data of the target object in the target scene by performing the following steps:

Step 14, acquiring at least two second track point data and a distance threshold of the target object in the target scene.

In this embodiment, the number of the second trajectory point data may be 2 or greater than 2. The second track point data includes a location and a time stamp. The distance threshold is a positive number, and the value of the distance threshold can be set according to actual requirements.

Step 15. Determine the distance between adjacent track point data sets, where the adjacent track point data sets include two second track point data with adjacent time stamps.

In this embodiment, the adjacent track point data set includes two second track point data with adjacent time stamps, and the distance between the adjacent track point data sets refers to the distance between the two track point data in the adjacent track point data set. distance.

For example, at least two second track point data include: track point data a, track point data b, and track point data c, wherein the time stamp of track point data a is t1, the time stamp of track point data b is t2, and the track The timestamp of point data c is t3. If t1 is greater than t2, and t2 is greater than t3, the adjacent track point data set A and the adjacent track point data set B can be determined based on at least two second track point data, wherein the adjacent track point data set A includes track point data a and track point data b, the adjacent track point data set B includes track point data b and track point data c.

If the position of track point data a is p1, the position of track point data b is p2, the position of track point data c is p3, and the distance between p1 and p2 is d1, and the distance between p2 and p3 is d2, then The distance between the adjacent track point data set A is d1, and the distance between the adjacent track point data set B is d2.

In order to reduce the data processing amount of the image processing device, before processing at least one second trajectory point data of the target object in the scene, the second trajectory point data with a closer distance may be combined. In some embodiments of the present application, the image processing device realizes the merging of the second track point data with a shorter distance by deleting the second track point data with a smaller time stamp.

In this embodiment, the image processing device judges whether the distance between adjacent track point data sets is relatively short based on the distance threshold, and then obtains more than one first track point data based on the judgment result and at least two first track point data. In some embodiments of the present application, the image processing device obtains more than one first track point data by performing one of steps 16 and 17:

Step 16. When the distance is less than the distance threshold, the old track point data is removed from at least two second track point data to obtain at least one first track point data, and the old track point data is the collection time of adjacent track point data. Poke the smallest second track point data.

Step 17. When the distance is less than the distance threshold, use at least two second track point data as more than one first track point data.

Those skilled in the art can understand that in the above method of specific implementation, the writing order of each step does not mean a strict execution order and constitutes any limitation on the implementation process. The specific execution order of each step should be based on its function and possible The inner logic is OK.

The method of the embodiment of the present application has been described in detail above, and the device of the embodiment of the present application is provided below.

Please refer to FIG. 12. FIG. 12 is a schematic structural diagram of an image processing device provided by an embodiment of the present application. The image processing device 1 includes: an acquisition unit 11 and a processing unit 12, wherein:

The acquisition unit 11 is configured to acquire a first three-dimensional model of a target scene and at least one first trajectory point data of a target object in the target scene;

The processing unit 12 is configured to display the trajectory of the target object in the first three-dimensional model according to the at least one first trajectory point data.

In some embodiments of the present application, the processing unit 12 includes: a first processing subunit configured to determine at least one first track point of the target object according to the at least one first track point data; The processing subunit is configured to determine from the first three-dimensional model a first three-dimensional area containing the at least one first trajectory point; the third processing subunit is configured to display in a predetermined manner in the first three-dimensional model displaying the first three-dimensional area, and displaying the track of the target object in the first three-dimensional area;

In some embodiments of the present application, the at least one first track point includes a second track point, a third track point, a fourth track point, at least one fifth track point, the timestamp of the second track point, The timestamp of the third trajectory point, the timestamp of the fourth trajectory point, and the timestamp of the one or more fifth trajectory points decrease in sequence; the second processing subunit is further configured to When the third trajectory point is located in the first area to be confirmed, the centroid of the first area to be confirmed is determined as the centroid of the at least one first trajectory point, and the first area to be confirmed is a polygonal area, so The apex of the first area to be confirmed is the second track point, the fourth track point, and the at least one fifth track point; when the third track point is located outside the first area to be confirmed, Determine the centroid of the second area to be confirmed as the centroid of the at least one first track point, the first area to be confirmed and the second area to be confirmed are polygonal areas, and the vertex of the first area to be confirmed is The second track point, the fourth track point, and the at least one fifth track point, the vertices of the second area to be confirmed are the second track point, the third track point, the first Four trajectory points, the at least one fifth trajectory point.

In some embodiments of the present application, when the number of the first trajectory point data is greater than 1, the acquisition unit 11 includes: a first acquisition subunit configured to acquire At least two second track point data and a distance threshold in the scene; determine the distance between adjacent track point data sets, and the adjacent track point data sets include two second track point data with adjacent time stamps; When the distance is less than the distance threshold, the old track point data is removed from the at least two second track point data to obtain the at least one first track point data, and the old track point data is all The second track point data with the smallest time stamp in the adjacent track point data sets; or the at least two second track point data as the one or more first track point data.

In some embodiments of the present application, the determining that the target object appears before the second floor, the acquisition unit is further configured to acquire a first image containing the target object; the processing unit is further It is configured to determine that the second image contains the target object by comparing the first image with a second image, and the second image is collected by a camera on the second floor.

The image processing device displays the trajectory of the target object in the first three-dimensional model according to at least one first trajectory point data of the target object in the target scene, which can more intuitively display the trajectory of the target object in the target scene.

In some embodiments, the functions or modules included in the apparatus provided in the embodiments of the present application can be used to execute the methods described in the above method embodiments, and the specific implementation can refer to the descriptions of the above method embodiments.

FIG. 13 is a schematic diagram of a hardware structure of an image processing device provided by an embodiment of the present application. The image processing device 2 includes a processor 21 , a memory 22 , an input device 23 and an output device 24 . The processor 21, the memory 22, the input device 23 and the output device 24 are coupled through connectors, and the connectors include various interfaces, transmission lines or buses, etc., which are not limited in this embodiment of the present application. It should be understood that in various embodiments of the present application, coupling refers to interconnection in a specific way, including direct connection or indirect connection through other devices, for example, connection through various interfaces, transmission lines, and buses.

The processor 21 can be one or more graphics processing units (Graphics Processing Unit, GPU), and in the case where the processor 21 is a GPU, the GPU can be a single-core GPU or a multi-core GPU. In some embodiments of the present application, the processor 21 may be a processor group composed of multiple GPUs, and the multiple processors are coupled to each other through one or more buses. In some embodiments of the present application, the processor may also be other types of processors, etc., which are not limited in this embodiment of the present application.

The memory 22 can be used to store computer program instructions and various computer program codes including program codes for implementing the solutions of the present application. Optionally, the memory includes but is not limited to Random Access Memory (Random Access Memory, RAM), Read Only Memory (Read Only Memory, ROM), Erasable Programmable Read Only Memory (Erasable Programmable Read Only Memory, EPROM) , or portable read-only memory (Compact Disc-Read Only Memory, CD-ROM), which is used for related instructions and data.

The input device 23 is used for inputting data and/or signals and the output device 24 is used for outputting data and/or signals. The input device 23 and the output device 24 can be independent devices, or an integrated device.

It can be understood that, in the embodiment of the present application, the memory 22 can be used not only to store relevant instructions, but also to store relevant data, for example, the memory 22 can be used to store the first three-dimensional model and at least one first trajectory point data obtained through the input device 23 , the embodiment of the present application does not limit the specific data stored in the memory.

It can be understood that FIG. 13 only shows a simplified design of an image processing device. In practical applications, the image processing device may also include other necessary components, including but not limited to any number of input/output devices, processors, memories, etc., and all image processing devices that can implement the embodiments of the present application are included in this within the scope of the application.

An embodiment of the present application provides an electronic device, including: a processor, a sending device, an input device, an output device, and a memory, the memory is used to store computer program codes, and the computer program codes include computer instructions. When the computer executes the computer instructions, the electronic device executes the image processing method as described in any one of the above embodiments.

Embodiments of the present disclosure also provide another computer program product, including a computer-readable storage medium storing program codes, where instructions included in the program codes can be configured to execute the image processing method described in the above method embodiments.

Those skilled in the art can appreciate that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed by hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may use different methods to implement the described functions for each specific application, but such implementation should not be regarded as exceeding the scope of the present application.

Those skilled in the art can clearly understand that for the convenience and brevity of the description, the specific working process of the above-described system, device and unit can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here. Those skilled in the art can also clearly understand that the descriptions of each embodiment of the present application have their own emphases. For the convenience and brevity of description, the same or similar parts may not be repeated in different embodiments. Therefore, in a certain embodiment For parts not described or not described in detail, reference may be made to the descriptions of other embodiments.

In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods may be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or May be integrated into another system, or some features may be ignored, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to realize the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit.

In the above embodiments, all or part of them may be implemented by software, hardware, firmware or any combination thereof. When implemented using software, it may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the processes or functions according to the embodiments of the present application will be generated in whole or in part. The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in or transmitted via a computer-readable storage medium. The computer instructions can be transmitted from a website site, computer, server or data center to Another website site, computer, server or data center for transmission. The computer-readable storage medium may be any available medium that can be accessed by a computer, or a data storage device such as a server or a data center integrated with one or more available media. The available medium may be a magnetic medium, (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a digital versatile disc (Digital Versatile Disc, DVD)), or a semiconductor medium (for example, a solid state disk (Solid State Disk, SSD) )Wait.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments are realized. The processes can be completed by computer programs to instruct related hardware. The programs can be stored in computer-readable storage media. When the programs are executed , may include the processes of the foregoing method embodiments. The aforementioned storage medium includes: ROM or RAM, a magnetic disk or an optical disk, and other various media that can store program codes.

Industrial Applicability

The embodiment of the present application discloses an image processing method, device, equipment, medium and program. The method is executed by an electronic device, and the method includes: acquiring a first three-dimensional model of a target scene and at least one first track point data of a target object in the target scene; according to the at least one first track point data, in The trajectory of the target object is displayed in the first three-dimensional model.

Claims

An image processing method, the method is executed by an electronic device, the method comprising:

Acquiring a first three-dimensional model of a target scene and at least one first trajectory point data of a target object in the target scene;

A trajectory of the target object is displayed in the first three-dimensional model according to the at least one first trajectory point data.
The method according to claim 1, wherein the displaying the trajectory of the target object in the first three-dimensional model according to the at least one first trajectory point data comprises:

determining at least one first track point of the target object according to the at least one first track point data;

determining a first three-dimensional region containing the at least one first trajectory point from the first three-dimensional model;

The first three-dimensional area is displayed in a predetermined display manner in the first three-dimensional model, and the track of the target object is displayed in the first three-dimensional area.
The method according to claim 2, wherein said determining a first three-dimensional area containing said at least one first trajectory point from said first three-dimensional model comprises:

Acquiring the size of the first three-dimensional area and the shape of the first three-dimensional area;

determining a centroid of the at least one first trajectory point;

Taking the centroid as the center of the first three-dimensional area, and determining the first three-dimensional area from the first three-dimensional model according to the size of the first three-dimensional area and the shape of the first three-dimensional area.
The method according to claim 3, wherein the at least one first track point includes a second track point, a third track point, a fourth track point, and at least one fifth track point, and the time of the second track point stamp, the timestamp of the third trajectory point, the timestamp of the fourth trajectory point, and the timestamp of the at least one fifth trajectory point decrease in turn;

The determining the centroid of the at least one first trajectory point includes:

In the case where the third track point is located in the first area to be confirmed, the centroid of the first area to be confirmed is determined as the centroid of the at least one first track point, and the first area to be confirmed is a polygonal area , the vertices of the first area to be confirmed are the second track point, the fourth track point, and the at least one fifth track point;

When the third track point is located outside the first area to be confirmed, determine the centroid of the second area to be confirmed as the center of mass of the at least one first track point, the first area to be confirmed and the second area to be confirmed The areas to be confirmed are all polygonal areas, the vertices of the first area to be confirmed are the second track point, the fourth track point, and the at least one fifth track point, and the vertices of the second area to be confirmed are are the second track point, the third track point, the fourth track point, and the at least one fifth track point.
The method according to any one of claims 2 to 4, wherein said determining at least one first track point of said target object according to said at least one first track point data comprises:

Determining the n track point data with the largest timestamp in the at least one first track point data as at least one valid track point data; the n is a positive integer;

At least one first track point of the target object is determined according to the at least one valid track point data.
The method according to any one of claims 1 to 5, wherein, when the number of the first track point data is greater than 1, said acquiring at least one first track of the target object in the target scene point data, including:

Acquiring at least two second trajectory point data and a distance threshold of the target object in the target scene;

determining the distance between adjacent track point data sets, the adjacent track point data sets including two of the second track point data with adjacent time stamps;

When the distance is less than the distance threshold, the old track point data is removed from the at least two second track point data to obtain the at least one first track point data, and the old track point data is The second track point data with the smallest time stamp in the adjacent track point data set; or the at least two second track point data as the one or more first track point data.
The method according to any one of claims 1 to 6, wherein the trajectory of the target object includes a first trajectory and a second trajectory, and the first trajectory is the trajectory point with the mth largest timestamp and the largest timestamp The trajectory between the trajectory points, the second trajectory is a trajectory other than the first trajectory in the trajectory of the target object, and the display method of the first trajectory is different from the display method of the second trajectory ; The m is a positive integer equal to or greater than 2.
The method according to any one of claims 1 to 7, wherein the first three-dimensional model includes a first floor and a second floor, and the height of the first floor is different from the height of the second floor, so The track of the target object is in the first floor, the method also includes:

In case it is determined that the target object is present on the second floor, the second floor is displayed in the first three-dimensional model.
The method according to claim 8, wherein said determining that said target object appears before said second floor, said method further comprises:

acquiring a first image containing the target object;

The determining that the target object appears on the second floor includes:

It is determined that the second image contains the target object by comparing the first image with the second image, and the second image is collected by a camera on the second floor.
An image processing device, the device comprising:

An acquisition unit configured to acquire a first three-dimensional model of a target scene and at least one first trajectory point data of a target object in the target scene;

A processing unit configured to display the trajectory of the target object in the first three-dimensional model according to the at least one first trajectory point data.
The device according to claim 10, wherein the processing unit comprises:

The first processing subunit is configured to determine at least one first track point of the target object according to the at least one first track point data;

A second processing subunit configured to determine a first three-dimensional area containing the at least one first trajectory point from the first three-dimensional model;

The third processing subunit is configured to display the first three-dimensional area in the first three-dimensional model in a predetermined display manner, and display the trajectory of the target object in the first three-dimensional area.
The device according to claim 11, wherein the second processing subunit is further configured to:

Acquiring the size of the first three-dimensional area and the shape of the first three-dimensional area;

determining a centroid of the at least one first trajectory point;

Taking the centroid as the center of the first three-dimensional area, and determining the first three-dimensional area from the first three-dimensional model according to the size of the first three-dimensional area and the shape of the first three-dimensional area.
The device according to claim 12, wherein the at least one first track point includes a second track point, a third track point, a fourth track point, and at least one fifth track point, and the time of the second track point stamp, the timestamp of the third trajectory point, the timestamp of the fourth trajectory point, and the timestamp of the at least one fifth trajectory point decrease in sequence, and the second processing subunit is further configured to:

In the case where the third track point is located in the first area to be confirmed, the centroid of the first area to be confirmed is determined as the centroid of the at least one first track point, and the first area to be confirmed is a polygonal area , the vertices of the first area to be confirmed are the second track point, the fourth track point, and the at least one fifth track point;

When the third track point is located outside the first area to be confirmed, determine the centroid of the second area to be confirmed as the center of mass of the at least one first track point, the first area to be confirmed and the second area to be confirmed The areas to be confirmed are all polygonal areas, the vertices of the first area to be confirmed are the second track point, the fourth track point, and the at least one fifth track point, and the vertices of the second area to be confirmed are are the second track point, the third track point, the fourth track point, and the at least one fifth track point.
The device according to any one of claims 11 to 13, wherein the first processing subunit is further configured to:

Determining the n track point data with the largest timestamp in the at least one first track point data as at least one valid track point data; the n is a positive integer;

At least one first track point of the target object is determined according to the at least one valid track point data.
The device according to any one of claims 10 to 14, wherein, when the number of the first trajectory point data is greater than 1, the acquisition unit includes: a first acquisition subunit configured to:

Acquiring at least two second trajectory point data and a distance threshold of the target object in the target scene;

determining the distance between adjacent track point data sets, the adjacent track point data sets including two of the second track point data with adjacent time stamps;

When the distance is less than the distance threshold, the old track point data is removed from the at least two second track point data to obtain the at least one first track point data, and the old track point data is The second track point data with the smallest time stamp in the adjacent track point data set; or the at least two second track point data as the one or more first track point data.
The device according to any one of claims 10 to 15, wherein the trajectory of the target object includes a first trajectory and a second trajectory, and the first trajectory is the trajectory point with the mth largest timestamp and the largest timestamp The trajectory between the trajectory points, the second trajectory is a trajectory other than the first trajectory in the trajectory of the target object, and the display method of the first trajectory is different from the display method of the second trajectory ; The m is a positive integer equal to or greater than 2.
The device according to any one of claims 10 to 16, wherein the first three-dimensional model includes a first floor and a second floor, and the height of the first floor is different from the height of the second floor, so The track of the target object is in the first floor, the device further includes: a display unit configured to:

In case it is determined that the target object is present on the second floor, the second floor is displayed in the first three-dimensional model.
The device according to claim 17, wherein it is determined that the target object appears before the second floor, and the acquisition unit is further configured to acquire a first image containing the target object;

The processing unit is further configured to determine that the second image contains the target object by comparing the first image with a second image, and the second image is collected by a camera on the second floor.
An electronic device, comprising: a processor and a memory, the memory is used to store computer program codes, the computer program codes include computer instructions, and when the processor executes the computer instructions, the electronic device executes A method as claimed in any one of claims 1 to 9.
A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, the computer program includes program instructions, and when the program instructions are executed by a processor, the processor is caused to perform the claims The method described in any one of 1 to 9.
A computer program product, the computer program product comprising a computer program or an instruction, when the computer program or instruction is run on an electronic device, the electronic device is made to execute any one of claims 1 to 9 described method.