CN110619807B - Method and device for generating global thermodynamic diagram - Google Patents
Method and device for generating global thermodynamic diagram Download PDFInfo
- Publication number
- CN110619807B CN110619807B CN201810638146.7A CN201810638146A CN110619807B CN 110619807 B CN110619807 B CN 110619807B CN 201810638146 A CN201810638146 A CN 201810638146A CN 110619807 B CN110619807 B CN 110619807B
- Authority
- CN
- China
- Prior art keywords
- target object
- coordinate
- dimensional space
- dimensional
- determining
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
- G09B29/00—Maps; Plans; Charts; Diagrams, e.g. route diagram
- G09B29/003—Maps
- G09B29/006—Representation of non-cartographic information on maps, e.g. population distribution, wind direction, radiation levels, air and sea routes
- G09B29/007—Representation of non-cartographic information on maps, e.g. population distribution, wind direction, radiation levels, air and sea routes using computer methods
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Mathematical Physics (AREA)
- Ecology (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Computer Hardware Design (AREA)
- Business, Economics & Management (AREA)
- Educational Administration (AREA)
- Educational Technology (AREA)
- General Physics & Mathematics (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a method and a device for generating a global thermodynamic diagram, and relates to the technical field of computers. One embodiment of the method comprises: acquiring a shot picture of a three-dimensional space, and determining a target object on the shot picture and a first coordinate of the target object on the shot picture; mapping the first coordinate according to the mapping model, and outputting a second coordinate of the target object on a two-dimensional plane graph of the three-dimensional space; and generating a global thermodynamic diagram of the three-dimensional space according to the second coordinates and the two-dimensional plane map. The embodiment associates each shooting device in the three-dimensional space with the two-dimensional plane graph of the three-dimensional space by using the mapping model, so that the target object in the shot picture can be mapped onto the two-dimensional plane graph, and the global thermodynamic diagram of the three-dimensional space can be generated.
Description
Technical Field
The invention relates to the technical field of computers, in particular to a method and a device for generating a global thermodynamic diagram.
Background
In the retail industry, the offline retail stores identify pedestrians and commodities by installing cameras, but if a thermodynamic diagram of the pedestrians in the whole plane of the stores is generated, a plurality of harsh environmental requirements are required for arranging the cameras. Meanwhile, with the continuous development of big data technology, the research on user requirements by utilizing thermodynamic diagrams becomes a big trend of social development. Therefore, how to generate the global thermodynamic diagram of the pedestrian in the shop has important social value.
The thermodynamic diagram is a diagram showing a page area enthusiastic to visitors and a geographical area where the visitors are located in a special highlight form. In the current method for generating the global thermodynamic diagram of the three-dimensional space, only the thermal analysis is performed on the image shot by a single camera, and the thermal region is directly drawn on the shot image, the thermodynamic diagram formed by each camera is relatively independent, and the global thermodynamic diagram of the three-dimensional space is not formed.
In the process of implementing the invention, the inventor finds that the prior art has at least the following problems: the camera parameter measuring step is complicated, the measured data is inaccurate, and the error is large; in the prior art, the thermodynamic diagrams of multiple cameras are displayed in a split screen mode, each camera is relatively independent, so that the multiple cameras in the same three-dimensional space cannot be matched with each other, and the global thermodynamic diagram of the three-dimensional space cannot be formed.
Disclosure of Invention
In view of this, embodiments of the present invention provide a method and an apparatus for generating a global thermodynamic diagram, which are capable of associating each shooting device in a three-dimensional space with a two-dimensional plane map of the three-dimensional space by using a mapping model, and mapping a target object in a shot picture onto the two-dimensional plane map, so as to generate the global thermodynamic diagram of the three-dimensional space.
To achieve the above object, according to an aspect of an embodiment of the present invention, there is provided a method of generating a global thermodynamic diagram.
The method for generating the global thermodynamic diagram comprises the following steps: acquiring a shot picture of a three-dimensional space, and determining a target object on the shot picture and a first coordinate of the target object on the shot picture; mapping the first coordinate according to a mapping model, and outputting a second coordinate of the target object on a two-dimensional plane graph of the three-dimensional space; and generating a global thermodynamic diagram of the three-dimensional space according to the second coordinate and the two-dimensional plane diagram.
Optionally, determining the target object on the shot picture and the first coordinate of the target object on the shot picture comprises: and detecting the shot picture based on a target detection model, and determining a target object on the shot picture and coordinates of the target object on a coordinate system of the shot picture.
Optionally, after determining the target object on the shot picture, the method further includes: and identifying the target object based on an identification model, then judging whether the target object exists in a storage unit, if so, deleting the target object, and otherwise, storing the target object in the storage unit.
Optionally, before mapping the first coordinate according to a mapping model, the method further comprises: for each shooting device in the three-dimensional space, acquiring a focal length of the shooting device and a coordinate of the shooting device on a two-dimensional plane map of the three-dimensional space; determining a depression angle of the shooting equipment and a horizontal rotation angle of the shooting equipment by utilizing the principle of few sampling points and based on a regression fitting algorithm; determining the mapping model of the shooting device according to the parameters of the mapping model corresponding to the shooting device, wherein the parameters comprise: the focal length of the shooting device, the coordinates of the shooting device on a two-dimensional plane diagram of a three-dimensional space, the depression angle of the shooting device and the horizontal rotation angle of the shooting device.
Optionally, mapping the first coordinate according to a mapping model, and outputting a second coordinate of the target object on a two-dimensional plane map of the three-dimensional space includes: inputting the first coordinate into the mapping model, and determining a third coordinate of the target object in a three-dimensional coordinate system of a shooting device; determining a fourth coordinate of the target object in a three-dimensional coordinate system of the three-dimensional space according to the third coordinate; and determining a second coordinate of the target object on the two-dimensional plane map of the three-dimensional space according to the fourth coordinate.
To achieve the above object, according to another aspect of the embodiments of the present invention, there is provided an apparatus for generating a global thermodynamic diagram.
The device for generating the global thermodynamic diagram comprises the following components: the acquisition module is used for acquiring a shot picture of a three-dimensional space and determining a target object on the shot picture and a first coordinate of the target object on the shot picture; the output module is used for mapping the first coordinate according to a mapping model and outputting a second coordinate of the target object on a two-dimensional plane graph of the three-dimensional space; and the generating module is used for generating a global thermodynamic diagram of the three-dimensional space according to the second coordinate and the two-dimensional plane diagram.
Optionally, the obtaining module is further configured to: and detecting the shot picture based on a target detection model, and determining a target object on the shot picture and coordinates of the target object on a coordinate system of the shot picture.
Optionally, the apparatus further includes a determining module, configured to: and identifying the target object based on an identification model, then judging whether the target object exists in a storage unit, if so, deleting the target object, and otherwise, storing the target object in the storage unit.
Optionally, the output module is further configured to: for each shooting device in the three-dimensional space, acquiring a focal length of the shooting device and a coordinate of the shooting device on a two-dimensional plane graph of the three-dimensional space; determining a depression angle of the shooting equipment and a horizontal rotation angle of the shooting equipment by using the principle of few sampling points and based on a regression fitting algorithm; determining a mapping model of the shooting device according to parameters of the mapping model corresponding to the shooting device, wherein the parameters comprise: the focal length of the shooting device, the coordinates of the shooting device on a two-dimensional plane diagram of a three-dimensional space, the depression angle of the shooting device and the horizontal rotation angle of the shooting device.
Optionally, the output module is further configured to: inputting the first coordinate into the mapping model, and determining a third coordinate of the target object in a three-dimensional coordinate system of a shooting device; determining a fourth coordinate of the target object in a three-dimensional coordinate system of the three-dimensional space according to the third coordinate; and determining a second coordinate of the target object on the two-dimensional plane map of the three-dimensional space according to the fourth coordinate.
To achieve the above object, according to still another aspect of embodiments of the present invention, there is provided an electronic apparatus.
An electronic device of an embodiment of the present invention includes: one or more processors; the storage device is used for storing one or more programs, and when the one or more programs are executed by one or more processors, the one or more processors implement the method for generating the global thermodynamic diagram of the embodiment of the invention.
To achieve the above object, according to still another aspect of an embodiment of the present invention, there is provided a computer-readable medium.
A computer-readable medium of an embodiment of the present invention has a computer program stored thereon, and when executed by a processor, the program implements the method of generating a global thermodynamic diagram of an embodiment of the present invention.
One embodiment of the above invention has the following advantages or benefits: each shooting device in the three-dimensional space can be associated with a two-dimensional plane graph of the three-dimensional space by using a mapping model, and a target object in a shot picture can be mapped onto the two-dimensional plane graph, so that a global thermodynamic diagram of the three-dimensional space can be generated; in the embodiment of the invention, the shot picture in the three-dimensional space is detected based on the target detection model, so that the target object on the picture can be obtained, and the coordinate of the target object on the coordinate system of the shot picture is determined; according to the embodiment of the invention, after the target object on the shot picture is acquired, the target object is identified and verified based on the identification model, so that the target object which is identified repeatedly can be deleted, and the accuracy of generating the global thermodynamic diagram is improved; in the embodiment of the invention, the parameters of each shooting device in the three-dimensional space are respectively determined according to a few point collecting modes and a regression fitting algorithm, and the mapping model of each shooting device is determined by using the parameters, so that the accuracy of the parameters of the shooting devices can be improved, and the accuracy of the mapping model is improved; according to the embodiment of the invention, the second coordinate of the target object is obtained by mapping the model through the three-dimensional coordinate system of the shooting equipment and the three-dimensional coordinate system of the three-dimensional space, so that the global thermodynamic diagram of the three-dimensional space can be generated according to the second coordinate.
Further effects of the above-mentioned non-conventional alternatives will be described below in connection with the embodiments.
Drawings
The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:
FIG. 1 is a schematic diagram of the main steps of a method of generating a global thermodynamic diagram according to an embodiment of the invention;
FIG. 2 is a schematic diagram of a regression fit between real coordinates and mapped coordinates according to an embodiment of the invention;
FIG. 3 is a schematic diagram of the overall flow of a mapping model according to an embodiment of the invention;
FIG. 4 is a schematic diagram of a main flow of rendering a target object on a two-dimensional plan view of a three-dimensional space in a method of generating a global thermodynamic diagram according to the present invention;
FIG. 5 is an overall architecture diagram of a method of generating a global thermodynamic diagram embodying the invention;
FIG. 6 is a schematic diagram of the main modules of an apparatus for generating a global thermodynamic diagram according to an embodiment of the invention;
FIG. 7 is an exemplary system architecture diagram in which embodiments of the present invention may be employed;
fig. 8 is a schematic structural diagram of a computer system suitable for implementing a terminal device or a server according to an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present invention are described below with reference to the accompanying drawings, in which various details of embodiments of the invention are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
Fig. 1 is a schematic diagram of the main steps of a method of generating a global thermodynamic diagram according to an embodiment of the present invention. As an embodiment of the present invention, as shown in fig. 1, a method for generating a global thermodynamic diagram of an embodiment of the present invention mainly includes the following steps:
step S101: the method comprises the steps of acquiring a shot picture of a three-dimensional space, and determining a target object on the shot picture and a first coordinate of the target object on the shot picture.
According to the invention, firstly, shooting equipment is deployed in a three-dimensional space, and then the deployed shooting equipment is utilized to shoot the three-dimensional space to obtain a shot picture of the three-dimensional space. In the present invention, the three-dimensional space may include: the system comprises a supermarket, a fruit store, a flower shop, a scenic spot watching house and the like, wherein deployed shooting equipment can be but is not limited to a common camera. Meanwhile, in order to generate a global thermodynamic diagram of a three-dimensional space, the present invention requires that all shooting ranges of deployed shooting devices cover the entire three-dimensional space. In the invention, firstly, a shot picture of a three-dimensional space is obtained, then a target object in the picture is obtained, and then the coordinates of the target object in a pixel coordinate system of the picture are determined. For example, the three-dimensional space is a supermarket, a photograph of the taken supermarket is first acquired, then the pedestrian is recognized as the target object from the photograph, and the coordinates of the pedestrian on the taken picture are determined.
As still another embodiment of the present invention, the determining the target object on the photographed picture and the first coordinate of the target object on the photographed picture in step S101 may include: and detecting the shot picture based on the target detection model, and determining a target object on the shot picture and the coordinates of the target object on the coordinate system of the shot picture. The target detection model may be yolo2 (i.e., a target detection model), and it is needless to say that other target detection models (e.g., yolo v3, ssd, etc.) may be used to analyze the captured image, which is not limited by the present invention.
Taking the example that the three-dimensional space is a supermarket, the pedestrian (i.e. the target object) recognition is carried out on the picture collected by the camera (i.e. the shooting equipment) through the target detection model yolo2, and the central point of the pedestrian frame and the width and height of the pedestrian frame returned by the detection model are detected. Considering that the camera is not a depth camera, if a point on a shot picture is mapped to a three-dimensional coordinate system of a supermarket, the depth information of a certain point in the camera needs to be known, the coordinates of the feet of a pedestrian can be directly adopted, and the obtained coordinates are considered to be at the same depth, so that the mapping of the global plane can be carried out. Therefore, the first coordinates in the present invention are coordinates of the pedestrian's foot in the captured picture.
In the method for generating the global thermodynamic diagram, a shot picture of at least one shooting device in a three-dimensional space needs to be analyzed, so that a situation that a target object is shot by a plurality of shooting devices at the same time can occur, for example, in a supermarket in the three-dimensional space, a pedestrian P is shot and recorded by a camera A, and P is also shot and recorded by a camera B, so that points recorded twice are drawn on the thermodynamic diagram, and errors can be caused. In order to avoid this, as another embodiment of the present invention, after determining the target object on the captured picture, the method for generating the global thermodynamic diagram may further include: and identifying the target object based on the identification model, then judging whether the target object exists in the storage unit, if so, deleting the target object, and otherwise, storing the target object in the storage unit. That is to say, after acquiring a target object from a captured picture, the present invention needs to identify the target object with a recognition model (for example, a recognition model triplet, which may be, but is not limited to, other recognition models), compare the identified target object with a target object in a storage unit (which may be, but is not limited to, a database), if the target object is already included in the storage unit, do not perform subsequent process processing on the target object, and if the target object is not included in the storage unit, store the target object in the storage unit, and perform subsequent processing on the target object. It should be noted that the global thermodynamic diagram generated in the present invention is a global thermodynamic diagram for a certain period of time, and therefore the data in the memory unit refers to the data in the certain period of time.
Step S102: and mapping the first coordinate according to the mapping model, and outputting a second coordinate of the target object on a two-dimensional plane graph of the three-dimensional space. After determining the first coordinates of the target object in step S101, the first coordinates are input into the mapping model, and the coordinates of the target object on the two-dimensional plan view of the three-dimensional space are output.
As another embodiment of the present invention, before mapping the first coordinate according to the mapping model in step S102, the method for generating a global thermodynamic diagram of the present invention may further include: a mapping model of the three-dimensional space is determined. Since at least one photographing apparatus is disposed in the three-dimensional space, determining the mapping model of the three-dimensional space refers to determining the mapping model of each photographing apparatus in the three-dimensional space. In the invention, aiming at a shooting device, firstly, the parameters of the shooting device are obtained, and then the mapping model of the shooting device is determined according to the parameters. The method for acquiring the parameters of the shooting device can comprise the following steps: acquiring the focal length of the shooting equipment through the specification of the shooting equipment; determining the coordinate of the shooting equipment on a two-dimensional plane graph of a three-dimensional space through measurement; and determining the depression angle of the shooting equipment and the horizontal rotation angle of the shooting equipment based on a regression fitting algorithm by using the principle of few sampling points. The horizontal rotation angle of the photographing apparatus refers to a horizontal direction in which the photographing apparatus photographs in a three-dimensional space.
In the invention, the step of determining the depression angle of the shooting equipment and the horizontal rotation angle of the shooting equipment based on the regression fitting algorithm by using the principle of few sampling points can be specifically understood as follows: finding a preset number of pixel points (only two points are verified, but three or more points are generally selected to ensure the accuracy of the result) from a shot picture of shooting equipment, acquiring the coordinates of the pixel points in the shot picture, inputting the coordinates into a mapping model, and obtaining the output result of the mapping model as mapping coordinates; finding out a point corresponding to the pixel point on a two-dimensional plane graph of the three-dimensional space, and determining the coordinate of the point on the two-dimensional plane graph as a real coordinate; and performing regression automatic fitting on the real coordinates and the mapping coordinates to obtain the optimal depression angle and the horizontal rotation angle of the shooting equipment. For example, three points (692, 173), (813, 219), and (861, 184) are selected from the captured picture, and their corresponding points on the two-dimensional plane map are (400, 194), (432, 220), and (451, 197), respectively; and respectively binding the 3 pairs of parameters to the input and the output of the mapping model, and performing regression auto-fitting on the output result of the mapping model and the bound output to obtain a fitting graph shown in the figure 2. Fig. 2 is a schematic diagram of the regression fitting between the real coordinates and the mapping coordinates according to the embodiment of the invention. As can be seen from fig. 2, when the depression angle of the photographing apparatus is 60 degrees and the horizontal rotation angle is 239.5 degrees, the error of the real coordinates from the mapped coordinates is minimized, and thus it is determined that the depression angle of the photographing apparatus is 60 degrees and the horizontal rotation angle is 239.5 degrees.
Further, in step S102, mapping the first coordinates according to the mapping model, and outputting the second coordinates of the target object on the two-dimensional plan view of the three-dimensional space may include: inputting the first coordinate into a mapping model, and determining a third coordinate of the target object in a three-dimensional coordinate system of the shooting device; determining a fourth coordinate of the target object in a three-dimensional coordinate system of the three-dimensional space according to the third coordinate; and determining second coordinates of the target object on the two-dimensional plane graph of the three-dimensional space according to the fourth coordinates. FIG. 3 is a schematic diagram of the overall flow of the mapping model according to an embodiment of the invention. As shown in fig. 3, taking a shooting device as a camera and a three-dimensional space as a shop as an example, a coordinate system of a shot picture of the camera is an RGB image coordinate system; the shop plan coordinate system is a two-dimensional plan of the whole shop, is a place for drawing the thermodynamic diagram on the plan finally, and the unit of the coordinate system is a pixel; the three-dimensional coordinate system of the camera is the position of the camera after installation and is measured in the three-dimensional coordinate system of the shop. The coordinate of the point D in the coordinate system of the camera can be obtained through the parameters of the camera, the coordinates of the multiple cameras in the coordinate system of the shop can be obtained through measurement, and the coordinates of the point D in the coordinate system of the shop plan can be obtained through conversion. Because the acquisition points in the picture taken by the camera are all the positions of the feet of the person and can be considered to be on the horizon, the depth information can be known, the height of the camera can be measured, z1 in the coordinate system of the camera and z' in the coordinate system of the shop can be set to be 0 through conversion, and therefore the four coordinate systems can be regarded as two-dimensional coordinate systems.
Step S103: and generating a global thermodynamic diagram of the three-dimensional space according to the second coordinates and the two-dimensional plane map. After the second coordinates are obtained in step S102, the obtained second coordinate points are plotted on a two-dimensional plan view of the three-dimensional space, and the darker the color is, the more the target object is represented. And drawing the target object in the shot picture of each camera in the three-dimensional space on a two-dimensional plane graph, so as to generate a global thermodynamic diagram of the three-dimensional space.
The invention aims to generate a global thermodynamic diagram of a three-dimensional space, and the principle is to analyze a picture shot in the three-dimensional space within a period of time, acquire a target object in the picture and draw the target object in a two-dimensional plane diagram of the three-dimensional space. Fig. 4 is a schematic diagram of a main flow of rendering a target object on a two-dimensional plan view of a three-dimensional space in the method of generating a global thermodynamic diagram according to the present invention. As shown in fig. 4, the main flow of rendering the target object on the two-dimensional plan view of the three-dimensional space may include:
step S401, detecting a shot picture of a three-dimensional space based on a target detection model to obtain a target object on the shot picture;
step S402, identifying the target object based on the identification model, and judging whether the target object exists in the storage unit, if so, executing step S403, otherwise, executing step S404;
step S403, deleting the acquired target object;
step S404, storing the target object in a storage unit, and determining the coordinates of the target object on a coordinate system of the shot picture;
step S405, determining a mapping model, inputting coordinates of a target object on a coordinate system of a shot picture into the mapping model, and determining coordinates of the target object in a three-dimensional coordinate system of shooting equipment;
step S406, determining the coordinate of the target object in the three-dimensional coordinate system of the three-dimensional space according to the determined coordinate of the target object in the three-dimensional coordinate system of the shooting device;
step S407, determining the coordinate of the target object on the two-dimensional plane graph of the three-dimensional space according to the determined coordinate of the target object in the three-dimensional coordinate system of the three-dimensional space;
step S408, according to the coordinates of the target object on the two-dimensional plane map of the three-dimensional space, the target object is drawn on the two-dimensional plane map.
Wherein, the step of determining the mapping model in step S405 may include: acquiring a focal length of shooting equipment corresponding to a shot picture (namely, if the picture M is shot by a camera T, the shooting equipment corresponding to the picture M is the camera T) and a coordinate of the shooting equipment on a two-dimensional plane graph of a three-dimensional space; determining a depression angle of the shooting equipment and a horizontal rotation angle of the shooting equipment based on a regression fitting algorithm by utilizing the principle of few sampling points; and determining a mapping model of the shooting equipment according to the focal length of the shooting equipment, the coordinate of the shooting equipment on a two-dimensional plane diagram of a three-dimensional space, the depression angle of the shooting equipment and the horizontal rotation angle of the shooting equipment. According to the invention, the mapping models of all shooting devices in the three-dimensional space can be determined through calculation, then the pictures shot by all the shooting devices in a certain time period are obtained, the pictures are analyzed, and the target object on the picture is drawn on the two-dimensional plane graph of the three-dimensional space, so that the global thermodynamic diagram of the three-dimensional space can be formed.
For convenience of understanding, the three-dimensional space is taken as the shop, the target object is taken as the pedestrian, the shooting equipment is taken as the camera, the target detection model is taken as the yolo2, the recognition model is taken as the triplet model, the overall framework for implementing the method for generating the global thermodynamic diagram is explained in detail, and the overall framework diagram for implementing the method for generating the global thermodynamic diagram is obtained, wherein the overall framework diagram is shown in fig. 5. Certainly, "shop", "pedestrian", "camera", "yolo2", and "triplet model" do not limit the protection scope of the technical solution of the present invention, and the specific settings of the three-dimensional space, the target object, the shooting device, the target detection model, and the recognition model may be adaptively adjusted and selected according to the specific service scenario.
As shown in fig. 5, in the overall architecture for implementing the method for generating a global thermodynamic diagram of the present invention, the parameters of the mapping model are first calculated, then the mapping model is determined by the parameters, and finally the thermodynamic diagram is generated by using the mapping model. The method for calculating the model parameters is described above in detail, and will not be described here. Determining a mapping model through model parameters, wherein the mapping process of four coordinate systems in the mapping model is as follows: the camera picture coordinate system is to the three-dimensional coordinate system of the camera, then to the three-dimensional coordinate system of the shop, and finally to the coordinate system of the two-dimensional plane graph of the shop. The subject process may include: arranging a plurality of common cameras in a shop, and requiring the shooting range of the cameras to cover the whole shop; acquiring a picture shot by a camera within a certain time period, and performing pedestrian detection on the picture shot by the camera through yolo2 to obtain a coordinate of a pedestrian on the picture; inputting the coordinates of the pedestrians on the picture into a camera mapping model for calculation; the mapping model outputs coordinates of a plane of a shop where the pedestrian is located in the picture shot by the camera; and drawing the coordinates of the pedestrians output in the time period on the plane of the store on a plane diagram of the store to obtain the global thermodynamic diagram of the store. It should be noted that when the picture shot by the camera is subjected to pedestrian detection through yolo2, whether the pedestrian is shot by the multiple cameras at the same time needs to be judged, in the invention, the pedestrian can be identified by using the triplet model and compared with the data in the database, and if the pedestrian exists in the database, the detected pedestrian is not subjected to subsequent processing. According to the invention, each shooting device in the shop can be associated with the shop plan map by using the mapping model to generate the global thermodynamic diagram of the shop, so that the density of pedestrians in different areas in the shop can be seen more intuitively, and products can be re-carded in areas with high density in time, so that the placement of the products is more reasonable.
According to the technical scheme for generating the global thermodynamic diagram, the mapping model can be used for associating each shooting device in the three-dimensional space with the two-dimensional plane diagram of the three-dimensional space, and the target object in the shot picture can be mapped onto the two-dimensional plane diagram, so that the global thermodynamic diagram of the three-dimensional space can be generated; in the embodiment of the invention, the shot picture in the three-dimensional space is detected based on the target detection model, so that the target object on the picture can be obtained, and the coordinate of the target object on the coordinate system of the shot picture is determined; according to the embodiment of the invention, after the target object on the shot picture is acquired, the target object is identified and verified based on the identification model, so that the target object which is identified repeatedly can be deleted, and the accuracy of generating the global thermodynamic diagram is improved; in the embodiment of the invention, the parameters of each shooting device in the three-dimensional space are respectively determined according to a few point collecting modes and a regression fitting algorithm, and the mapping model of each shooting device is determined by using the parameters, so that the accuracy of the parameters of the shooting devices can be improved, and the accuracy of the mapping model is improved; according to the embodiment of the invention, the second coordinate of the target object is obtained by mapping the model through the three-dimensional coordinate system of the shooting equipment and the three-dimensional coordinate system of the three-dimensional space, so that the global thermodynamic diagram of the three-dimensional space can be generated according to the second coordinate.
Fig. 6 is a schematic diagram of the main modules of an apparatus for generating a global thermodynamic diagram according to an embodiment of the present invention. As shown in fig. 6, an apparatus 600 for generating a global thermodynamic diagram according to an embodiment of the present invention mainly includes the following modules: an acquisition module 601, an output module 602, and a generation module 603. The obtaining module 601 may be configured to obtain a captured picture of a three-dimensional space, and determine a target object on the captured picture and a first coordinate of the target object on the captured picture. The output module 602 may be configured to map the first coordinates according to the mapping model and output second coordinates of the target object on a two-dimensional plane graph of the three-dimensional space. The generating module 603 may be configured to generate a global thermodynamic diagram of the three-dimensional space based on the second coordinates and the two-dimensional plan.
In this embodiment of the present invention, the obtaining module 601 may further be configured to: and detecting the shot picture based on the target detection model, and determining a target object on the shot picture and the coordinates of the target object on the coordinate system of the shot picture.
In this embodiment of the present invention, the apparatus 600 for generating a global thermodynamic diagram may further include: a decision block (not shown). The determination module may be to: and identifying the target object based on the identification model, then judging whether the target object exists in the storage unit, if so, deleting the target object, and otherwise, storing the target object in the storage unit.
In this embodiment of the present invention, the output module 602 may further be configured to: aiming at each shooting device in the three-dimensional space, acquiring the focal length of the shooting device and the coordinate of the shooting device on a two-dimensional plane graph of the three-dimensional space; determining a depression angle of the shooting equipment and a horizontal rotation angle of the shooting equipment by using the principle of a small number of sampling points and based on a regression fitting algorithm; determining a mapping model of the shooting device according to parameters of the mapping model corresponding to the shooting device, wherein the parameters comprise: the focal length of the shooting device, the coordinates of the shooting device on a two-dimensional plane graph of a three-dimensional space, the depression angle of the shooting device and the horizontal rotation angle of the shooting device.
In this embodiment of the present invention, the output module 602 may further be configured to: inputting the first coordinate into the mapping model, and determining a third coordinate of the target object in a three-dimensional coordinate system of the shooting device; determining a fourth coordinate of the target object in a three-dimensional coordinate system of the three-dimensional space according to the third coordinate; and determining second coordinates of the target object on the two-dimensional plane graph of the three-dimensional space according to the fourth coordinates.
As can be seen from the above description, each photographing device in a three-dimensional space can be associated with a two-dimensional plane map of the three-dimensional space by using a mapping model, and a target object in a photographed picture can be mapped onto the two-dimensional plane map, so that a global thermodynamic diagram of the three-dimensional space can be generated; in the embodiment of the invention, the shot picture in the three-dimensional space is detected based on the target detection model, so that the target object on the picture can be obtained, and the coordinate of the target object on the coordinate system of the shot picture is determined; according to the embodiment of the invention, after the target object on the shot picture is acquired, the target object is identified and verified based on the identification model, so that the target object which is identified repeatedly can be deleted, and the accuracy of generating the global thermodynamic diagram is improved; in the embodiment of the invention, the parameters of each shooting device in the three-dimensional space are respectively determined according to a few point collecting modes and a regression fitting algorithm, and the mapping model of each shooting device is determined by using the parameters, so that the accuracy of the parameters of the shooting devices can be improved, and the accuracy of the mapping model is improved; according to the embodiment of the invention, the second coordinate of the target object is obtained by mapping the model through the three-dimensional coordinate system of the shooting equipment and the three-dimensional coordinate system of the three-dimensional space, so that the global thermodynamic diagram of the three-dimensional space can be generated according to the second coordinate.
Fig. 7 illustrates an exemplary system architecture 700 of a method of generating a global thermodynamic diagram or an apparatus for generating a global thermodynamic diagram to which embodiments of the invention may be applied.
As shown in fig. 7, the system architecture 700 may include terminal devices 701, 702, 703, a network 704, and a server 705. The network 704 serves to provide a medium for communication links between the terminal devices 701, 702, 703 and the server 705. Network 704 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
A user may use the terminal devices 701, 702, 703 to interact with a server 705 over a network 704, to receive or send messages or the like. The terminal devices 701, 702, 703 may have installed thereon various communication client applications, such as a shopping-like application, a web browser application, a search-like application, an instant messaging tool, a mailbox client, social platform software, etc. (by way of example only).
The terminal devices 701, 702, 703 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, and the like.
The server 705 may be a server providing various services, such as a background management server (for example only) providing support for shopping websites browsed by users using the terminal devices 701, 702, 703. The backend management server may analyze and perform other processing on the received data such as the product information query request, and feed back a processing result (for example, target push information, product information — just an example) to the terminal device.
It should be noted that the method for generating the global thermodynamic diagram according to the embodiment of the present invention is generally performed by the server 705, and accordingly, the apparatus for generating the global thermodynamic diagram is generally disposed in the server 705.
It should be understood that the number of terminal devices, networks, and servers in fig. 7 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for an implementation.
Referring now to FIG. 8, shown is a block diagram of a computer system 800 suitable for use with a terminal device implementing an embodiment of the present invention. The terminal device shown in fig. 8 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present invention.
As shown in fig. 8, the computer system 800 includes a Central Processing Unit (CPU) 801 that can perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM) 802 or a program loaded from a storage section 808 into a Random Access Memory (RAM) 803. In the RAM 803, various programs and data necessary for the operation of the system 800 are also stored. The CPU 801, ROM 802, and RAM 803 are connected to each other via a bus 804. An input/output (I/O) interface 805 is also connected to bus 804.
The following components are connected to the I/O interface 805: an input portion 806 including a keyboard, a mouse, and the like; an output section 807 including components such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and a speaker; a storage portion 808 including a hard disk and the like; and a communication section 809 including a network interface card such as a LAN card, a modem, or the like. The communication section 809 performs communication processing via a network such as the internet. A drive 810 is also connected to the I/O interface 805 as necessary. A removable medium 811 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 810 as necessary, so that the computer program read out therefrom is mounted on the storage section 808 as necessary.
In particular, according to embodiments of the present disclosure, the processes described above with reference to the flow diagrams may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer-readable medium, the computer program comprising program code for performing the method illustrated by the flow chart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication section 809 and/or installed from the removable medium 811. The computer program executes the above-described functions defined in the system of the present invention when executed by the Central Processing Unit (CPU) 801.
It should be noted that the computer readable medium shown in the present invention can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present invention, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In contrast, in the present invention, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules described in the embodiments of the present invention may be implemented by software or hardware. The described modules may also be provided in a processor, which may be described as: a processor includes an acquisition module, an output module, and a generation module. Where the names of these modules do not constitute a limitation on the unit itself in some cases, for example, the determination module may also be described as "a module that takes a taken picture of a three-dimensional space and determines a target object on the taken picture and a first coordinate of the target object on the taken picture".
As another aspect, the present invention also provides a computer-readable medium that may be contained in the apparatus described in the above embodiments; or may be separate and not incorporated into the device. The computer readable medium carries one or more programs which, when executed by a device, cause the device to comprise: acquiring a shot picture of a three-dimensional space, and determining a target object on the shot picture and a first coordinate of the target object on the shot picture; mapping the first coordinate according to the mapping model, and outputting a second coordinate of the target object on a two-dimensional plane graph of the three-dimensional space; and generating a global thermodynamic diagram of the three-dimensional space according to the second coordinates and the two-dimensional plane map.
According to the technical scheme of the embodiment of the invention, each shooting device in the three-dimensional space can be associated with the two-dimensional plane graph of the three-dimensional space by using the mapping model, and the target object in the shot picture can be mapped onto the two-dimensional plane graph, so that the global thermodynamic diagram of the three-dimensional space can be generated; in the embodiment of the invention, the shot picture in the three-dimensional space is detected based on the target detection model, so that the target object on the picture can be obtained, and the coordinate of the target object on the coordinate system of the shot picture is determined; according to the embodiment of the invention, after the target object on the shot picture is acquired, the target object is identified and verified based on the identification model, so that the target object which is identified repeatedly can be deleted, and the accuracy of generating the global thermodynamic diagram is improved; in the embodiment of the invention, the parameters of each shooting device in the three-dimensional space are respectively determined according to a few point collecting modes and a regression fitting algorithm, and the mapping model of each shooting device is determined by using the parameters, so that the accuracy of the parameters of the shooting devices can be improved, and the accuracy of the mapping model is improved; according to the embodiment of the invention, the second coordinate of the target object is obtained by mapping the model through the three-dimensional coordinate system of the shooting equipment and the three-dimensional coordinate system of the three-dimensional space, so that the global thermodynamic diagram of the three-dimensional space can be generated according to the second coordinate.
The above-described embodiments should not be construed as limiting the scope of the invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions can occur, depending on design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (6)
1. A method of generating a global thermodynamic diagram, comprising:
acquiring a shot picture of a three-dimensional space, and determining a target object on the shot picture and a first coordinate of the target object on the shot picture; wherein determining a target object on the shot picture and a first coordinate of the target object on the shot picture comprises: detecting the shot picture based on a target detection model, and determining a target object on the shot picture and coordinates of the target object on a coordinate system of the shot picture;
after determining the target object on the shot picture, the method further comprises the following steps: identifying the target object based on an identification model, then judging whether the target object exists in a storage unit, if so, deleting the target object, otherwise, storing the target object in the storage unit;
mapping the first coordinate according to a mapping model, and outputting a second coordinate of the target object on a two-dimensional plane graph of the three-dimensional space; prior to mapping the first coordinates according to a mapping model, the method further comprises: for each shooting device in the three-dimensional space, acquiring a focal length of the shooting device and a coordinate of the shooting device on a two-dimensional plane graph of the three-dimensional space; determining a depression angle of the shooting equipment and a horizontal rotation angle of the shooting equipment by using the principle of few sampling points and based on a regression fitting algorithm; determining the mapping model of the shooting device according to the parameters of the mapping model corresponding to the shooting device, wherein the parameters comprise: the focal length of the shooting equipment, the coordinate of the shooting equipment on a two-dimensional plane diagram of a three-dimensional space, the depression angle of the shooting equipment and the horizontal rotation angle of the shooting equipment;
and generating a global thermodynamic diagram of the three-dimensional space according to the second coordinate and the two-dimensional plane graph.
2. The method of claim 1, wherein mapping the first coordinates according to a mapping model, and outputting second coordinates of the target object on a two-dimensional plan view of the three-dimensional space comprises:
inputting the first coordinate into the mapping model, and determining a third coordinate of the target object in a three-dimensional coordinate system of a shooting device;
determining a fourth coordinate of the target object in a three-dimensional coordinate system of the three-dimensional space according to the third coordinate;
and determining a second coordinate of the target object on the two-dimensional plane map of the three-dimensional space according to the fourth coordinate.
3. An apparatus that generates a global thermodynamic diagram, comprising:
the acquisition module is used for acquiring a shot picture of a three-dimensional space and determining a target object on the shot picture and a first coordinate of the target object on the shot picture; the acquisition module is further used for detecting the shot picture based on a target detection model, and determining a target object on the shot picture and the coordinates of the target object on the coordinate system of the shot picture;
the judging module is used for identifying the target object based on an identification model after determining the target object on the shot picture, then judging whether the target object exists in a storage unit, if so, deleting the target object, otherwise, storing the target object in the storage unit;
the output module is used for mapping the first coordinate according to a mapping model and outputting a second coordinate of the target object on a two-dimensional plane graph of the three-dimensional space; prior to mapping the first coordinates according to a mapping model, the output module is further to: for each shooting device in the three-dimensional space, acquiring a focal length of the shooting device and a coordinate of the shooting device on a two-dimensional plane map of the three-dimensional space; determining a depression angle of the shooting equipment and a horizontal rotation angle of the shooting equipment by using the principle of few sampling points and based on a regression fitting algorithm; determining a mapping model of the shooting device according to parameters of the mapping model corresponding to the shooting device, wherein the parameters comprise: the focal length of the shooting equipment, the coordinate of the shooting equipment on a two-dimensional plane diagram of a three-dimensional space, the depression angle of the shooting equipment and the horizontal rotation angle of the shooting equipment;
and the generating module is used for generating a global thermodynamic diagram of the three-dimensional space according to the second coordinate and the two-dimensional plane diagram.
4. The apparatus of claim 3, wherein the output module is further configured to:
inputting the first coordinate into the mapping model, and determining a third coordinate of the target object in a three-dimensional coordinate system of a shooting device;
determining a fourth coordinate of the target object in a three-dimensional coordinate system of the three-dimensional space according to the third coordinate;
and determining a second coordinate of the target object on the two-dimensional plane map of the three-dimensional space according to the fourth coordinate.
5. An electronic device, comprising:
one or more processors;
a storage device to store one or more programs,
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method recited in any of claims 1-2.
6. A computer-readable medium, on which a computer program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1-2.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810638146.7A CN110619807B (en) | 2018-06-20 | 2018-06-20 | Method and device for generating global thermodynamic diagram |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810638146.7A CN110619807B (en) | 2018-06-20 | 2018-06-20 | Method and device for generating global thermodynamic diagram |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110619807A CN110619807A (en) | 2019-12-27 |
| CN110619807B true CN110619807B (en) | 2022-12-02 |
Family
ID=68920924
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810638146.7A Active CN110619807B (en) | 2018-06-20 | 2018-06-20 | Method and device for generating global thermodynamic diagram |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110619807B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113362090A (en) * | 2020-03-03 | 2021-09-07 | 北京沃东天骏信息技术有限公司 | User behavior data processing method and device |
| CN111783549A (en) * | 2020-06-04 | 2020-10-16 | 北京海益同展信息科技有限公司 | Distribution diagram generation method and system, inspection robot and control terminal |
| CN111970490B (en) * | 2020-08-06 | 2023-04-18 | 万翼科技有限公司 | User traffic monitoring method and related equipment |
| CN112200112A (en) * | 2020-10-19 | 2021-01-08 | 烟台创迹软件有限公司 | System for thermal analysis of store panoramic crowd |
| CN112532874B (en) * | 2020-11-23 | 2022-03-29 | 北京三快在线科技有限公司 | Method and device for generating plane thermodynamic diagram, storage medium and electronic equipment |
Family Cites Families (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH06162290A (en) * | 1992-11-26 | 1994-06-10 | Shimizu Corp | Device for monitoring waiting people staying state in waiting area |
| EP2309453A3 (en) * | 1998-07-31 | 2012-09-26 | Panasonic Corporation | Image displaying apparatus and image displaying method |
| JP2009236696A (en) * | 2008-03-27 | 2009-10-15 | Toppan Printing Co Ltd | Three-dimensional image measurement method, measurement system, and measurement program for subject |
| CN102148965B (en) * | 2011-05-09 | 2014-01-15 | 厦门博聪信息技术有限公司 | Video monitoring system for multi-target tracking close-up shooting |
| US9749594B2 (en) * | 2011-12-22 | 2017-08-29 | Pelco, Inc. | Transformation between image and map coordinates |
| CN103327293B (en) * | 2012-03-23 | 2017-09-19 | 罗普特（厦门）科技集团有限公司 | A kind of monitoring device and method being combined by video calibration with electronic map |
| CN104424630A (en) * | 2013-08-20 | 2015-03-18 | 华为技术有限公司 | Three-dimension reconstruction method and device, and mobile terminal |
| CN105678684B (en) * | 2014-11-18 | 2020-11-03 | 中兴通讯股份有限公司 | Method and device for intercepting image |
| CN106157360A (en) * | 2015-04-28 | 2016-11-23 | 宇龙计算机通信科技(深圳)有限公司 | A kind of three-dimensional modeling method based on dual camera and device |
| WO2017000115A1 (en) * | 2015-06-29 | 2017-01-05 | 北京旷视科技有限公司 | Person re-identification method and device |
| CN204924967U (en) * | 2015-08-18 | 2015-12-30 | 浙江欧威科技有限公司 | Image acquisition mechanism of full -automatic AOI complete machine |
| WO2017155126A1 (en) * | 2016-03-08 | 2017-09-14 | 一般社団法人 日本画像認識協会 | Information transmitting system, information transmitting device, information receiving device, and computer program |
| CN107315824B (en) * | 2017-07-04 | 2021-01-22 | 百度在线网络技术（北京）有限公司 | Method and device for generating thermodynamic diagram |
-
2018
- 2018-06-20 CN CN201810638146.7A patent/CN110619807B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN110619807A (en) | 2019-12-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110619807B (en) | Method and device for generating global thermodynamic diagram | |
| US10699431B2 (en) | Method and apparatus for generating image generative model | |
| US9129435B2 (en) | Method for creating 3-D models by stitching multiple partial 3-D models | |
| CN106845470B (en) | Map data acquisition method and device | |
| CN109040960A (en) | A kind of method and apparatus for realizing location-based service | |
| US20130083990A1 (en) | Using Videogrammetry to Fabricate Parts | |
| US10614621B2 (en) | Method and apparatus for presenting information | |
| CN109118456B (en) | Image processing method and device | |
| US20210264198A1 (en) | Positioning method and apparatus | |
| CN108846899B (en) | Method and system for improving area perception of user for each function in house source | |
| US10921131B1 (en) | Systems and methods for interactive digital maps | |
| CN110657760B (en) | Method and device for measuring space area based on artificial intelligence and storage medium | |
| CN109978753B (en) | Method and device for drawing panoramic thermodynamic diagram | |
| WO2019164830A1 (en) | Apparatus, systems, and methods for tagging building features in a 3d space | |
| CN110895833A (en) | Method and device for three-dimensional modeling of indoor scene | |
| CN110673717A (en) | Method and apparatus for controlling output device | |
| CN109901988A (en) | A kind of page elements localization method and device for automatic test | |
| CN109426878B (en) | Method and apparatus for outputting information | |
| US20200412949A1 (en) | Device, system, and method for capturing and processing data from a space | |
| CN110662015A (en) | Method and apparatus for displaying image | |
| CN108427110A (en) | Distance measuring method, device and electronic equipment | |
| CN113362090A (en) | User behavior data processing method and device | |
| CN109034214B (en) | Method and apparatus for generating a mark | |
| CN113808134B (en) | Oil tank layout information generation method, oil tank layout information generation device, electronic apparatus, and medium | |
| CN110634159A (en) | Target detection method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |