CN112837214A - Three-dimensional portrait acquisition method, video analysis method, device, equipment and medium - Google Patents
Three-dimensional portrait acquisition method, video analysis method, device, equipment and medium Download PDFInfo
- Publication number
- CN112837214A CN112837214A CN202110179244.0A CN202110179244A CN112837214A CN 112837214 A CN112837214 A CN 112837214A CN 202110179244 A CN202110179244 A CN 202110179244A CN 112837214 A CN112837214 A CN 112837214A
- Authority
- CN
- China
- Prior art keywords
- dimensional
- portrait
- target object
- indoor
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 91
- 238000004458 analytical method Methods 0.000 title claims abstract description 43
- 238000006243 chemical reaction Methods 0.000 claims abstract description 84
- 239000011159 matrix material Substances 0.000 claims abstract description 56
- 238000001514 detection method Methods 0.000 claims abstract description 18
- 238000004590 computer program Methods 0.000 claims description 30
- 230000008569 process Effects 0.000 claims description 27
- 230000006399 behavior Effects 0.000 claims description 22
- 238000012545 processing Methods 0.000 claims description 20
- 230000009466 transformation Effects 0.000 description 12
- 238000010586 diagram Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 2
- 238000003062 neural network model Methods 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012806 monitoring device Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/08—Projecting images onto non-planar surfaces, e.g. geodetic screens
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The application relates to a three-dimensional portrait acquisition method, a video analysis method, a device, equipment and a medium. The method comprises the following steps: carrying out target object detection on a two-dimensional indoor scene graph to obtain a target object image and a portrait in the indoor scene graph; performing two-dimensional to three-dimensional coordinate conversion on the target object image according to a preset table shape to obtain a conversion matrix; and performing two-dimensional to three-dimensional coordinate conversion on the portrait according to the conversion matrix to obtain the three-dimensional portrait. The method improves portability.
Description
Technical Field
The present application relates to the field of image processing technologies, and in particular, to a method, an apparatus, a device, and a medium for acquiring a three-dimensional portrait.
Background
With the development of computer technology, people often need to convert a two-dimensional image into a three-dimensional space for observation in order to analyze the image, so that the blocked part in the two-dimensional image can be observed conveniently.
The conventional conversion process needs to firstly calibrate by using a special calibration board such as a black and white chessboard, and then realize the two-dimensional to three-dimensional conversion of the image based on calibrated parameters so as to observe the target in the image. However, the conventional method requires a calibration plate to be carried for calibration, and thus is inconvenient to operate.
Disclosure of Invention
In view of the above, it is necessary to provide a method for acquiring a three-dimensional portrait, a method for video analysis, an apparatus, a computer device, and a storage medium, which can be easily operated.
In a first aspect, an embodiment of the present application provides a method for acquiring a three-dimensional portrait, where the method includes:
carrying out target object detection on a two-dimensional indoor scene graph to obtain a target object image and a portrait in the indoor scene graph;
according to the appearance parameters of the target object in the target object image, performing two-dimensional to three-dimensional coordinate conversion on the target object image to obtain a conversion matrix;
and performing two-dimensional to three-dimensional coordinate conversion on the portrait according to the conversion matrix to obtain the three-dimensional portrait.
In one embodiment, the performing two-dimensional to three-dimensional coordinate transformation on the object image according to the shape parameter of the object in the object image to obtain a transformation matrix includes:
acquiring a plurality of vertexes of the target object image;
and taking the plurality of vertexes as calibration points, and performing two-dimensional to three-dimensional coordinate conversion on the target object image according to the appearance parameters of the target object to obtain the conversion matrix.
In one embodiment, the performing two-dimensional to three-dimensional coordinate transformation on the portrait according to the transformation matrix to obtain the three-dimensional portrait includes:
and multiplying the portrait by the conversion matrix to obtain the three-dimensional portrait.
In one embodiment, the indoor scene graph is a conference room scene graph inside a conference room, and the object image is a conference table image.
In a second aspect, an embodiment of the present application provides a video analysis method, where the method includes:
acquiring a plurality of indoor video frame images;
processing each indoor video frame image by adopting the three-dimensional portrait acquisition method in the embodiment to obtain a three-dimensional portrait corresponding to each indoor video frame image;
and determining human body motion tracks corresponding to different human images according to the three-dimensional human images corresponding to the indoor video frame images.
In one embodiment, behavior analysis is performed according to the human motion trajectory to obtain a behavior analysis result.
In a third aspect, an embodiment of the present application provides an apparatus for acquiring a three-dimensional portrait, where the apparatus includes:
the detection module is used for detecting a target object in a two-dimensional indoor scene graph to obtain a target object image and a portrait in the indoor scene graph;
and the processing module is used for performing two-dimensional to three-dimensional coordinate conversion on the target object image according to the appearance parameters of the target object in the target object image to obtain a conversion matrix, and performing two-dimensional to three-dimensional coordinate conversion on the portrait according to the conversion matrix to obtain the three-dimensional portrait.
In a fourth aspect, an embodiment of the present application provides a video analysis apparatus, including:
the system comprises an acquisition module, a processing module and a display module, wherein the acquisition module is used for acquiring a plurality of indoor video frame images;
the processing module is used for processing each indoor video frame image by adopting the method for acquiring the three-dimensional portrait described in the embodiment to obtain the three-dimensional portrait corresponding to each indoor video frame image;
and the analysis module is used for determining human motion tracks corresponding to different human images according to the three-dimensional human images corresponding to the indoor video frame images.
In a fifth aspect, an embodiment of the present application provides a computer device, including a memory and a processor, where the memory stores a computer program, and the processor implements the following steps when executing the computer program:
carrying out target object detection on a two-dimensional indoor scene graph to obtain a target object image and a portrait in the indoor scene graph;
according to the appearance parameters of the target object in the target object image, performing two-dimensional to three-dimensional coordinate conversion on the target object image to obtain a conversion matrix;
and performing two-dimensional to three-dimensional coordinate conversion on the portrait according to the conversion matrix to obtain the three-dimensional portrait.
In a sixth aspect, an embodiment of the present application provides a computer device, including a memory and a processor, where the memory stores a computer program, and the processor implements the following steps when executing the computer program:
acquiring a plurality of indoor video frame images;
processing each indoor video frame image by adopting the three-dimensional portrait acquisition method in the embodiment to obtain a three-dimensional portrait corresponding to each indoor video frame image;
and determining human body motion tracks corresponding to different human images according to the three-dimensional human images corresponding to the indoor video frame images.
In a seventh aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the following steps:
carrying out target object detection on a two-dimensional indoor scene graph to obtain a target object image and a portrait in the indoor scene graph;
according to the appearance parameters of the target object in the target object image, performing two-dimensional to three-dimensional coordinate conversion on the target object image to obtain a conversion matrix;
and performing two-dimensional to three-dimensional coordinate conversion on the portrait according to the conversion matrix to obtain the three-dimensional portrait.
In an eighth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the following steps:
acquiring a plurality of indoor video frame images;
processing each indoor video frame image by adopting the three-dimensional portrait acquisition method in the embodiment to obtain a three-dimensional portrait corresponding to each indoor video frame image;
and determining human body motion tracks corresponding to different human images according to the three-dimensional human images corresponding to the indoor video frame images.
The three-dimensional portrait acquiring method, the video analyzing method and the device, the computer equipment and the storage medium are characterized in that the computer equipment detects a target object through a two-dimensional indoor scene graph to obtain a target object image and a portrait in the indoor scene graph, then according to the preset appearance parameters of the target object, the two-dimensional to three-dimensional coordinate conversion is carried out on the target object image to obtain a conversion matrix, thereby realizing the calibration based on the physical dimension of the known target object in the existing indoor scene image, and finally the two-dimensional to three-dimensional coordinate conversion is carried out on the portrait according to the conversion matrix to obtain a three-dimensional portrait, occlusion problems in indoor scenes observed through two-dimensional images can be avoided, the angle of an observed object is more comprehensive, the observation result is more accurate, and the application scenes of the image three-dimensional portrait are further enriched. The method can realize the calibration of the camera through the regular known shape parameters of the target object to obtain the conversion matrix, and avoid the inconvenience of carrying a calibration plate for calibration, thereby ensuring that the operation is more convenient and quicker.
Drawings
FIG. 1 is a diagram illustrating an internal structure of a computer device according to an embodiment;
FIG. 2 is a schematic flow chart of a method for acquiring a three-dimensional portrait according to an embodiment;
fig. 3 is a schematic flow chart of a method for acquiring a three-dimensional portrait according to another embodiment;
FIG. 4 is a flow diagram illustrating a video analysis method, according to an embodiment;
FIG. 5 is a schematic structural diagram of an apparatus for acquiring a three-dimensional portrait according to an embodiment;
fig. 6 is a schematic structural diagram of a video analysis apparatus according to an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
The method for acquiring the three-dimensional portrait and the method for analyzing the video provided by the embodiment of the application can be applied to the computer equipment shown in fig. 1. The computer device comprises a processor, a memory, a network interface, a database, a display screen and an input device which are connected through a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database of the computer device is used for storing the indoor scene graph in the following embodiments, and the specific description of the indoor scene graph refers to the specific description in the following embodiments. The network interface of the computer device may be used to communicate with other devices outside over a network connection. Optionally, the computer device may be a server, a desktop, a personal digital assistant, other terminal devices such as a tablet computer, a mobile phone, and the like, or a cloud or a remote server, and the specific form of the computer device is not limited in the embodiment of the present application. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on the shell of the computer equipment, an external keyboard, a touch pad or a mouse and the like. Of course, the input device and the display screen may not belong to a part of the computer device, and may be external devices of the computer device.
Those skilled in the art will appreciate that the architecture shown in fig. 1 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
The following describes the technical solutions of the present application and how to solve the above technical problems with specific examples. The following several specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments. Embodiments of the present application will be described below with reference to the accompanying drawings.
It should be noted that the execution subjects of the following method embodiments may be a three-dimensional portrait acquisition device and a video analysis device, respectively, and the device may be implemented as part of or all of the computer device by software, hardware, or a combination of software and hardware. The following method embodiments are described by taking the execution subject as the computer device as an example.
Fig. 2 is a schematic flow chart of a method for acquiring a three-dimensional portrait according to an embodiment. The embodiment relates to a specific process of calibrating and obtaining a three-dimensional portrait by using a table for computer equipment. As shown in fig. 2, includes:
and S11, carrying out target object detection on the two-dimensional indoor scene graph to obtain a target object image and a portrait in the indoor scene graph.
Specifically, the computer device may obtain a two-dimensional indoor scene graph, where the indoor scene graph may be an indoor image captured by a camera or a frame image in a video captured by the camera. Alternatively, the computer device may be an indoor scene graph read from the storage device, or may receive an indoor scene graph transmitted from another device, for example, a camera or the like. The indoor scene graph includes an image of an object placed on a table in a room and a portrait of a person located in the room. The computer device may perform target object detection on the indoor scene graph by using a preset target detection algorithm, so as to obtain a target object image and a portrait in the indoor scene graph through detection, and optionally, may obtain a rectangular frame in which the target object image is located and a rectangular frame in which the portrait is located. Alternatively, the target detection algorithm may be a Young Only Look Once (YOLO) algorithm, or may also be a Region-Convolutional Neural Networks (R-CNN) algorithm, or another target detection algorithm, which is not limited in this embodiment as long as the target object can be detected from the two-dimensional image.
And S12, according to preset shape parameters of the target object, performing two-dimensional to three-dimensional coordinate conversion on the target object image to obtain a conversion matrix.
Specifically, the computer device may obtain preset shape parameters of the target object image, and generally, the shape of the target object image is regular, so that the computer device may perform two-dimensional to three-dimensional coordinate conversion on the target object image according to the known shape parameters of the target object in the actual space, which may include the actual shape structure and the actual size of the target object, so as to obtain a conversion matrix from a two-dimensional plane where the indoor scene graph is located to the three-dimensional space. It should be noted that the process of obtaining the transformation matrix is a process of calibrating the image capturing device to obtain the camera internal parameter, the camera external parameter, or the camera distortion parameter. Optionally, the computer device may fill the pixel points of the target object image at corresponding positions according to the transformation matrix, so as to obtain a three-dimensional target object model. Alternatively, the target object may be an object with a regular shape such as a table, a chair, or a window, and the target object image may be a table image, a chair image, or a window image.
And S13, converting the portrait into a three-dimensional coordinate according to the conversion matrix to obtain the three-dimensional portrait.
Specifically, the computer device performs two-dimensional to three-dimensional coordinate conversion on the two-dimensional portrait according to the conversion matrix, and optionally, the pixel points of the two-dimensional portrait may be filled in corresponding positions according to the conversion matrix, so as to obtain a three-dimensional portrait.
In this embodiment, the computer device performs target object detection on the two-dimensional indoor scene graph to obtain a target object image and a portrait in the indoor scene graph, then performs two-dimensional to three-dimensional coordinate conversion on the target object image according to preset shape parameters of the target object to obtain a conversion matrix, so that the calibration of the shape size of an entity of a known target object in the existing indoor scene graph is realized, and finally performs two-dimensional to three-dimensional coordinate conversion on the portrait according to the conversion matrix to obtain a three-dimensional portrait. The method can realize the calibration of the camera through the regular known shape parameters of the target object to obtain the conversion matrix, and avoid the inconvenience of carrying a calibration plate for calibration, thereby ensuring that the operation is more convenient and quicker.
Optionally, on the basis of the foregoing embodiment, a possible implementation manner of the step S12 may be as shown in fig. 3, and includes:
and S121, acquiring a plurality of vertexes of the target object image.
Specifically, the computer device obtains a plurality of vertices in the image of the target object, optionally, the vertices may include four vertices of a rectangular desktop or vertices of other parts of the desktop, such as a plurality of vertices of a table leg, and the number and the selection manner of the vertices are not limited in this embodiment.
And S122, taking the plurality of vertexes as calibration points, and performing two-dimensional to three-dimensional coordinate conversion on the target object image according to the shape parameters of the target object to obtain the conversion matrix.
Specifically, the computer device performs two-dimensional to three-dimensional coordinate transformation on the target image according to known shape parameters of the target with the plurality of vertices as the calibration points, thereby obtaining the transformation matrix. Generally, when the number of vertices is large, the obtained conversion matrix is more accurate, but the amount of computation increases, and when the number of vertices is small, the amount of computation decreases, and thus the processing efficiency is higher. Optionally, the computer device may select four vertices of the desktop as the calibration points, and since the four vertices of the desktop are easy to identify and the desktop size specification is uniform, the target object image is subjected to two-dimensional to three-dimensional coordinate conversion based on the four vertices of the desktop, and the obtained conversion matrix is more accurate. Optionally, the method for obtaining the transformation matrix may adopt an active visual calibration method, a camera self-calibration method, a traditional camera calibration method, and the like, for example, a zhangyoutiao calibration method, which is not limited in this embodiment of the present application.
In this embodiment, the computer device obtains a plurality of vertexes of the target image, performs two-dimensional to three-dimensional coordinate conversion on the target image according to the shape parameters of the target, and obtains the conversion matrix, thereby implementing calibration of the plurality of vertexes based on the known shape of the target, so that the obtained conversion matrix from the two-dimensional plane to the three-dimensional space is more accurate, the calibration process does not depend on an external calibration board, and therefore, the operation is more convenient and simplified.
Optionally, one possible implementation manner of the step S13 may include: and multiplying the portrait by the conversion matrix to obtain the three-dimensional portrait. The computer device can multiply the two-dimensional coordinates of the pixel points in the portrait with the conversion matrix, so that the process of converting the two-dimensional coordinates of each pixel point into three-dimensional coordinates is realized.
In this embodiment, the target object is a desktop, and the target object image is a desktop image, and the method for acquiring the three-dimensional figure can be applied to all indoor scenes with desktops, and can acquire the three-dimensional figure image from a two-dimensional indoor scene image. Indoor scenes with desktops include, but are not limited to, indoor office scenes, conference room scenes, and the like. Accordingly, the indoor scene graph includes, but is not limited to, an indoor office scene graph, a conference room scene graph, and the like.
For example, the indoor scene graph is a conference room scene graph inside a conference room, and the object image is a conference table image. Therefore, the computer equipment can realize the process of acquiring the portrait of the three-dimensional conference participants based on the known appearance of the conference table by the method in the embodiment, thereby avoiding the process of carrying a calibration plate for calibration, and being more convenient to operate. In the above embodiment, a method for obtaining a three-dimensional portrait by calibrating a computer device based on known shape parameters of a target object is described, and a specific process of performing video analysis by using the method for obtaining a three-dimensional portrait will be described below.
Fig. 4 is a flowchart illustrating a video analysis method according to an embodiment. The embodiment relates to a specific process of obtaining a human motion trajectory by calibrating and obtaining a three-dimensional portrait by using the known appearance parameters of a target object by using computer equipment as described in the above embodiments. As shown in fig. 4, includes:
and S21, acquiring a plurality of indoor video frame images.
Specifically, the computer device may read a plurality of indoor video frame images stored in the storage device, and may also receive a plurality of indoor video frame images sent by other devices, for example, an indoor monitoring device. It should be noted that the plurality of indoor video frame images are obtained by shooting based on an indoor scene, and the plurality of indoor video frame images may be a plurality of images sequentially arranged in time order.
And S22, processing each indoor video frame image by adopting the three-dimensional portrait acquisition method of any one of the above embodiments to obtain a three-dimensional portrait corresponding to each indoor video frame image.
Specifically, the computer device processes each of the intra-video frame images by using the method for acquiring a three-dimensional portrait in any of the embodiments, that is, calibrates each of the intra-video frame images based on a target object with a known appearance parameter in the intra-video frame image, so as to obtain a three-dimensional portrait corresponding to each of the intra-video frame images, thereby avoiding inconvenience in operation caused by calibration using an external calibration board.
And S23, determining human body motion tracks corresponding to different human images according to the three-dimensional human images corresponding to the indoor video frame images.
Specifically, the computer device may determine the human motion trajectories corresponding to different human images according to the coordinate change rule of different portions of the three-dimensional human image corresponding to each indoor video frame image. For example, the computer device may record positions of different portions of a target portrait in a three-dimensional portrait corresponding to each frame of indoor video frame image, so as to obtain human motion trajectories of different portions of the target portrait, for example, record a position of a left hand of the target portrait, so as to obtain a human motion trajectory of the left hand of the target portrait.
In this embodiment, the computer device obtains a plurality of indoor video frame images, and processes each indoor video frame image by using the method for obtaining a three-dimensional portrait in this embodiment to obtain a three-dimensional portrait corresponding to each indoor video frame image, thereby avoiding inconvenience of carrying a calibration board. And then the human motion tracks corresponding to different human images can be determined according to the three-dimensional human images corresponding to the indoor video frame images. According to the method, the three-dimensional portrait acquisition method in the embodiment is adopted to process each indoor video frame image, and the calibration is realized based on the known appearance parameters of the target object, so that the inconvenience of carrying a calibration plate is avoided, and the behavior analysis result obtained by the video analysis method in the embodiment is more convenient.
Optionally, as shown in fig. 4, the foregoing embodiment may further include the steps of: and S24, performing behavior analysis according to the human body motion track to obtain a behavior analysis result. Specifically, the computer device may perform behavior analysis on the human motion trajectory. Alternatively, the computer device may determine the behavior analysis result corresponding to the human motion trajectory according to a corresponding relationship between the human motion trajectory and a preset behavior analysis result, for example, the human motion trajectory of the head of the person is from a front door to a back door in a room, so that the corresponding behavior analysis result is walking, and then, if the human motion trajectory of the left hand of the person is from the height of a desktop to the height of the head of the person, the corresponding behavior result may be hand-lifting, and the like. Optionally, the human motion trajectory may be input to a preset neural network model by a computer device for behavior recognition, so as to obtain a behavior analysis result, and it should be noted that the neural network model is obtained by training a sample based on a large number of human motion trajectories, so that a more accurate behavior analysis result can be obtained. According to the method, the three-dimensional portrait acquisition method in the embodiment is adopted to process each indoor video frame image, and the calibration is realized based on the known appearance parameters of the target object, so that the inconvenience of carrying a calibration plate is avoided, and the behavior analysis result obtained by the video analysis method in the embodiment is more convenient.
It should be understood that although the various steps in the flow charts of fig. 2-4 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in fig. 2-4 may include multiple sub-steps or multiple stages that are not necessarily performed at the same time, but may be performed at different times, and the order of performance of the sub-steps or stages is not necessarily sequential, but may be performed in turn or alternating with other steps or at least some of the sub-steps or stages of other steps.
In one embodiment, as shown in fig. 5, there is provided an apparatus for acquiring a three-dimensional portrait, including:
the detection module 100 is configured to perform target object detection on a two-dimensional indoor scene graph to obtain a target object image and a portrait in the indoor scene graph;
and the processing module 200 is configured to perform two-dimensional to three-dimensional coordinate conversion on the target object image according to the shape parameter of the target object in the target object image to obtain a conversion matrix, and perform two-dimensional to three-dimensional coordinate conversion on the portrait according to the conversion matrix to obtain the three-dimensional portrait.
In an embodiment, the processing module 200 is specifically configured to obtain a plurality of vertices of the target image, use the vertices as calibration points, and perform two-dimensional to three-dimensional coordinate transformation on the target image according to the shape parameters of the target to obtain the transformation matrix.
In an embodiment, the processing module 200 is specifically configured to multiply the portrait with the transformation matrix to obtain the three-dimensional portrait.
In one embodiment, the indoor scene graph is a conference room scene graph inside a conference room, and the object image is a conference table image.
In one embodiment, as shown in fig. 6, there is provided a video analysis apparatus including:
an obtaining module 300, configured to obtain a plurality of indoor video frame images;
a processing module 400, configured to process each indoor video frame image by using the method for acquiring a three-dimensional portrait according to any one of the embodiments described above, so as to obtain a three-dimensional portrait corresponding to each indoor video frame image;
and the analysis module 500 is configured to determine human motion trajectories corresponding to different human images according to the three-dimensional human image corresponding to each of the indoor video frame images.
In an embodiment, the analysis module 500 may be further configured to perform behavior analysis according to the human motion trajectory to obtain a behavior analysis result.
For specific limitations of the three-dimensional portrait acquisition apparatus and the video analysis apparatus, reference may be made to the above limitations of the three-dimensional portrait acquisition method and the video analysis method, respectively, and details are not repeated here. All or part of the modules in the three-dimensional portrait acquisition device and the video analysis device can be realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a computer device is provided, comprising a memory and a processor, the memory having a computer program stored therein, the processor implementing the following steps when executing the computer program:
carrying out target object detection on a two-dimensional indoor scene graph to obtain a target object image and a portrait in the indoor scene graph;
according to preset appearance parameters of a target object, performing two-dimensional to three-dimensional coordinate conversion on the target object image corresponding to the target object to obtain a conversion matrix;
and performing two-dimensional to three-dimensional coordinate conversion on the portrait according to the conversion matrix to obtain the three-dimensional portrait.
In one embodiment, the processor, when executing the computer program, further performs the steps of:
acquiring a plurality of vertexes of the target object image;
and taking the plurality of vertexes as calibration points, and performing two-dimensional to three-dimensional coordinate conversion on the target object image according to the appearance parameters of the target object to obtain the conversion matrix.
In one embodiment, the processor, when executing the computer program, further performs the steps of:
and multiplying the portrait by the conversion matrix to obtain the three-dimensional portrait.
In one embodiment, the indoor scene graph is a conference room scene graph inside a conference room, and the object image is a conference table image.
It should be clear that, in the embodiments of the present application, the process of executing the computer program by the processor is consistent with the process of executing the steps in the above method, and specific reference may be made to the description above.
In one embodiment, a computer device is provided, comprising a memory and a processor, the memory having a computer program stored therein, the processor implementing the following steps when executing the computer program:
acquiring a plurality of indoor video frame images;
processing each indoor video frame image by adopting the three-dimensional portrait acquisition method of any one of the embodiments to obtain a three-dimensional portrait corresponding to each indoor video frame image;
and determining human body motion tracks corresponding to different human images according to the three-dimensional human images corresponding to the indoor video frame images.
In one embodiment, the processor, when executing the computer program, further performs the steps of:
and performing behavior analysis according to the human body motion track to obtain a behavior analysis result.
It should be clear that, in the embodiments of the present application, the process of executing the computer program by the processor is consistent with the process of executing the steps in the above method, and specific reference may be made to the description above.
In one embodiment, a computer-readable storage medium is provided, having a computer program stored thereon, which when executed by a processor, performs the steps of:
carrying out target object detection on a two-dimensional indoor scene graph to obtain a target object image and a portrait in the indoor scene graph;
according to preset appearance parameters of a target object, performing two-dimensional to three-dimensional coordinate conversion on the target object image corresponding to the target object to obtain a conversion matrix;
and performing two-dimensional to three-dimensional coordinate conversion on the portrait according to the conversion matrix to obtain the three-dimensional portrait.
In one embodiment, the computer program when executed by the processor further performs the steps of:
acquiring a plurality of vertexes of the target object image;
and taking the plurality of vertexes as calibration points, and performing two-dimensional to three-dimensional coordinate conversion on the target object image according to the appearance parameters of the target object to obtain the conversion matrix.
In one embodiment, the computer program when executed by the processor further performs the steps of:
and multiplying the portrait by the conversion matrix to obtain the three-dimensional portrait.
In one embodiment, the indoor scene graph is a conference room scene graph inside a conference room, and the object image is a conference table image.
It should be clear that, in the embodiments of the present application, the process of executing the computer program by the processor is consistent with the process of executing the steps in the above method, and specific reference may be made to the description above.
In one embodiment, a computer-readable storage medium is provided, having a computer program stored thereon, which when executed by a processor, performs the steps of:
acquiring a plurality of indoor video frame images;
processing each indoor video frame image by adopting the three-dimensional portrait acquisition method of any one of the embodiments to obtain a three-dimensional portrait corresponding to each indoor video frame image;
and determining human body motion tracks corresponding to different human images according to the three-dimensional human images corresponding to the indoor video frame images.
In one embodiment, the computer program when executed by the processor further performs the steps of:
and performing behavior analysis according to the human body motion track to obtain a behavior analysis result.
It should be clear that, in the embodiments of the present application, the process of executing the computer program by the processor is consistent with the process of executing the steps in the above method, and specific reference may be made to the description above.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (10)
1. A method for acquiring a three-dimensional portrait, the method comprising:
carrying out target object detection on a two-dimensional indoor scene graph to obtain a target object image and a portrait in the indoor scene graph;
according to preset appearance parameters of a target object, performing two-dimensional to three-dimensional coordinate conversion on the target object image corresponding to the target object to obtain a conversion matrix;
and performing two-dimensional to three-dimensional coordinate conversion on the portrait according to the conversion matrix to obtain the three-dimensional portrait.
2. The method according to claim 1, wherein the converting the two-dimensional to three-dimensional coordinate of the target image according to the preset shape parameter of the target to obtain a conversion matrix comprises:
acquiring a plurality of vertexes of the target object image;
and taking the plurality of vertexes as calibration points, and performing two-dimensional to three-dimensional coordinate conversion on the target object image according to the appearance parameters of the target object to obtain the conversion matrix.
3. The method according to claim 1 or 2, wherein the converting the portrait into three-dimensional coordinates according to the conversion matrix to obtain the three-dimensional portrait comprises:
and multiplying the portrait by the conversion matrix to obtain the three-dimensional portrait.
4. The method of claim 1, wherein the indoor scene graph is a meeting room scene graph inside a meeting room, and the object image is a conference table image.
5. A method of video analysis, the method comprising:
acquiring a plurality of indoor video frame images;
processing each indoor video frame image by adopting the three-dimensional portrait acquisition method according to any one of claims 1 to 4 to obtain a three-dimensional portrait corresponding to each indoor video frame image;
and determining human body motion tracks corresponding to different human images according to the three-dimensional human images corresponding to the indoor video frame images.
6. The method of claim 5, further comprising:
and performing behavior analysis according to the human body motion track to obtain a behavior analysis result.
7. An apparatus for acquiring a three-dimensional portrait, the apparatus comprising:
the detection module is used for detecting a target object in a two-dimensional indoor scene graph to obtain a target object image and a portrait in the indoor scene graph;
and the processing module is used for performing two-dimensional to three-dimensional coordinate conversion on the target object image according to the appearance parameters of the target object in the target object image to obtain a conversion matrix, and performing two-dimensional to three-dimensional coordinate conversion on the portrait according to the conversion matrix to obtain the three-dimensional portrait.
8. A video analysis apparatus, characterized in that the apparatus comprises:
the system comprises an acquisition module, a processing module and a display module, wherein the acquisition module is used for acquiring a plurality of indoor video frame images;
a processing module, configured to process each of the indoor video frame images by using the method for acquiring a three-dimensional portrait according to any one of claims 1 to 4, so as to obtain a three-dimensional portrait corresponding to each of the indoor video frame images;
and the analysis module is used for determining human motion tracks corresponding to different human images according to the three-dimensional human images corresponding to the indoor video frame images.
9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor, when executing the computer program, implements the steps of the method of any of claims 1 to 6.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 6.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2020101016002 | 2020-02-19 | ||
| CN202010101600.2A CN111325659A (en) | 2020-02-19 | 2020-02-19 | Three-dimensional portrait acquisition method, video analysis method, device, equipment and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112837214A true CN112837214A (en) | 2021-05-25 |
Family
ID=71172693
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010101600.2A Pending CN111325659A (en) | 2020-02-19 | 2020-02-19 | Three-dimensional portrait acquisition method, video analysis method, device, equipment and medium |
| CN202110179244.0A Pending CN112837214A (en) | 2020-02-19 | 2021-02-09 | Three-dimensional portrait acquisition method, video analysis method, device, equipment and medium |
Family Applications Before (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010101600.2A Pending CN111325659A (en) | 2020-02-19 | 2020-02-19 | Three-dimensional portrait acquisition method, video analysis method, device, equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (2) | CN111325659A (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111915684A (en) * | 2020-08-13 | 2020-11-10 | 浙江大华技术股份有限公司 | Method and device for determining coordinate transformation matrix, storage medium and electronic device |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105427345A (en) * | 2015-11-30 | 2016-03-23 | 北京正安维视科技股份有限公司 | Three-dimensional people stream movement analysis method based on camera projection matrix |
| US20170210011A1 (en) * | 2016-01-22 | 2017-07-27 | The Boeing Company | Apparatus and Method to Optically Locate Workpiece for Robotic Operations |
-
2020
- 2020-02-19 CN CN202010101600.2A patent/CN111325659A/en active Pending
-
2021
- 2021-02-09 CN CN202110179244.0A patent/CN112837214A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105427345A (en) * | 2015-11-30 | 2016-03-23 | 北京正安维视科技股份有限公司 | Three-dimensional people stream movement analysis method based on camera projection matrix |
| US20170210011A1 (en) * | 2016-01-22 | 2017-07-27 | The Boeing Company | Apparatus and Method to Optically Locate Workpiece for Robotic Operations |
Non-Patent Citations (1)
| Title |
|---|
| 汪亚明;翟俊鹏;莫燕;韩永华;蒋明峰;: "基于正交匹配追踪及加速近端梯度的人体三维重建", 中国生物医学工程学报, no. 04 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111325659A (en) | 2020-06-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109901123B (en) | Sensor calibration method, device, computer equipment and storage medium | |
| CN115294275A (en) | Method and device for reconstructing three-dimensional model and computer readable storage medium | |
| CN110992243B (en) | Intervertebral disc cross-section image construction method, device, computer equipment and storage medium | |
| CN113240769A (en) | Spatial link relation identification method and device and storage medium | |
| CN110111364B (en) | Motion detection method and device, electronic equipment and storage medium | |
| CN114640833A (en) | Projection picture adjusting method and device, electronic equipment and storage medium | |
| CN109740487B (en) | Point cloud labeling method and device, computer equipment and storage medium | |
| CN111080571A (en) | Camera shielding state detection method and device, terminal and storage medium | |
| WO2020223940A1 (en) | Posture prediction method, computer device and storage medium | |
| CN110688950B (en) | Face living body detection method and device based on depth information | |
| CN113158974A (en) | Attitude estimation method, attitude estimation device, computer equipment and storage medium | |
| CN112837214A (en) | Three-dimensional portrait acquisition method, video analysis method, device, equipment and medium | |
| CN113963072A (en) | Binocular camera calibration method and device, computer equipment and storage medium | |
| CN113706583A (en) | Image processing method, image processing device, computer equipment and storage medium | |
| CN111918127B (en) | Video clipping method and device, computer readable storage medium and camera | |
| CN111968160A (en) | Image matching method and storage medium | |
| CN107958236A (en) | The generation method and terminal of recognition of face sample image | |
| JP5439343B2 (en) | 3D information acquisition apparatus and 3D information acquisition program | |
| CN108876713B (en) | Mapping method and device of two-dimensional template image, terminal equipment and storage medium | |
| CN112241984A (en) | Binocular vision sensor calibration method and device, computer equipment and storage medium | |
| CN113139893B (en) | Image translation model construction method and device and image translation method and device | |
| CN110807423B (en) | Method and device for processing fingerprint image under screen and electronic equipment | |
| CN113989377A (en) | External parameter calibration method and device for camera, storage medium and terminal equipment | |
| CN113393498A (en) | Image registration method and device, computer equipment and storage medium | |
| CN118424151B (en) | Multi-device measurement method, system, electronic device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |